Skip to content

vllm_omni.entrypoints.utils

PROJECT_ROOT module-attribute 

PROJECT_ROOT = parent

logger module-attribute 

logger = init_logger(__name__)

coerce_param_message_types 

coerce_param_message_types(
    params: list[OmniSamplingParams], is_streaming: bool
)

Iterate over the sampling params and convert to the message types to DELTA messages, if streaming is enabled, or FINAL_ONLY if it's disabled, while respecting .skip_clone on the params.

This is needed to avoid emitting redundant multimodal data.

detect_pid_host 

detect_pid_host() -> bool

filter_dataclass_kwargs 

filter_dataclass_kwargs(cls: Any, kwargs: dict) -> dict

Filter kwargs to only include fields defined in the dataclass.

Parameters:

Name Type Description Default
cls Any

Dataclass type

 required
kwargs dict

Keyword arguments to filter

 required

Returns:

Type Description
dict

Filtered keyword arguments containing only valid dataclass fields

filter_stages 

filter_stages(
    config_path: str | None,
    stage_configs: list,
    kwargs: dict | None,
) -> list

Filter stage configs by mode when YAML defines a modes section.

The YAML can define, e.g.:

modes:
  - mode: text-to-image
    stages: [1]
  - mode: image-to-text
    stages: [0]

When users pass mode="image-to-text" into Omni(**kwargs), only the stages listed for that mode are returned. If no mode is provided, defaults to "text-to-image". If no modes are defined or filtering fails, returns the original stage_configs unchanged.

Parameters:

Name Type Description Default
config_path str | None

Path to the YAML config (used to read modes).

 required
stage_configs list

Loaded list of stage configs.

 required
kwargs dict | None

Engine/caller kwargs; may contain "mode".

 required

Returns:

Type Description
list

Filtered list of stage configs (or original list if filtering not applied).

get_final_stage_id_for_e2e 

get_final_stage_id_for_e2e(
    output_modalities: list[str] | None,
    default_modalities: list[str],
    stage_list: list,
) -> int

Get the final stage id for e2e.

Parameters:

Name Type Description Default
stage_list list

List of stage configurations

 required

Returns:

Type Description
int

Final stage id for e2e

has_pid_host 

has_pid_host() -> bool | None

Returns:

Type Description
bool | None

True -> very likely running with --pid=host (host PID namespace)
bool | None

False -> very likely isolated PID namespace (default)
bool | None

None -> cannot determine

in_container 

in_container() -> bool

inject_omni_kv_config 

inject_omni_kv_config(
    stage: Any,
    omni_conn_cfg: dict[str, Any],
    omni_from: str,
    omni_to: str,
) -> None

Inject connector configuration into stage engine arguments.

load_and_resolve_stage_configs 

load_and_resolve_stage_configs(
    model: str,
    stage_configs_path: str | None,
    kwargs: dict | None,
    default_stage_cfg_factory: Any = None,
    deploy_config_path: str | None = None,
    stage_overrides: dict[str, dict[str, Any]]
    | None = None,
) -> tuple[str, list]

Load stage configurations from model or YAML file with fallback to defaults.

Parameters:

Name Type Description Default
model str

Model name or path

 required
stage_configs_path str | None

Optional path to legacy YAML (stage_args format)

 required
kwargs dict | None

Engine arguments to merge with stage configs

 required
default_stage_cfg_factory Any

Optional callable that takes no args and returns default stage config list when no configs are found

 None
deploy_config_path str | None

Optional path to deploy YAML (new format). Mutually exclusive with stage_configs_path.

 None
stage_overrides dict[str, dict[str, Any]] | None

Per-stage overrides from --stage-overrides JSON. Keys are stage_id strings, values are dicts of overrides.

 None

Returns:

Type Description
tuple[str, list]

Tuple of (config_path, stage_configs)

load_stage_configs_from_model 

load_stage_configs_from_model(
    model: str,
    base_engine_args: dict | None = None,
    deploy_config_path: str | None = None,
    stage_overrides: dict[str, dict[str, Any]]
    | None = None,
) -> list

Load stage configurations from model's default config file.

For models registered in the pipeline registry (new path), uses StageConfigFactory.create_from_model() which merges PipelineConfig + DeployConfig + CLI overrides.

For other models (legacy path), loads stage configs from YAML.

Parameters:

Name Type Description Default
model str

Model name or path (used to determine model_type)

 required
base_engine_args dict | None

Base engine args to merge as CLI overrides.

 None
deploy_config_path str | None

Optional explicit deploy config path.

 None
stage_overrides dict[str, dict[str, Any]] | None

Per-stage overrides from --stage-overrides.

 None

Returns:

Type Description
list

List of stage configuration dictionaries

load_stage_configs_from_yaml 

load_stage_configs_from_yaml(
    config_path: str,
    base_engine_args: dict | None = None,
    prefer_stage_engine_args: bool = True,
) -> list

Load stage configurations from a YAML file (legacy OmegaConf path).

TODO(@lishunyang12): remove once all models use PipelineConfig + DeployConfig.

Parameters:

Name Type Description Default
config_path str

Path to the YAML configuration file

 required
base_engine_args dict | None

Engine args supplied by the caller.

 None
prefer_stage_engine_args bool

When True, YAML stage args override caller engine args. When False, caller engine args override YAML defaults.

 True

Returns:

Type Description
list

List of stage configuration dictionaries from the file's stage_args

maybe_coerce_to_message_type 

maybe_coerce_to_message_type(
    params: SamplingParams, is_streaming: bool
)

If this is a CUMULATIVE message, coerce it to DELTA if streaming, otherwise FINAL_ONLY.

resolve_model_config_path 

resolve_model_config_path(model: str) -> str

Resolve the stage config file path from the model name.

Resolves stage configuration path based on the model type and device type. First tries to find a device-specific YAML file from stage_configs/{device_type}/ directory. If not found, falls back to the default config file.

Parameters:

Name Type Description Default
model str

Model name or path (used to determine model_type)

 required

Returns:

Type Description
str

String path to the stage configuration file

Raises:

Type Description
ValueError

If model_type cannot be determined
FileNotFoundError

If no stage config file exists for the model type