vllm_omni ¶

Bases: ModelConfig

Configuration for Omni models, extending the base ModelConfig.

This configuration class extends the base vLLM ModelConfig with omni-specific fields for multi-stage pipeline processing.

Attributes: hf_config: The model's HF Transformers config (default: None) hf_text_config: The sub text_config of the model's hf_config (default: None) stage_id: Identifier for the stage in a multi-stage pipeline (default: 0) async_chunk: If set to True, perform async chunk session_mode: Request lifecycle mode, either turn-based or duplex model_stage: Stage type identifier, e.g., "thinker" or "talker" (default: "thinker") model_arch: Model architecture name (default: "Qwen2_5OmniForConditionalGeneration") worker_type: Model Type, e.g., "ar" or "generation" engine_output_type: Optional output type specification for the engine. Used to route outputs to appropriate processors (e.g., "image", "audio", "latents"). If None, output type is inferred. stage_connector_config: Stage connector configuration dictionary. Contains "name" (connector name), "extra" (extra connector config). task_type: Model-defined startup task type. Each model validates its supported values and applies the corresponding behavior.

The correct way to initialize this class is via vLLM config, as most of the logic for handling values is in the ModelConfig's post_init.

Example: >>> config = OmniModelConfig.from_vllm_model_config( ... vllm_config, ... stage_id=0, ... model_stage="thinker", ... model_arch="Qwen2_5OmniForConditionalGeneration" ... )

active_stream_window: int = 0

architectures: list[str]

async_chunk: bool = False

codec_frame_rate_hz: float | None = None

custom_process_next_stage_input_func: str | None = None

duplex_max_sessions: int = 1

embedding_size

enable_sleep_mode: bool = False

engine_output_type: str | None = None

final_output: bool = False

has_sampling_extra_args: bool = False

hf_config_name: str | None = None

model_arch: str | None = None

model_stage: str = 'thinker'

omni_kv_config: dict | None = None

pooling_output_decoder: str | None = None

registry

requires_full_payload_input: bool = False

retains_state_across_chunks: bool = False

sampling_extra_args_keys: tuple[str, ...] = ()

session_mode: str = 'turn'

silence_ban_frames: int = 0

stage_connector_config: dict[str, Any] = field(
    default_factory=lambda: {
        "name": "SharedMemoryConnector",
        "extra": {},
    }
)

stage_id: int = 0

subtalker_sampling_params: dict[str, Any] | None = None

supports_native_mrv2_data_plane: bool = False

task_type: str | None = None

use_v2_model_runner: bool = False

uses_mrope: bool

worker_type: str | None = None

add_defaults_to_omni_kwargs(omni_kwargs)

draw_hf_text_config()

from_vllm_model_config(
    model_config: ModelConfig, **omni_kwargs
)

get_inputs_embeds_size() -> int

get_model_arch_config()