Skip to content

vllm_omni.config.pipeline_registry

Pipeline registry and factory for vllm-omni.

OMNI_PIPELINES maps each model_type to either a PipelineConfig instance or a resolver callable that accepts an optional HF config and returns a PipelineConfig.

To add a new pipeline
  1. Define the PipelineConfig instance as a module-level variable in vllm_omni/.../pipeline.py.
  2. If the model needs to support several configurations, e.g., because some stages are optional, implement a resolver that consumes the HF config and returns a PipelineConfig.
  3. Update the registry to map the key to the new config object (in the case of new keys) or to the resolver func. Keep OMNI_PIPELINES sorted alphabetically by key.

Out of tree pipeline configs or resolvers can also be registered with register_pipeline.

NOTE: Generic single-stage diffusion is selected by config.resolver when no registered Omni pipeline matches.

OMNI_PIPELINES module-attribute 

OMNI_PIPELINES: dict[
    str, PipelineConfig | PipelineResolverFunc
] = {
    "arktts": AUDIO8_TTS_PIPELINE,
    "audex_s2s": AUDEX_S2S_PIPELINE,
    "audex_thinker_only": AUDEX_THINKER_ONLY_PIPELINE,
    "audex_tta": AUDEX_TTA_PIPELINE,
    "audex_tts": AUDEX_TTS_PIPELINE,
    "auk": AUK_PIPELINE,
    "aura_omni": AURA_OMNI_PIPELINE,
    "bagel": BAGEL_PIPELINE,
    "bagel_single_stage": BAGEL_SINGLE_STAGE_PIPELINE,
    "bagel_think": BAGEL_THINK_PIPELINE,
    "breeze": BREEZE_TTS_2_PIPELINE,
    "cosmos3_omni_deploy": COSMOS3_OMNI_DEPLOY_PIPELINE,
    "cosmos3_policy": COSMOS3_POLICY_PIPELINE,
    "cosyvoice3": COSYVOICE3_PIPELINE,
    "covo_audio": COVO_AUDIO_PIPELINE,
    "dreamzero": DREAMZERO_PIPELINE,
    "fish_qwen3_omni": FISH_SPEECH_PIPELINE,
    "gepard": GEPARD_PIPELINE,
    "glm_image": GLM_IMAGE_PIPELINE,
    "glm_tts": GLM_TTS_PIPELINE,
    "Gr00tN1d7": GR00T_N1D7_PIPELINE,
    "higgs_audio_v2": HIGGS_AUDIO_V2_PIPELINE,
    "higgs_multimodal_qwen3": HIGGS_AUDIO_V3_PIPELINE,
    "hunyuan_image3_ar": HUNYUAN_IMAGE3_AR_PIPELINE,
    "hunyuan_image3_dit": HUNYUAN_IMAGE3_DIT_PIPELINE,
    "hunyuan_image_3_moe": HUNYUAN_IMAGE3_PIPELINE,
    "hunyuan_video_15": HUNYUAN_VIDEO_15_PIPELINE,
    "indextts2": INDEXTTS2_PIPELINE,
    "indextts2_5": INDEXTTS25_PIPELINE,
    "joyai_vl_interaction": JOYAI_VL_INTERACTION_PIPELINE,
    "lance": LANCE_PIPELINE,
    "lingbot_world": LINGBOT_WORLD_PIPELINE,
    "mammoth_moda2": MAMMOTH_MODA2_PIPELINE,
    "mammoth_moda2_ar": MAMMOTH_MODA2_AR_PIPELINE,
    "mimo_audio": MIMO_AUDIO_PIPELINE,
    "ming_flash_omni": MING_FLASH_OMNI_PIPELINE,
    "ming_flash_omni_image": MING_FLASH_OMNI_IMAGE_PIPELINE,
    "ming_flash_omni_thinker_only": MING_FLASH_OMNI_THINKER_ONLY_PIPELINE,
    "ming_flash_omni_tts": MING_FLASH_OMNI_TTS_PIPELINE,
    "ming_image": MING_IMAGE_PIPELINE,
    "ming_tts": MING_TTS_PIPELINE,
    "ming_tts_moe": MING_TTS_MOE_PIPELINE,
    "minicpmo_4_5": MINICPMO_4_5_PIPELINE,
    "minimax_h3_disaggregated": MINIMAX_H3_PIPELINE,
    "minimax_music3": MINIMAX_MUSIC3_PIPELINE,
    "moss_tts_delay": MOSS_TTS_PIPELINE,
    "moss_tts_local": MOSS_TTS_LOCAL_PIPELINE,
    "moss_tts_nano": MOSS_TTS_NANO_PIPELINE,
    "moss_tts_realtime": MOSS_TTS_REALTIME_PIPELINE,
    "nemotron_labs_audex": AUDEX_TTS_PIPELINE,
    "nemotron_labs_voicechat": NEMOTRON_VOICECHAT_PIPELINE,
    "nemotron_voicechat": NEMOTRON_VOICECHAT_PIPELINE,
    "omnivoice": OMNIVOICE_PIPELINE,
    "personaplex": PERSONAPLEX_PIPELINE,
    "pi0": PI0_PIPELINE,
    "pi05": PI05_PIPELINE,
    "qwen2_5_omni": QWEN2_5_OMNI_PIPELINE,
    "qwen2_5_omni_thinker_only": QWEN2_5_OMNI_THINKER_ONLY_PIPELINE,
    "qwen3_omni_moe": resolve_qwen3_omni_pipeline,
    "qwen3_omni_moe_thinker_only": QWEN3_OMNI_THINKER_ONLY_PIPELINE,
    "qwen3_tts": QWEN3_TTS_PIPELINE,
    "step_audio_2": STEP_AUDIO2_PIPELINE,
    "step_audio_2_asr": STEP_AUDIO2_ASR_PIPELINE,
    "voxcpm2": VOXCPM2_PIPELINE,
    "voxtral_tts": VOXTRAL_TTS_PIPELINE,
    "wan2_2_ti2v": WAN2_2_TI2V_PIPELINE,
}

PipelineResolverFunc module-attribute 

PipelineResolverFunc: TypeAlias = Callable[
    [PretrainedConfig | None], PipelineConfig | None
]

logger module-attribute 

logger = init_logger(__name__)

register_pipeline 

register_pipeline(
    pipeline: PipelineConfig | PipelineResolverFunc,
    model_type: str | None = None,
)

Register an out of tree pipeline or PipelineResolverFunc to a model_type key. If a PipelineConfig is provided, model_type is optional, and pipeline.model_type will be used by default. If a callable is provided, model_type must be provided, since resolvers can return multiple different PipelineConfigs depending on the consumed config.

resolve_pipeline_config 

resolve_pipeline_config(
    model_type: str,
    hf_config: PretrainedConfig | None = None,
) -> PipelineConfig | None

Resolve a registry key to a concrete pipeline config.