Features¶
Use this section to choose and configure vLLM-Omni runtime and optimization features. The navigation follows the active Feature Design taxonomy whenever an implementation contract has a user-facing workflow.
User guides, design documents, and recipes
User guides explain how to enable a feature and where it is supported. Design documents define internal contracts and do not, by themselves, imply general or production support. For model-specific launch flags and hardware requirements, use the linked examples and validated recipes from Supported Models.
Runtime and Stage Execution¶
| Goal | User guide | Related design contract |
|---|---|---|
| Choose serial, batched, step-wise, or streaming diffusion execution | Execution Modes and Streaming | Diffusion Continuous Batching, Async Diffusion Output |
| Reclaim stage memory without restarting the server | Sleep Mode | Runtime lifecycle behavior is documented in the user guide |
The Prefill-Decode Disaggregation design reference describes the experimental Qwen3-Omni runtime topology and its current integration boundaries. It is not yet a supported launch recipe.
Some runtime designs are deliberately not promoted as standalone User Guide features yet:
- Generic, OmniConnector-based Disaggregated Inference is not yet a standalone User Guide feature. Its OmniConnector implementations describe transport contracts, while practical connector configuration remains under Pipeline and deploy configurations.
- Async Chunk and Async Omni Output Materialization are model- and pipeline-dependent. Use the selected model's deploy configuration and recipe for the supported settings.
- Automatic Prefix Caching remains a design-level contract until its user-facing configuration and compatibility surface is consolidated.
Quantization¶
Quantization is a cross-model feature rather than a diffusion-only optimization. The unified quantization_config guide covers diffusion-only models, multi-stage omni/TTS models, and multi-stage diffusion models. Its design contract defines the shared configuration and backend extension points.
Diffusion Acceleration¶
| Goal | User guide | Related design contract |
|---|---|---|
| Compare acceleration methods and supported combinations | Overview, Feature Compatibility | Diffusion acceleration designs |
| Move weights between host and device memory | CPU Offloading | CPU Offloading |
| Reuse denoising computation | Cache Acceleration | Cache-DiT, TeaCache |
| Distribute diffusion work across devices | Parallelism | Parallelism designs |
| Select dense, sparse, or quantized attention paths | Attention Backends | Attention Backend Selection |
| Compile repeated diffusion regions | Regional Compilation | User-facing optimization guide |
| Add generated video frames | Frame Interpolation | User-facing extension guide |
| Reduce diffusion model startup time | Startup and Loading | User-facing loading guide |
| Apply diffusion adapters | LoRA | User-facing extension guide |
The Pipeline Parallelism guide remains available by direct link, but it is not promoted in the primary User Guide navigation until its user-facing support and placement are settled.
Experimental¶
Session State Manager is opt-in and experimental. Its APIs and compatibility may change without notice.