Ming-flash-omni 2.0¶

Source https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/ming_flash_omni.

Ming-flash-omni-2.0 is an omni-modal model supporting text, image, video, and audio understanding, with text and speech outputs.

vLLM-Omni supports three deployment modes:

For standalone TTS (talker only), see the Ming-flash-omni-TTS section in the Text-To-Speech hub.

Setup¶

Please refer to the stage configuration documentation to configure memory allocation appropriately for your hardware setup.

When no --deploy-config is passed, the model registry auto-loads the full thinker+talker vllm_omni/deploy/ming_flash_omni.yaml (See Omni-Speech).

For text-only output without spinning up the talker, pass:

--deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml

Run examples¶

The end-to-end script defaults to built-in assets; pass --image-path, --audio-path, or --video-path to override.

Multi-Modality Understanding (Standalone Thinker)¶

Here we pass thinker-only deploy yaml:

python examples/offline_inference/ming_flash_omni/end2end.py --deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml --query-type text
python examples/offline_inference/ming_flash_omni/end2end.py --deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml --query-type use_image
python examples/offline_inference/ming_flash_omni/end2end.py --deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml --query-type use_audio
python examples/offline_inference/ming_flash_omni/end2end.py --deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml --query-type use_video --num-frames 16

Reasoning (Thinking Mode)¶

Reasoning ("detailed thinking on") is applied by the script when --query-type reasoning is set. The default prompt matches Ming's cookbook and expects the reference figure from the upstream repo — see get_reasoning_query in end2end.py.

python examples/offline_inference/ming_flash_omni/end2end.py \
    --deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml \
    --query-type reasoning \
    --image-path ./3_0.png

Omni-Speech (Thinker + Talker)¶

The default deploy YAML already runs thinker+talker, so spoken output only requires requesting audio (or text,audio) modalities. The thinker processes your multimodal input, generates text, then the talker synthesises the response as speech.

Audio-only output (speech response, no text):

python examples/offline_inference/ming_flash_omni/end2end.py \
    --query-type text \
    --modalities audio \
    --output-dir output_ming_omni_speech

Both text and audio output:

python examples/offline_inference/ming_flash_omni/end2end.py \
    --query-type use_audio \
    --modalities text,audio \
    --output-dir output_ming_omni_speech

Generated .wav files are saved to --output-dir (default output_ming), one per request.

The default deploy YAML allocates thinker on GPUs 0–3 and talker on GPU 3 for a common device topology (4 rather than 5 devices that talker on its own device). Adjust devices in a copied YAML and pass it via --deploy-config to match your hardware or requirements.

Modality control¶

Pass --deploy-config /path/to/your_deploy.yaml to any of the commands above to override the bundled deploy config.

Image generation (text-to-image / img2img)¶

The diffusion-stage image-generation path is currently only wired through the online OpenAI-compatible chat endpoint (/v1/chat/completions with "modalities": ["image"]). For payloads, optional knobs (steps/cfg/seed/byte5_text/negative_prompt), and the img2img reference-image flow, see the image-gen section in the online-serving README. end2end.py does not yet exercise the imagegen stage; that is tracked as follow-up work.

Online serving¶

For online serving via the OpenAI-compatible API, see examples/online_serving/ming_flash_omni/README.md.

Example materials¶