Skip to content

vllm_omni.entrypoints.async_omni_base

AsyncOmniBase: the asyncio-side foundation shared by AsyncOmni and DuplexOmni.

Owns the engine output pump (one consumer of the engine output queue that routes messages to per-request queues, ACK resolvers, or a subclass hook), engine-dead fan-out, health/shutdown, and the config accessors. Turn-based request handling lives in AsyncOmni; duplex sessions in DuplexOmni.

ABORT_TIMEOUT_S module-attribute 

ABORT_TIMEOUT_S = float(
    os.environ.get("VLLM_OMNI_ABORT_TIMEOUT", 2.0)
)

logger module-attribute 

logger = init_logger(__name__)

AsyncEventResolver

A generic signal aggregator designed for synchronized handshakes in distributed or multi-stage environments. Supports waiting for a specified number (expected_count) of worker signals in both inline and multiprocess modes.

orchestrator instance-attribute 

orchestrator = orchestrator

resolve async 

resolve(ack: OmniACK)

watch_task 

watch_task(task_id: str, expected_count: int = 1) -> Future

AsyncOmniBase

Bases: OmniBase

Shared asyncio foundation of the async entrypoints (see module docstring).

config_path instance-attribute 

config_path = self.engine.config_path

dead_error property 

dead_error: BaseException

EngineClient abstract property implementation.

endpoint_restrictions instance-attribute 

endpoint_restrictions = self.engine.endpoint_restrictions

event_resolver instance-attribute 

event_resolver = AsyncEventResolver(orchestrator=self)

final_output_task instance-attribute 

final_output_task: Task | None = None

input_processor instance-attribute 

input_processor = self.engine.input_processor

io_processor instance-attribute 

io_processor = None

is_running property 

is_running: bool

Check if the engine is running.

is_stopped property 

is_stopped: bool

EngineClient abstract property implementation.

model_config property 

model_config

Return the model config for the comprehension stage when present.

renderer property 

renderer

Return the renderer from the engine input processor when available.

vllm_config property 

vllm_config

Return the vLLM config for the comprehension stage when present.

check_health async 

check_health() -> None

Check engine health by verifying the Orchestrator process is alive.

get_diffusion_od_config 

get_diffusion_od_config() -> Any | None

Return the diffusion-stage config when the pipeline has one.

get_vllm_config async 

get_vllm_config() -> Any

Compatibility helper for call sites expecting async vllm config access.

shutdown 

shutdown(timeout: float | None = None) -> None

Shutdown the engine.