Skip to content

vllm_omni.engine.rpc_result_router ¶

RpcCorrelationKey `module-attribute` ¶

RpcCorrelationKey: TypeAlias = tuple[str, str]

RpcWaiter `module-attribute` ¶

RpcWaiter: TypeAlias = queue.Queue[EngineQueueMessage]

logger `module-attribute` ¶

logger = init_logger(__name__)

CorrelatedRpcClient ¶

Own request submission and correlated result waiting as one lifecycle.

close ¶

close() -> None

execute ¶

execute(
    key: RpcCorrelationKey,
    message: EngineQueueMessage,
    *,
    timeout: float | None,
    timeout_message: str,
    block_on_submit: bool = False,
) -> EngineQueueMessage

RpcResultRouter ¶

Single consumer that dispatches shared RPC results by correlation ID.

close ¶

close() -> None

register ¶

register(key: RpcCorrelationKey) -> RpcWaiter

unregister ¶

unregister(
    key: RpcCorrelationKey, waiter: RpcWaiter
) -> None