Skip to content

vllm_omni.engine.orchestrator_monitor

Optional orchestrator window monitor (diagnostic only).

Enable with

--enable-orch-monitor

Optional output path override

export VLLM_OMNI_ORCH_MONITOR_PATH=/path/to/monitor.json

Each 1s window records
  • duration_s, loop_idle, loop_active (orchestrator poll-loop counts)
  • per-replica outputs_queue_size (MP client outputs_queue backlog)
  • per-replica inflight (requests currently bound/routed to the replica)

See docs/contributing/profiling.md (Orchestrator Monitor).

ReplicaSample module-attribute 

ReplicaSample = tuple[int, int]

ReplicaSampler module-attribute 

ReplicaSampler = Callable[[], dict[str, ReplicaSample]]

logger module-attribute 

logger = init_logger(__name__)

OrchestratorMonitor

flush 

flush() -> None

note_loop 

note_loop(*, idle: bool) -> None

register_replica 

register_replica(stage_id: int, replica_id: int) -> None

OrchestratorMonitorBase

Bases: Protocol

flush 

flush() -> None

note_loop 

note_loop(*, idle: bool) -> None

register_replica 

register_replica(stage_id: int, replica_id: int) -> None

create_orch_monitor 

create_orch_monitor(
    *, enabled: bool, replica_sampler: ReplicaSampler
) -> OrchestratorMonitorBase

replica_key 

replica_key(stage_id: int, replica_id: int) -> str