Entrypoints and serving boundaries¶
Entrypoints translate offline, CLI, and serving requests into stable engine operations and translate engine outputs into public responses.
AsyncOmni explicitly rejects the process checkpoint methods exposed by newer vLLM EngineClient versions (checkpoint_prepare, checkpoint_restore, and checkpoint_state). This keeps ordinary serving instantiable without claiming checkpoint support. All three raise NotImplementedError before accessing or changing stage state; sleep and weight-transfer APIs retain their own contracts.
Contract status¶
This document describes current entrypoint responsibilities plus the boundary under review in the open roadmap #5227 and helper-move PR #5453. In-flight helper locations are not treated as current paths.
Ownership boundary¶
This document owns offline API semantics, CLI and serve composition, supported OpenAI-compatible routes, request validation and normalization, response conversion, streaming/session behavior, and engine handoff.
The serving composition removes inherited /tokenize, /detokenize, and optional /tokenizer_info routes when no tokenization handler is configured (including pure diffusion mode). These endpoints return HTTP 404 and are absent from OpenAPI in that mode; configured handlers retain upstream vLLM routing.
An optional --video-output-warmup request runs through the streaming video handler before HTTP serving starts. The frontend owns request validation, draining generation and encoding, and the startup deadline. It does not expose /health or admit traffic until that request completes successfully. Failure aborts the internal request and unwinds engine startup. The request explicitly supplies model-specific inputs and a finite frame count; playback pacing is rejected because there is no player. Omitting the option skips this warmup.
When --api-server-count is greater than one, the serve composition root starts the frontend process manager while a parent-owned StageRuntime launches the shared local stage engines. Each frontend receives only its own stage-client channel configuration and does not launch or retire backend processes. CLI validation rejects unsupported distributed, diffusion, fault-tolerant, elastic-EP, Ray, and runtime-LoRA combinations before startup.
It does not own configuration precedence, cross-stage routing, stage placement, payload implementation, or semantic error classification. entrypoints/openai/errors.py is an explicit primary-path exception owned by error_contracts.md.
It also owns vllm_omni/protocol/**, the shared wire codec promoted by RFC #6592 P0a. It follows the three tiers docs/serving/realtime_duplex_api.md already defines. protocol/realtime/** is Tier 1, identical to OpenAI: 30 server events, 10 client commands, their typed fields and pure wire rendering, audio-format negotiation, conversation-item rules and the error-envelope shape. protocol/duplex/** is Tier 2 (OpenAI names carrying vLLM-Omni extensions, each a subclass of its Tier 1 twin declaring only what it adds) and Tier 3 (ours alone, including our error-code vocabulary), kept separate so a GA-only consumer is not handed a vocabulary its clients never send. That package sits outside entrypoints/ on purpose --- the engine depends on it, so it belongs to neither layer --- but deciding what a Realtime client may put on the wire is this document's subject, and the same reasoning already places vllm_omni/errors.py, inputs/ and outputs/ under contract documents rather than under a consumer.
It is deliberately not owned by ../fullduplex.md, even though duplex is the codec's only consumer today. RFC #6592 asks for a named owner precisely so a second Realtime consumer does not need duplex review to change the codec (#6592 open question 8). Ownership here is review scope, not a dependency: the codec may not import the engine, the entrypoints or the model code, which tests/protocol/realtime/test_protocol_import_boundary.py enforces. When a second consumer lands (#6592 P0b) the package is expected to graduate to its own module document with owners drawn from both consumers.
Candidate invariants¶
These identifiers are proposals while the document is draft.
ENTRY-INV-001: Entrypoints adapt but do not orchestrate¶
Rule: Entrypoints MUST NOT implement cross-stage routing or stage lifecycle policy.
ENTRY-INV-002: The shared Realtime codec does not depend on its consumers¶
Rule: vllm_omni/protocol/** MUST NOT import vllm_omni.engine, vllm_omni.entrypoints, vllm_omni.model_executor, vllm_omni.worker or vllm_omni.clients; protocol/duplex/** MAY depend on protocol/realtime/** but never the reverse, and a Tier 1 class MUST NOT carry a vLLM-Omni extension field; and each protocol/codec behavior MUST have exactly one implementation that every consumer uses. A wire type MUST NOT carry a consumer's internal representation: the duplex mailbox rendering (DuplexCommand.payload(), whose channel differs from the client event for session.update and the conversation.item.* commands) stays engine-side. A duplex consumer (engine/duplex/**, entrypoints/duplex/**, the duplex clients) MUST import vllm_omni.protocol.duplex and MUST NOT import vllm_omni.protocol.realtime directly, so the tier boundary has exactly one extension point.
ENTRY-INV-100: Public requests are normalized once¶
Rule: Public protocol values MUST be validated and converted to internal request contracts before engine submission.
ENTRY-INV-101: Streaming preserves request identity¶
Rule: Every streamed response MUST remain associated with the request and output modality that produced it.
The reviewer-proposed rule that model-specific behavior should stay behind a common adapter or processor abstraction remains an unnumbered candidate until the entrypoint refactor makes that boundary enforceable.
Invariant namespace¶
ENTRY-INV reserves 001-099 for boundary and dependency direction, 100-199 for normalization, sessions, and streaming, 200-299 for disconnect, cancellation, rendering, and cleanup, and 300-399 for upstream route and renderer compatibility. Numbers become append-only after normative promotion.
Safe-change guide¶
Test request validation, protocol conversion, model-adapter routing, streaming, session identity, disconnect/cancellation, and error mapping for each affected offline or serving entrypoint. Sleep must wait for in-flight generate() admission before EngineCore offload; wake_up does not resume admission — callers must resume_generation(). Sleeping tags are tracked per stage so wake_up(stage_ids=[0]) does not skip a later wake_up(stage_ids=[1]). Streaming input pumps take an admission slot immediately before each EngineCore ADD or update, not while waiting for the next client chunk. Frontend abort keeps request_states until generate() consumes the terminal output. Video output sessions shield final task cleanup and unfinished-request abort from ASGI cancellation, including disconnects immediately after session.done. Video encoding and encoder close run on one worker per stream so encoding does not block control handling. Cancellation queues close after any running encode; the event loop does not join that worker while it drains cleanup. Abort cleanup bounds both its coroutine and the engine acknowledgement wait with ABORT_TIMEOUT_S; an unresponsive orchestrator must not hold the shield indefinitely. Multi-API serving must preserve one client rank and one channel set per frontend, include every frontend in readiness/failure observation, and keep stage lifecycle ownership in the parent composition root.
Promotion gate¶
- Reconcile the page after #5227's P0 ownership work and #5453's final disposition.
- Publish a supported route/transport matrix rather than claiming blanket OpenAI parity.
- Verify one normalization and handoff path per route family and one terminal outcome per streaming transport.
- Promote the
api_server.pycomposition-root rule only after helper moves and import boundaries enforce it. - Obtain approval from a technical owner and an independent validation reviewer.