Skip to content

Entrypoints and serving boundaries

Entrypoints translate offline, CLI, and serving requests into stable engine operations and translate engine outputs into public responses.

Contract status

This document describes current entrypoint responsibilities plus the boundary under review in the open roadmap #5227 and helper-move PR #5453. In-flight helper locations are not treated as current paths.

Ownership boundary

This document owns offline API semantics, CLI and serve composition, supported OpenAI-compatible routes, request validation and normalization, response conversion, streaming/session behavior, and engine handoff.

It does not own configuration precedence, cross-stage routing, stage placement, payload implementation, or semantic error classification. entrypoints/openai/errors.py is an explicit primary-path exception owned by error_contracts.md.

Candidate invariants

These identifiers are proposals while the document is draft.

ENTRY-INV-001: Entrypoints adapt but do not orchestrate

Rule: Entrypoints MUST NOT implement cross-stage routing or stage lifecycle policy.

ENTRY-INV-100: Public requests are normalized once

Rule: Public protocol values MUST be validated and converted to internal request contracts before engine submission.

ENTRY-INV-101: Streaming preserves request identity

Rule: Every streamed response MUST remain associated with the request and output modality that produced it.

The reviewer-proposed rule that model-specific behavior should stay behind a common adapter or processor abstraction remains an unnumbered candidate until the entrypoint refactor makes that boundary enforceable.

Invariant namespace

ENTRY-INV reserves 001-099 for boundary and dependency direction, 100-199 for normalization, sessions, and streaming, 200-299 for disconnect, cancellation, rendering, and cleanup, and 300-399 for upstream route and renderer compatibility. Numbers become append-only after normative promotion.

Safe-change guide

Test request validation, protocol conversion, model-adapter routing, streaming, session identity, disconnect/cancellation, and error mapping for each affected offline or serving entrypoint. Sleep must wait for in-flight generate() admission before EngineCore offload; wake_up does not resume admission — callers must resume_generation(). Sleeping tags are tracked per stage so wake_up(stage_ids=[0]) does not skip a later wake_up(stage_ids=[1]). Streaming input pumps take an admission slot immediately before each EngineCore ADD or update, not while waiting for the next client chunk. Frontend abort keeps request_states until generate() consumes the terminal output.

Promotion gate

  • Reconcile the page after #5227's P0 ownership work and #5453's final disposition.
  • Publish a supported route/transport matrix rather than claiming blanket OpenAI parity.
  • Verify one normalization and handoff path per route family and one terminal outcome per streaming transport.
  • Promote the api_server.py composition-root rule only after helper moves and import boundaries enforce it.
  • Obtain approval from a technical owner and an independent validation reviewer.