Skip to content

Model integration

Model integration adapts model-specific configuration, preprocessing, loading, and execution behavior to stable runtime contracts.

Candidate invariants

MODEL-INV-001: Registration is explicit

Rule: A model integration MUST declare how its model class, loader, input processor, and pipeline configuration are selected.

MODEL-INV-002: Model code does not route stages

Rule: Model-specific code MUST NOT select or invoke a downstream omni stage.

MODEL-INV-003: Upstream behavior is reused deliberately

Rule: An upstream vLLM implementation SHOULD be reused when its contract is sufficient; an override MUST document the behavioral difference.

Safe-change guide

Test registration, checkpoint loading, input conversion, and representative model execution. Test shared utilities against more than one integration.

Platform-specific model adapters

Drama 3's Ascend attention compatibility, hybrid model state, and FP8 expert adapter live under platforms/npu/models/ and platforms/npu/quant/. Shared model code selects these adapters lazily on NPU; CUDA keeps the upstream implementations. Generic eager kernels and request-history snapshots remain in the shared model implementation.