Environment Variables¶

Environment variables are process-level configuration shared by the CLI, deploy YAML, Python APIs, worker processes, and model integrations. They are therefore documented under User Guide → Configuration, rather than under CLI Reference.

This page describes the public vLLM-Omni environment-variable contract on the latest documentation branch. Use the documentation for your installed vLLM-Omni release when behavior differs.

Naming and implementation boundary¶

New public variables owned by vLLM-Omni must use the VLLM_OMNI_ prefix. Existing public names with older prefixes remain supported for compatibility, but they do not establish naming precedent for new settings. Inherited vLLM and third-party names retain the prefix chosen by their owning project.

The source environment-variable inventory records ownership, migration disposition, and redaction metadata. Unlike vllm.envs, it is not an executable value resolver: existing consumers still own parsing, defaults, precedence, and evaluation time. The public tables below describe those live consumer contracts. Stable model or stage behavior should move to typed configuration, and request-varying behavior should move to a request schema instead of adding another environment variable.

Precedence and evaluation time¶

There is no repository-wide rule that makes environment variables override (or lose to) every CLI, YAML, or request option. The tables below state precedence for each public setting. In general:

• request fields should control behavior that can vary per request;
• typed deploy or stage configuration should control stable model and stage behavior;
• environment variables are appropriate for process launch, platform integration, emergency diagnostics, and compatibility fallbacks.

An environment variable can be read while a module is imported, when server or model configuration is constructed, when a worker is spawned, or whenever a property is accessed. Set import- and startup-time variables before invoking vllm serve or importing vLLM-Omni. Changing the parent shell after workers start does not update existing worker environments.

The lifecycle labels used below are:

• Stable — reviewed public configuration; changes require compatibility consideration.
• Experimental — public for evaluation, but its contract may still evolve.
• Diagnostic — supported for investigation or temporary mitigation, not as a normal tuning interface.
• Deprecated — compatibility alias; migrate to the named replacement.

Public vLLM-Omni variables¶

Diffusion backend and cache selection¶

Backend names for DIFFUSION_ATTENTION_BACKEND are the members of DiffusionAttentionBackendEnum, such as FLASH_ATTN, TORCH_SDPA, SAGE_ATTN, FLASHINFER_ATTN, and TRTLLM_ATTN. Platform support still depends on the installed kernels and model path.

Serving and runtime¶

Server storage¶

Storage names use Pydantic's nested-settings delimiter (__). They are read when ServerSettings is initialized, normally during server import/startup. Invalid integer values raise a Pydantic validation error.

The deprecated aliases emit DeprecationWarning. They do not affect file TTL settings.

Quantization diagnostics and performance¶

QUACK_CACHE_DIR is owned by the external quack library and is not an Omni-owned variable, even though vLLM-Omni supplies a persistent default when it is unset.

Per-stage environment¶

Deploy configurations can set arbitrary environment keys for one stage:

stages:
  - stage_id: 0
    env:
      HCCL_IF_BASE_PORT: 23000
      MY_MODEL_CACHE: /mnt/model-cache

The legacy nested form under runtime.env is also accepted. vLLM-Omni converts keys and values to strings, applies them while the stage engine or child process is launched, logs keys but not values, and restores the parent process's prior environment afterward. A child process inherits the scoped values. This keeps sibling stages from permanently changing one another's parent environment.

This scoping has two consequences:

1. A setting read inside stage/model initialization can see the stage value.
2. A setting already cached by an import that occurred before stage launch cannot be changed retroactively with stage env.

Stage env is an escape hatch for process and third-party settings, not a replacement for typed stage configuration. Do not put request-varying behavior there. Secret values may reach child processes; logs and diagnostics must emit their keys only.

Inherited vLLM variables¶

vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to the vLLM 0.28 environment-variable reference for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm, XPU, ModelScope, and FlashInfer workspace settings.

An inherited name does not become Omni-owned because Omni reads or forwards it. Where behavior differs, the Omni-specific page should document only that difference. For example, device-visibility values can be further narrowed by a stage's devices setting.

Platform, external, and secret variables¶

Hardware runtimes, distributed launchers, and third-party libraries retain ownership of their variables. Common groups include:

• device visibility (ASCEND_RT_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES, and MUSA_VISIBLE_DEVICES);
• distributed launch (MASTER_ADDR, MASTER_PORT, RANK, and WORLD_SIZE);
• Hugging Face cache and authentication (HF_HOME, HF_TOKEN, and related names);
• accelerator libraries such as NCCL, FlashInfer, MindIE, MORI, and quack.

Consult the owning project's documentation for accepted values and support. HF_TOKEN, HUGGINGFACE_HUB_TOKEN, and OPENAI_API_KEY are explicitly marked for redaction in the reviewed inventory. Never print their values in environment collection, examples, bug reports, or logs.

Model-specific variables¶

The audit found 56 variables read by a single model or pipeline family. They are not listed as public usage options here because doing so would turn implementation escape hatches into an accidental compatibility contract.

Every audited model-specific name has a migration disposition in the environment-variable inventory:

The disposition is a migration target, not a statement that the existing environment switch is stable. Promote or request-scope work should land in model-owner-reviewed follow-up changes with validation and model-page updates. The family-sized migration packages and their ownership gate are tracked in #6232.

Environment collection and troubleshooting¶

Run the repository collector when preparing a bug report:

python collect_env.py

The environment section includes the safe public Omni allowlist, registered vLLM variables, and selected platform prefixes. It omits names marked for redaction and names containing common secret terms. Review the output before sharing it. A key supplied through stage env is not collected merely because it appears in deploy configuration; the same allowlist and prefix rules apply.

If a setting appears ineffective, first check its read time in the tables above. Restart processes after changing import- or startup-time values, verify that the variable reaches the intended worker, and check whether an explicit typed configuration field takes precedence.

Benchmark and internal variables¶

The reviewed inventory marks 20 benchmark variables as transitional while benchmark configuration moves to CLI/request schemas in issue #5376. They are not part of the serving/runtime contract.

Internal process-coordination and debug variables are deliberately excluded from this page. The CI scanner resolves string literals and module-level string constants used by direct os access, local wrapper helpers, aliased os imports, and environment membership checks. Server-storage names generated by Pydantic settings are checked separately against their model fields.

Names assembled from cross-module metadata and arbitrary keys supplied through stage env cannot be inferred reliably. Contributors must classify those names explicitly. CI fails when a statically resolved name has no classification or when a reviewed public name is absent from this page.