Regional Compilation¶

Regional compilation applies torch.compile to the repeated transformer blocks declared by a diffusion model. It is the default compilation scope when diffusion inference runs without --enforce-eager.

Configuration¶

Dynamic compilation is enabled by default so the compiled regions can handle mixed resolutions. For a fixed-shape workload, disable it explicitly:

vllm serve <model> --omni --no-diffusion-compile-dynamic

The equivalent per-stage deploy configuration is:

stages:
  - stage_id: 0
    diffusion_compile_granularity: regional
    diffusion_compile_dynamic: false

For an experimental whole-transformer compile scope, set --diffusion-compile-granularity full or use diffusion_compile_granularity: full in the stage configuration. Full scope may still contain graph breaks; it does not force one graph. It is rejected when HSDP, sequence parallelism, CPU offload, or layerwise offload is enabled. Use regional scope with those features.

These settings control the generic model-runner compilation path. Pipelines that provide their own setup_compile() implementation manage their compilation policy independently. Compilation is lazy, so backend or graph errors can first surface on the initial request.

Use --enforce-eager to disable the model runner's generic compile setup. Pipelines that compile internally define their own eager-mode behavior.