Cache-DiT Guide¶

Table of Content¶

• Overview
• Quick Start
• Example Script
• Request-Scoped Quality
• Acceleration Methods
• Configuration Parameters
• Best Practices
• Troubleshooting
• Summary
• Additional Resources

Overview¶

Cache-DiT accelerates diffusion transformer models through intelligent caching mechanisms, providing significant speedup with minimal quality loss. It supports multiple acceleration techniques that can be combined for optimal performance:

• DBCache: Dual Block Cache for reducing redundant computations
• TaylorSeer: Taylor expansion-based forecasting for faster inference
• SCM: Step Computation Masking for selective step computation

See supported models list in Supported Models.

Quick Start¶

Basic Usage¶

Enable cache-dit acceleration by simply setting cache_backend="cache_dit":

from vllm_omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams

omni = Omni(
    model="Qwen/Qwen-Image",
    cache_backend="cache_dit",  # Enable Cache-DiT with defaults
)

outputs = omni.generate(
    "a beautiful landscape",
    OmniDiffusionSamplingParams(num_inference_steps=50),
)

Note: When cache_config is not provided, Cache-DiT uses optimized default values. See the Configuration Parameters section for details.

Custom Configuration¶

To customize cache-dit settings, provide a cache_config dictionary, for example:

omni = Omni(
    model="Qwen/Qwen-Image",
    cache_backend="cache_dit",
    cache_config={
        "Fn_compute_blocks": 1,
        "Bn_compute_blocks": 0,
        "max_warmup_steps": 4,
        "residual_diff_threshold": 0.12,
    },
)

Example Script¶

Multi-Stage Models (MammothModa2)¶

MammothModa2 is a multi-stage AR→DiT pipeline; the diffusion runner owns the Cache-DiT lifecycle of its DiT stage. Configure it on the DiT stage entry of the deploy YAML so the acceleration only applies to the denoising stage:

# deploy YAML (see vllm_omni/deploy/mammoth_moda2.yaml)
stages:
  - stage_id: 1
    # ... other DiT stage settings ...
    cache_backend: cache_dit
    cache_config:
      Fn_compute_blocks: 1
      Bn_compute_blocks: 0
      max_warmup_steps: 4
      residual_diff_threshold: 0.24
      max_continuous_cached_steps: 3
    enable_cache_dit_summary: true   # log skip-ratio stats per request

Behavior notes:

• Only the repeated main-layer stack is cached; the Q-Former refiners always run.
• Sequential-CFG parity requires the unconditional pass on every denoise step, so the cfg_range skip optimization is not used: outside the interval CFG is neutralized with scale=1.0 instead of skipping the uncond forward.
• Requests with text_guidance_scale = 1.0 run with cache hooks disabled.
• residual_diff_threshold defaults to 0.24 (measured ~1.7-2.2x at 50 steps, PSNR 27-34 dB vs uncached). For 20-step generation — especially with a partial cfg_range — use 0.12 for noticeably closer parity at a modest speed cost.

Offline Inference¶

Use the example script under examples/offline_inference/text_to_image:

cd examples/offline_inference/text_to_image
python text_to_image.py \
    --model Qwen/Qwen-Image \
    --prompt "a cup of coffee on the table" \
    --cache-backend cache_dit \
    --num-inference-steps 50

See the text_to_image.py for detailed configuration options.

The script uses cache-dit acceleration with a hybrid configuration combining DBCache, SCM, and TaylorSeer:

omni = Omni(
    model="Qwen/Qwen-Image",
    cache_backend="cache_dit",
    cache_config={
        # Scheme: Hybrid DBCache + SCM + TaylorSeer
        "Fn_compute_blocks": 1,  # Optimized for single-transformer models
        "Bn_compute_blocks": 0,  # Number of backward compute blocks
        "max_warmup_steps": 4,  # Maximum warmup steps (works for few-step models)
        "residual_diff_threshold": 0.24,  # Higher threshold for more aggressive caching
        "max_continuous_cached_steps": 3,  # Limit to prevent precision degradation
        # TaylorSeer parameters [cache-dit only]
        "enable_taylorseer": False,  # Disabled by default (not suitable for few-step models)
        "taylorseer_order": 1,  # TaylorSeer polynomial order
        # SCM (Step Computation Masking) parameters [cache-dit only]
        "scm_steps_mask_policy": None,  # SCM mask policy: None (disabled), "slow", "medium", "fast", "ultra"
        "scm_steps_policy": "dynamic",  # SCM steps policy: "dynamic" or "static"
    }
)

You can customize the configuration by modifying the cache_config dictionary to use only specific methods (e.g., DBCache only, DBCache + SCM, etc.) based on your quality and speed requirements.

For image-to-image tasks, use the example script under examples/offline_inference/image_to_image:

cd examples/offline_inference/image_to_image
python image_edit.py \
    --model Qwen/Qwen-Image-Edit \
    --prompt "make the sky more colorful" \
    --image path/to/input/image.jpg \
    --cache-backend cache_dit \
    --num-inference-steps 50 \
    --cache-dit-max-continuous-cached-steps 3 \
    --cache-dit-residual-diff-threshold 0.24 \
    --cache-dit-enable-taylorseer

See the image_edit.py for detailed configuration options.

Online Serving¶

# Default configuration (recommended)
vllm serve Qwen/Qwen-Image --omni --port 8091 --cache-backend cache_dit

# Custom configuration
vllm serve Qwen/Qwen-Image --omni --port 8091 \
  --cache-backend cache_dit \
  --cache-config '{"Fn_compute_blocks": 1, "residual_diff_threshold": 0.12}'

Request-Scoped Quality (MiniMax H3)¶

MiniMax H3 can switch Cache-DiT at the request boundary without enabling a cache backend at server startup. Add a quality field to an online request:

# Native reference path for this request.
-F 'quality=lossless'

# H3's conservative Cache-DiT profile for this request.
-F 'quality=high'

For offline inference, set the same field on the sampling parameters:

OmniDiffusionSamplingParams(
    quality="high",
    num_inference_steps=50,
    extra_args={"task": "t2va", "duration": 5.0, "audio_flow_shift": 3.0},
)

The startup option is only needed when omitted-quality requests should use the server-configured Cache-DiT profile by default.

See the MiniMax H3 recipe for a complete request and measured trade-off.

Acceleration Methods¶

For comprehensive illustration, please view Cache-DiT User Guide.

1. DBCache (Dual Block Cache)¶

DBCache intelligently caches intermediate transformer block outputs when the residual differences between consecutive steps are small, reducing redundant computations without sacrificing quality.

Example Configuration:

cache_config={
    "Fn_compute_blocks": 8,           # Use first 8 blocks for difference computation
    "Bn_compute_blocks": 0,           # No additional fusion blocks
    "max_warmup_steps": 8,            # Cache after 8 warmup steps
    "residual_diff_threshold": 0.12,  # Lower threshold for faster inference
    "max_cached_steps": -1,           # No limit on cached steps
}

Performance Tips:

• Default Fn_compute_blocks=1 works well for most cases. Some models (e.g., FLUX.2-klein) use a larger value for Fn_compute_blocks for a balanced performance.
• Increase residual_diff_threshold (e.g., 0.12-0.15) for faster inference with slight quality trade-off, or decrease from default 0.24 for higher quality.
• Default max_warmup_steps=4 is optimized for few-step models. Increase to 6-8 for more steps if needed.

2. TaylorSeer¶

TaylorSeer uses Taylor expansion to forecast future hidden states, allowing the model to skip some computation steps while maintaining quality.

Example Configuration:

cache_config={
    "enable_taylorseer": True,
    "taylorseer_order": 1,  # First-order Taylor expansion
}

Performance Tips:

• TaylorSeer is not suitable for few-step distilled models.
• Use taylorseer_order=1 for most cases (good balance of speed and quality).
• Combine with DBCache for maximum acceleration.
• Higher orders (2-3) may improve quality but reduce speed gains.

3. SCM (Step Computation Masking)¶

SCM allows you to specify which steps must be computed and which can use cached results, similar to LeMiCa/EasyCache style acceleration.

scm_steps_mask_policy options (number of compute steps out of 28):

Example Configuration:

cache_config={
    "scm_steps_mask_policy": "medium",  # Balanced speed/quality
    "scm_steps_policy": "dynamic",      # Use dynamic cache
}

Performance Tips:

• SCM is disabled by default. Enable it by setting a policy value if you need additional acceleration.
• Start with "medium" policy and adjust based on quality requirements.
• Use "fast" or "ultra" for maximum speed when quality can be slightly compromised.
• "dynamic" policy generally provides better quality than "static".
• SCM mask is automatically regenerated when num_inference_steps changes during inference.

Configuration Parameters¶

In cache_config passed to Omni constructor, it accepts the arguments of DBCacheConfig (Cache-DiT API Reference). Key parameters are listed below:

Best Practices¶

When to Use¶

Good for:

• Production deployments requiring fast inference
• Diffusion transformer models (DiT architecture)
• Scenarios where 1.5x-3x speedup is valuable

Not for:

• Non-DiT architectures (use model-specific acceleration instead)
• Models already using few-step distillation (< 10 steps)

Troubleshooting¶

Common Issue 1: Quality Degradation¶

Symptoms: Generated images have visible artifacts or lower quality

Solution:

# Reduce aggressiveness - use more conservative settings
cache_config={
    "residual_diff_threshold": 0.20,  # Lower threshold (closer to default 0.24)
    "Fn_compute_blocks": 8,            # Use more blocks for better decisions
    "max_warmup_steps": 6,             # Longer warmup
    "scm_steps_mask_policy": "slow",   # More compute steps
}

Summary¶

Using Cache-DiT acceleration:

1. ✅ Enable Cache-DiT - Set cache_backend="cache_dit" to get 1.5x-3x speedup with optimized defaults
2. ✅ (Optional) Customize - Adjust cache_config parameters for specific speed/quality trade-offs

Additional Resources¶

• Cache-DiT documentation
• Cache-DiT API reference