Skip to content

vllm.models.step5.amd.csa_cache

Step-5 CSA region summaries as their own KV cache group, and the per-step metadata the CSA op reads from it.

Classes:

Step5CSASummaryCacheLayer

Bases: Step5SparseSummaryCacheLayer

FP8 region summaries in their own KV cache group: under the layer-outer ROCm KV layout they use a 384-token block, so the CSA op reads them through that group's metadata.

Source code in vllm/models/step5/amd/csa_cache.py 
148
149
150
151
152
153
154
class Step5CSASummaryCacheLayer(Step5SparseSummaryCacheLayer):
    """FP8 region summaries in their own KV cache group: under the
    layer-outer ROCm KV layout they use a 384-token block, so the CSA op
    reads them through that group's metadata."""

    def get_attn_backend(self) -> type[AttentionBackend]:
        return Step5CSABackend