Skip to content

vllm.v1.attention.ops.mxfp4_mla

Row layout of the mxfp4_mla KV cache.

OCP MXFP4, group 32: E2M1 elements packed two per byte (the low nibble holds the lower index) and one E8M0 scale per group. A row is the packed data followed by its scales, row_bytes(d) == d // 2 + d // 32.

Modules:

  • read –

    Triton read-side unpack for the mxfp4_mla KV cache.

  • store –

    Triton store kernel for the mxfp4_mla KV cache.

Functions:

  • row_bytes –

    Packed row size: data bytes + scale bytes. 272 for a 512-wide latent.

row_bytes(latent_dim, group=GROUP_SIZE)

Packed row size: data bytes + scale bytes. 272 for a 512-wide latent.

Source code in vllm/v1/attention/ops/mxfp4_mla/__init__.py 
20
21
22
23
24
def row_bytes(latent_dim: int, group: int = GROUP_SIZE) -> int:
    """Packed row size: data bytes + scale bytes. 272 for a 512-wide latent."""
    if latent_dim % group:
        raise ValueError(f"latent dim {latent_dim} not divisible by group {group}")
    return latent_dim // 2 + latent_dim // group