Skip to content

vllm.v1.hisparse.types

Classes:

SparseKVOffloadCommand dataclass

Opaque scheduler-to-worker command for one model step.

Source code in vllm/v1/hisparse/types.py 
27
28
29
30
31
@dataclass
class SparseKVOffloadCommand:
    """Opaque scheduler-to-worker command for one model step."""

    page_transfers: list[SparseKVPageTransfer]

SparseKVPageTransfer dataclass

Copy one logical KV page between cache-manager and worker-owned tiers.

Source code in vllm/v1/hisparse/types.py 
 7
 8
 9
10
11
12
13
14
15
@dataclass(frozen=True)
class SparseKVPageTransfer:
    """Copy one logical KV page between cache-manager and worker-owned tiers."""

    transfer_id: int
    host_block_id: int
    resident_block_ids: tuple[int, ...]
    after_forward: bool
    restore: bool = False

SparseKVRowMirror dataclass

Mirror one contiguous resident-row span into the host cache.

Source code in vllm/v1/hisparse/types.py 
18
19
20
21
22
23
24
@dataclass(frozen=True)
class SparseKVRowMirror:
    """Mirror one contiguous resident-row span into the host cache."""

    source_starts: tuple[int, ...]
    destination_start: int
    num_rows: int