vllm.v1.worker.cpu.input_batch

CPU implementations for Model Runner V2 input-batch kernels.

Classes:

• BatchReqState –

  CPU request state for a scheduled batch, in batch (sorted) order.

BatchReqState

Bases: NamedTuple

CPU request state for a scheduled batch, in batch (sorted) order.

Source code in vllm/v1/worker/cpu/input_batch.py

class BatchReqState(NamedTuple):
    """CPU request state for a scheduled batch, in batch (sorted) order."""

    req_ids: list[str]
    num_scheduled_tokens: np.ndarray  # [num_reqs]
    num_tokens: int
    idx_mapping_np: np.ndarray  # [num_reqs]
    prefill_len_np: np.ndarray  # [num_reqs]
    num_computed_prefill_tokens_np: np.ndarray  # [num_reqs]
    is_prefilling_np: np.ndarray  # [num_reqs]
    max_seq_len_np: np.ndarray | None  # [num_reqs], None in non-pp case
    has_prefill: bool
    prefill_runs_as_decode_np: np.ndarray | None  # [num_reqs]
    decode_graph_eligible: bool