Skip to content

vllm.models.glm5next.nvidia.specialized_decode.cutedsl.policy

Copy of Kimi K3's vllm/models/kimi_k3/nvidia/specialized_decode/cutedsl/common/policy.py, unchanged apart from imports.

Per-batch policy tables.

Functions:

  • batch_table –

    Per-batch table of integers ("1-2:96,3-8:120"), tuples ("16:64.16.4.8") or names

batch_table(spec)

Per-batch table of integers ("1-2:96,3-8:120"), tuples ("16:64.16.4.8") or names ("16-64:trtllm"); later entries override earlier ones.

Source code in vllm/models/glm5next/nvidia/specialized_decode/cutedsl/policy.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
def batch_table(spec):
    """Per-batch table of integers ("1-2:96,3-8:120"), tuples ("16:64.16.4.8") or names
    ("16-64:trtllm"); later entries override earlier ones."""
    table = {}
    for item in filter(None, spec.split(",")):
        span, value = item.split(":")
        lo, _, hi = span.partition("-")
        if "." in value:
            value = tuple(map(int, value.split(".")))
        elif value.lstrip("-").isdigit():
            value = int(value)
        table.update({b: value for b in range(int(lo), int(hi or lo) + 1)})
    return table