vllm.distributed.engram_pool.nvlink
¶
Peer HBM over NVLink: every GPU maps every shard (cuMem FABRIC handles), so a fetch is one kernel that loads each row from its owner's memory.
Classes:
-
NvlinkTransport–groupprovides rank_in_group and all_gather(bytes).
NvlinkTransport
¶
group provides rank_in_group and all_gather(bytes).
Methods:
-
landed–Rows are in place: see
wait. -
shard–Allocate this GPU's shard and map every GPU's. Returns the shard, its
-
wait–Rows are in place once
startreturns: SMs moved them.
Source code in vllm/distributed/engram_pool/nvlink.py
landed(fetch, table)
staticmethod
¶
shard(num_bytes, name)
¶
Allocate this GPU's shard and map every GPU's. Returns the shard, its
meta arrays (shard addresses) and what keeps them alive. Collective.
Source code in vllm/distributed/engram_pool/nvlink.py
_gather_kernel(ids, meta, out, num_items, TABLES, K, WORDS, BLOCK)
¶
Load [..., TABLES, K] rows from their owners into [TABLES, ..., K]
rows, one 8-byte word per lane; dead ids give zeros.