vllm.models.deepseek_v41.amd.ops.engram_host_table
¶
Engram host tables in HIP VMM host memory, shared across DP ranks by dmabuf FD.
The ROCm counterpart of nvidia/ops/engram_fabric_table.py, enabled by
VLLM_ROCM_ENGRAM_HIP_VMM. HIP cannot export host allocations as FABRIC handles,
so rank 0 sends POSIX FDs to its peers over a unix socket. Placing a table this
way is faster than filling /dev/shm.
Classes:
-
ExportedHostTable–What a peer needs to map a table; the FDs travel over
socket_path.
Functions:
-
host_memory_supported–Whether HIP host allocations export POSIX FDs that the CPU can map.
-
share_host_table–Map one table on every rank of a node-local group, or None on all of them.
ExportedHostTable
¶
Bases: NamedTuple
What a peer needs to map a table; the FDs travel over socket_path.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
_check_reserve(numa_node, created, padded_bytes)
¶
HIP may place host memory on another node, so watch the whole host too.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
_create_table(num_bytes)
¶
Allocate a table and export one FD per chunk.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
_map_table(table, num_bytes, fds, handles)
¶
Map a table on the GPU and the CPU, importing its FDs unless given handles.
Imported handles are appended to handles; the caller releases them and
closes the FDs, since the mappings keep the allocations alive. HIP grants
no CPU access to imported host allocations, so the CPU maps the dmabuf FDs
directly. Returns CPU and GPU uint8 views; either keeps the mappings alive.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 | |
_page_bytes()
cached
¶
The recommended host allocation granularity, which sets the GPU page size.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
_try_call(name, *args)
¶
Calls a HIP function and returns its error, clearing it: a failed hipMem* call otherwise fails the next torch launch on this thread.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
host_memory_supported()
cached
¶
Whether HIP host allocations export POSIX FDs that the CPU can map.
Source code in vllm/models/deepseek_v41/amd/ops/engram_host_table.py
share_host_table(num_bytes, group)
¶
Map one table on every rank of a node-local group, or None on all of them.
Rank 0 creates the table and sends its FDs to the others over a unix socket in a private directory. Returns CPU and GPU uint8 views of the table.