Skip to content

vllm_omni.patch

release_cumem_allocations_for_shutdown

release_cumem_allocations_for_shutdown(allocator) -> int

Retire every allocation still tracked by an inline worker.

CuMemAllocator.release_pools() only drops the Python MemPool and pluggable-allocator wrappers. Tensors retained outside the model runner can therefore keep entries in pointer_to_data after an inline engine shuts down, which prevents the next sleep-enabled engine from starting in the same process.

The caller must first stop worker activity and synchronize the device. Mapped allocations are unmapped here; already-asleep allocations were unmapped by sleep(). Both are removed from the live-usage registry and retained as swept handles so the later MemPool/tensor free callback can finish releasing their virtual addresses safely.