vllm_omni.patch ¶
release_cumem_allocations_for_shutdown ¶
release_cumem_allocations_for_shutdown(allocator) -> int
Retire every allocation still tracked by an inline worker.
CuMemAllocator.release_pools() only drops the Python MemPool and pluggable-allocator wrappers. Tensors retained outside the model runner can therefore keep entries in pointer_to_data after an inline engine shuts down, which prevents the next sleep-enabled engine from starting in the same process.
The caller must first stop worker activity and synchronize the device. Mapped allocations are unmapped here; already-asleep allocations were unmapped by sleep(). Both are removed from the live-usage registry and retained as swept handles so the later MemPool/tensor free callback can finish releasing their virtual addresses safely.