vllm_omni.distributed.ray_utils ¶

Modules:

Name	Description
`utils`

calculate_total_bytes ¶

calculate_total_bytes(size_args, dtype)

Calculate total bytes for a tensor allocation, handling nested tuples in size args.

is_ray_initialized()

Check if Ray is initialized without hard dependency on Ray.

maybe_disable_pin_memory_for_ray(
    obj, size_bytes, threshold=32 * 1024 * 1024
)

Context manager to temporarily disable pin_memory if running in Ray and the allocation size exceeds the threshold.

This is a workaround for Ray workers often having low ulimit -l (locked memory), causing OS call failed errors when allocating large pinned buffers.