vllm_omni.distributed.omni_connectors.utils.serialization ¶

This implementation is adapted from vLLM’s MsgpackDecoder. However, zero-copy support has not been implemented yet.

Automatically reconstructs torch.Tensor, numpy.ndarray, PIL.Image, RequestOutput and CompletionOutput from their dict representations. TODO: Enable zero-copy support.

decoder = Decoder()

decode(data: bytes | bytearray | memoryview) -> Any

This implementation is adapted from vLLM’s MsgpackEncoder. However, zero-copy support has not been implemented yet. Handles torch.Tensor, numpy.ndarray, PIL.Image, RequestOutput and CompletionOutput by converting them to serializable dict representations. TODO: Enable zero-copy support.

encoder = Encoder(enc_hook=_enc_hook)

encode(obj: Any) -> bytes

Serialization/deserialization handler for Omni IPC.

decoder = OmniMsgpackDecoder()

encoder = OmniMsgpackEncoder()

deserialize(data: bytes | bytearray | memoryview) -> Any

serialize(obj: Any) -> bytes