Skip to content

vllm_omni.distributed.omni_connectors.connectors.base

logger module-attribute 

logger = get_connector_logger(__name__)

OmniConnectorBase

Bases: ABC

Base class for all OmniConnectors.

supports_raw_data class-attribute instance-attribute 

supports_raw_data: bool = False

cleanup abstractmethod 

cleanup(request_id: str) -> None

Clean up resources for a request.

close abstractmethod 

close() -> None

Release resources held by this connector.

Subclasses must implement this to clean up transport-specific resources (connections, memory pools, threads, etc.). Implementations should be idempotent (safe to call multiple times).

deserialize_obj staticmethod 

deserialize_obj(data: bytes) -> Any

Deserialize bytes to Python object using centralized serializer.

get abstractmethod 

get(
    from_stage: str,
    to_stage: str,
    get_key: str,
    metadata: dict[str, Any] | None = None,
) -> tuple[Any, int] | None

Retrieve Python object and payload size (bytes).

Parameters:

Name Type Description Default
from_stage str

Source stage identifier

 required
to_stage str

Destination stage identifier

 required
get_key str

Unique request identifier

 required
metadata dict[str, Any] | None

Optional transport-specific metadata. When provided, the connector uses it directly (e.g. source_host, source_port, data_size) instead of querying the sender. For heterogeneous TP the manager may supply partial metadata (host/port only); the connector will query the sender at that address to fill in data_size.

 None

Returns:

Type Description
tuple[Any, int] | None

Tuple of (Python object, serialized byte size) if found, None otherwise

health abstractmethod 

health() -> dict[str, Any]

Return health status and metrics.

put abstractmethod 

put(
    from_stage: str, to_stage: str, put_key: str, data: Any
) -> tuple[bool, int, dict[str, Any] | None]

Store Python object, internal serialization handled by connector.

Parameters:

Name Type Description Default
from_stage str

Source stage identifier

 required
to_stage str

Destination stage identifier

 required
put_key str

Unique request identifier

 required
data Any

Python object to store

 required

Returns:

Name Type Description
tuple tuple[bool, int, dict[str, Any] | None]

(success: bool, serialized_size: int, metadata: Optional[dict]) Metadata may contain transport-specific handles or inline data.

serialize_obj staticmethod 

serialize_obj(obj: Any) -> bytes

Serialize a Python object to bytes using centralized serializer.