Skip to content

vllm.model_executor.model_loader.weight_cache.ipc_backends.base

The local weight-sharing backend contract.

Classes:

  • IPCBackendBase –

    Materialize shared storage and export rebuildable entries.

IPCBackendBase

Bases: ABC

Materialize shared storage and export rebuildable entries.

Methods:

  • export_storage –

    Describe a storage without copying it for each consumer.

  • materialize –

    Return records for the canonical storages after warmup.

Source code in vllm/model_executor/model_loader/weight_cache/ipc_backends/base.py
class IPCBackendBase(ABC):
    """Materialize shared storage and export rebuildable entries."""

    def materialize(self, storages: list[torch.UntypedStorage]) -> list[StorageRecord]:
        """Return records for the canonical storages after warmup."""
        return [StorageRecord(owner=storage, storage=storage) for storage in storages]

    @abstractmethod
    def export_storage(self, record: StorageRecord) -> StorageEntry:
        """Describe a storage without copying it for each consumer."""

export_storage(record) abstractmethod

Describe a storage without copying it for each consumer.

Source code in vllm/model_executor/model_loader/weight_cache/ipc_backends/base.py
@abstractmethod
def export_storage(self, record: StorageRecord) -> StorageEntry:
    """Describe a storage without copying it for each consumer."""

materialize(storages)

Return records for the canonical storages after warmup.

Source code in vllm/model_executor/model_loader/weight_cache/ipc_backends/base.py
def materialize(self, storages: list[torch.UntypedStorage]) -> list[StorageRecord]:
    """Return records for the canonical storages after warmup."""
    return [StorageRecord(owner=storage, storage=storage) for storage in storages]