vllm.model_executor.models.adapters

Module Contents

Functions

as_classification_model	Subclass an existing vLLM model to support classification.
as_embedding_model	Subclass an existing vLLM model to support embeddings.
as_reward_model	Subclass an existing vLLM model to support reward modeling.

API

vllm.model_executor.models.adapters.as_classification_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T

Subclass an existing vLLM model to support classification.

By default, the class probabilities are extracted from the softmaxed hidden state corresponding to the last token.

.. note::

We assume that the classification head is a single linear layer stored as the attribute score of the top-level model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_embedding_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T

Subclass an existing vLLM model to support embeddings.

By default, the embeddings of the whole prompt are extracted from the normalized hidden state corresponding to the last token.

.. note::

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_reward_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T

Subclass an existing vLLM model to support reward modeling.

By default, we return the hidden states of each token directly.

.. note::

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.