Model Adapters

Module Contents

vllm.model_executor.models.adapters.as_embedding_model(cls: _T) → _T

Subclass an existing vLLM model to support embeddings.

By default, the embeddings of the whole prompt are extracted from the normalized hidden state corresponding to the last token.

Note

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_classification_model(cls: _T) → _T

Subclass an existing vLLM model to support classification.

By default, the class probabilities are extracted from the softmaxed hidden state corresponding to the last token.

Note

We assume that the classification head is a single linear layer stored as the attribute score of the top-level model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_reward_model(cls: _T) → _T

Subclass an existing vLLM model to support reward modeling.

By default, we return the hidden states of each token directly.

Note

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.