LLM Inputs#

vllm.inputs.PromptType#: alias of Union[str, TextPrompt, TokensPrompt, ExplicitEncoderDecoderPrompt]

class vllm.inputs.TextPrompt[source]#

Bases: TypedDict

Schema for a text prompt.

prompt: str#: The input text to be tokenized before passing to the model.

multi_modal_data: NotRequired[MultiModalDataDict]#: Optional multi-modal data to pass to the model, if the model supports it.

mm_processor_kwargs: NotRequired[Dict[str, Any]]#: Optional multi-modal processor kwargs to be forwarded to the multimodal input mapper & processor. Note that if multiple modalities have registered mappers etc for the model being considered, we attempt to pass the mm_processor_kwargs to each of them.

class vllm.inputs.TokensPrompt[source]#

Bases: TypedDict

Schema for a tokenized prompt.

prompt_token_ids: List[int]#: A list of token IDs to pass to the model.

multi_modal_data: NotRequired[MultiModalDataDict]#

Optional multi-modal data to pass to the model, if the model supports it.

Type:: DEPRECATED

mm_processor_kwargs: NotRequired[Dict[str, Any]]#

Optional multi-modal processor kwargs to be forwarded to the multimodal input mapper & processor. Note that if multiple modalities have registered mappers etc for the model being considered, we attempt to pass the mm_processor_kwargs to each of them.

Type:: DEPRECATED

LLM Inputs

Contents

LLM Inputs#