LLM Inputs#

vllm.inputs.PromptStrictInputs#

The inputs to the LLM, which can take one of the following forms:

alias of Union[str, TextPrompt, TokensPrompt]

class vllm.inputs.TextPrompt(*args, **kwargs)[source]#

Bases: dict

Schema for a text prompt.

prompt: str#

The input text to be tokenized before passing to the model.

multi_modal_data: typing_extensions.NotRequired[MultiModalData]#

Optional multi-modal data to pass to the model, if the model supports it.

class vllm.inputs.TokensPrompt(*args, **kwargs)[source]#

Bases: dict

Schema for a tokenized prompt.

prompt_token_ids: List[int]#

A list of token IDs to pass to the model.

multi_modal_data: typing_extensions.NotRequired[MultiModalData]#

Optional multi-modal data to pass to the model, if the model supports it.