Skip to content

vllm_omni.inputs.data

OmniPromptType module-attribute 

OmniPromptType: TypeAlias = (
    PromptType
    | OmniTextPrompt
    | OmniTokensPrompt
    | OmniEmbedsPrompt
    | OmniCustomPrompt
)

OmniSamplingParams module-attribute 

OmniSamplingParams: TypeAlias = (
    SamplingParams | OmniDiffusionSamplingParams
)

OmniSingletonPrompt module-attribute 

OmniSingletonPrompt: TypeAlias = (
    str
    | list[int]
    | OmniTextPrompt
    | OmniTokensPrompt
    | OmniEmbedsPrompt
)

Omni singleton prompt type extending vLLM's SingletonPrompt.

OmniCustomPrompt

Bases: TypedDict

Custom prompt type for diffusion pipelines with pre-tokenized inputs.

Allows users to pass pre-tokenized prompt IDs, attention masks, and extra arguments directly, bypassing the tokenization stage in the pipeline.

Attributes:

Name Type Description
prompt_ids list[int] | list[list[int]]

Pre-tokenized prompt token IDs (single or batched)
negative_prompt_ids list[int] | list[list[int]]

Pre-tokenized negative prompt token IDs
prompt_mask Tensor

Attention mask tensor for the prompt
negative_prompt_mask Tensor

Attention mask tensor for the negative prompt
extra_args dict[str, Any]

Additional pipeline-specific arguments

extra_args instance-attribute 

extra_args: dict[str, Any]

negative_prompt_ids instance-attribute 

negative_prompt_ids: list[int] | list[list[int]]

negative_prompt_mask instance-attribute 

negative_prompt_mask: Tensor

prompt_ids instance-attribute 

prompt_ids: list[int] | list[list[int]]

prompt_mask instance-attribute 

prompt_mask: Tensor

OmniDiffusionSamplingParams dataclass

The collection of sampling parameters passed to diffusion pipelines.

This dataclass contains all information needed during the diffusion pipeline execution, allowing methods to update specific components without needing to manage numerous individual parameters.

STA_param class-attribute instance-attribute 

STA_param: list | None = None

VSA_sparsity class-attribute instance-attribute 

VSA_sparsity: float = 0.0

audio_latents class-attribute instance-attribute 

audio_latents: Tensor | None = None

batch_size property 

batch_size

boundary_ratio class-attribute instance-attribute 

boundary_ratio: float | None = None

cfg_active_branch class-attribute instance-attribute 

cfg_active_branch: str | None = None

cfg_branch_kv_metadata class-attribute instance-attribute 

cfg_branch_kv_metadata: dict[str, dict[str, Any]] | None = (
    None
)

cfg_branch_past_key_values class-attribute instance-attribute 

cfg_branch_past_key_values: dict[str, Any] | None = None

cfg_branch_roles class-attribute instance-attribute 

cfg_branch_roles: list[str] | None = None

cfg_img_kv_metadata class-attribute instance-attribute 

cfg_img_kv_metadata: dict[str, Any] | None = None

cfg_img_past_key_values class-attribute instance-attribute 

cfg_img_past_key_values: Any | None = None

cfg_kv_request_ids class-attribute instance-attribute 

cfg_kv_request_ids: dict[str, str] | None = None

cfg_normalize class-attribute instance-attribute 

cfg_normalize: bool = False

cfg_text_kv_metadata class-attribute instance-attribute 

cfg_text_kv_metadata: dict[str, Any] | None = None

cfg_text_past_key_values class-attribute instance-attribute 

cfg_text_past_key_values: Any | None = None

debug class-attribute instance-attribute 

debug: bool = False

decode_noise_scale class-attribute instance-attribute 

decode_noise_scale: float | list[float] | None = None

decode_timestep class-attribute instance-attribute 

decode_timestep: float | list[float] | None = None

do_classifier_free_guidance class-attribute instance-attribute 

do_classifier_free_guidance: bool = False

enable_frame_interpolation class-attribute instance-attribute 

enable_frame_interpolation: bool = False

eta class-attribute instance-attribute 

eta: float = 0.0

extra_args class-attribute instance-attribute 

extra_args: dict[str, Any] = field(default_factory=dict)

extra_step_kwargs class-attribute instance-attribute 

extra_step_kwargs: dict[str, Any] = field(
    default_factory=dict
)

fps class-attribute instance-attribute 

fps: int | None = None

frame_interpolation_exp class-attribute instance-attribute 

frame_interpolation_exp: int = 1

frame_interpolation_model_path class-attribute instance-attribute 

frame_interpolation_model_path: str | None = None

frame_interpolation_scale class-attribute instance-attribute 

frame_interpolation_scale: float = 1.0

frame_rate class-attribute instance-attribute 

frame_rate: float | None = None

generator class-attribute instance-attribute 

generator: Generator | list[Generator] | None = None

generator_device class-attribute instance-attribute 

generator_device: str | None = None

guidance_rescale class-attribute instance-attribute 

guidance_rescale: float = 0.0

guidance_scale class-attribute instance-attribute 

guidance_scale: float = 0.0

guidance_scale_2 class-attribute instance-attribute 

guidance_scale_2: float | None = None

guidance_scale_provided class-attribute instance-attribute 

guidance_scale_provided: bool = False

height class-attribute instance-attribute 

height: int | None = None

height_latents class-attribute instance-attribute 

height_latents: list[int] | int | None = None

height_not_provided class-attribute instance-attribute 

height_not_provided: bool = False

image_latent class-attribute instance-attribute 

image_latent: Tensor | None = None

is_cfg_negative class-attribute instance-attribute 

is_cfg_negative: bool = False

is_prompt_processed class-attribute instance-attribute 

is_prompt_processed: bool = False

kv_metadata class-attribute instance-attribute 

kv_metadata: dict[str, Any] | None = None

latents class-attribute instance-attribute 

latents: Tensor | None = None

layers class-attribute instance-attribute 

layers: int = 4

lora_request class-attribute instance-attribute 

lora_request: LoRARequest | None = None

lora_scale class-attribute instance-attribute 

lora_scale: float = 1.0

mask_search_final_result_neg class-attribute instance-attribute 

mask_search_final_result_neg: list[list] | None = None

mask_search_final_result_pos class-attribute instance-attribute 

mask_search_final_result_pos: list[list] | None = None

max_sequence_length class-attribute instance-attribute 

max_sequence_length: int | None = None

modules class-attribute instance-attribute 

modules: dict[str, Any] = field(default_factory=dict)

n_tokens class-attribute instance-attribute 

n_tokens: int | None = None

need_kv_receive class-attribute instance-attribute 

need_kv_receive: bool = True

noise_pred class-attribute instance-attribute 

noise_pred: Tensor | None = None

num_frames class-attribute instance-attribute 

num_frames: int = 1

num_frames_round_down class-attribute instance-attribute 

num_frames_round_down: bool = False

num_inference_steps class-attribute instance-attribute 

num_inference_steps: int | None = None

num_outputs_per_prompt class-attribute instance-attribute 

num_outputs_per_prompt: int = 1

num_profiled_timesteps class-attribute instance-attribute 

num_profiled_timesteps: int = 8

output class-attribute instance-attribute 

output: Tensor | None = None

output_type class-attribute instance-attribute 

output_type: str | None = None

past_key_values class-attribute instance-attribute 

past_key_values: Any | None = None

profile class-attribute instance-attribute 

profile: bool = False

prompt_template class-attribute instance-attribute 

prompt_template: dict[str, Any] | None = None

raw_latent_shape class-attribute instance-attribute 

raw_latent_shape: Tensor | None = None

resolution class-attribute instance-attribute 

resolution: int = 640

resolved_frame_rate property 

resolved_frame_rate: float | None

return_frames class-attribute instance-attribute 

return_frames: bool = False

return_trajectory_decoded class-attribute instance-attribute 

return_trajectory_decoded: bool = False

return_trajectory_latents class-attribute instance-attribute 

return_trajectory_latents: bool = False

save_output class-attribute instance-attribute 

save_output: bool = True

seed class-attribute instance-attribute 

seed: int | None = None

sigmas class-attribute instance-attribute 

sigmas: list[float] | None = None

step_index class-attribute instance-attribute 

step_index: int | None = None

strength class-attribute instance-attribute 

strength: float | None = None

timestep class-attribute instance-attribute 

timestep: Tensor | float | int | None = None

timesteps class-attribute instance-attribute 

timesteps: Tensor | None = None

trajectory_latents class-attribute instance-attribute 

trajectory_latents: Tensor | None = None

trajectory_timesteps class-attribute instance-attribute 

trajectory_timesteps: list[Tensor] | None = None

true_cfg_scale class-attribute instance-attribute 

true_cfg_scale: float | None = None

use_en_prompt class-attribute instance-attribute 

use_en_prompt: bool = False

width class-attribute instance-attribute 

width: int | None = None

width_latents class-attribute instance-attribute 

width_latents: list[int] | int | None = None

width_not_provided class-attribute instance-attribute 

width_not_provided: bool = False

clone 

clone() -> OmniDiffusionSamplingParams

OmniEmbedsPrompt

Bases: EmbedsPrompt

Embeddings prompt with optional additional information.

Extends EmbedsPrompt to support additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Optional dictionary containing additional information (tensors or lists) to pass along with the prompt

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniTextPrompt

Bases: TextPrompt

Text prompt with optional embeddings and additional information.

Extends TextPrompt to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Optional dictionary containing additional information (tensors or lists) to pass along with the prompt

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

modalities instance-attribute 

modalities: NotRequired[list[str]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[Tensor]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniTokenInputs

Bases: TokensInput

Token inputs with optional embeddings and additional information.

Extends TokensInput to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings aligned with token IDs
additional_information NotRequired[dict[str, Any]]

Optional dictionary containing additional information (tensors or lists) to pass along with the inputs

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

OmniTokensPrompt

Bases: TokensPrompt

Tokens prompt with optional embeddings and additional information.

Extends TokensPrompt to support prompt embeddings and additional information payloads for direct transfer between pipeline stages.

Attributes:

Name Type Description
prompt_embeds NotRequired[Tensor]

Optional tensor containing prompt embeddings
additional_information NotRequired[dict[str, Any]]

Optional dictionary containing additional information (tensors or lists) to pass along with the prompt

additional_information instance-attribute 

additional_information: NotRequired[dict[str, Any]]

negative_prompt instance-attribute 

negative_prompt: NotRequired[str]

negative_prompt_embeds instance-attribute 

negative_prompt_embeds: NotRequired[list[Tensor] | None]

The embeddings of the prompt.

prompt_embeds instance-attribute 

prompt_embeds: NotRequired[Tensor]

token_inputs_omni 

token_inputs_omni(
    prompt_token_ids: list[int],
    prompt: str | None = None,
    cache_salt: str | None = None,
    prompt_embeds: Tensor | None = None,
    additional_information: dict[str, Any] | None = None,
) -> OmniTokenInputs

Construct token inputs with optional embeddings and metadata.

Creates an OmniTokenInputs object with token IDs and optional embeddings and additional information for pipeline stage transfer.

Parameters:

Name Type Description Default
prompt_token_ids list[int]

List of token IDs for the prompt

 required
prompt str | None

Optional prompt string

 None
cache_salt str | None

Optional cache salt for prefix caching

 None
prompt_embeds Tensor | None

Optional tensor containing prompt embeddings

 None
additional_information dict[str, Any] | None

Optional dictionary containing additional information (tensors or lists)

 None

Returns:

Type Description
OmniTokenInputs

OmniTokenInputs instance with the provided data