vllm_omni.diffusion.models.ltx2.pipeline_ltx2_3_image2video ¶

LTX-2.3 image-to-video pipeline.

LTX23ImageToVideoPipeline ¶

Bases: LTX23Pipeline

LTX-2.3 image-to-video pipeline.

This keeps the LTX-2.3 prompt connector, x0-space CFG, sigma prompt modulation, and audio branch semantics from LTX23Pipeline while reusing the existing LTX image-conditioning contract: the first video latent frame is encoded from the input image and remains fixed during denoising.

support_image_input `class-attribute` `instance-attribute` ¶

support_image_input = True

video_processor `instance-attribute` ¶

video_processor = VideoProcessor(
    vae_scale_factor=self.vae_spatial_compression_ratio,
    resample="bilinear",
)

check_inputs ¶

check_inputs(
    image,
    height,
    width,
    prompt,
    latents=None,
    prompt_embeds=None,
    negative_prompt_embeds=None,
    prompt_attention_mask=None,
    negative_prompt_attention_mask=None,
)

forward ¶

forward(
    req: DiffusionRequestBatch,
    image: Image
    | Tensor
    | list[Image | Tensor]
    | None = None,
    prompt: str | list[str] | None = None,
    negative_prompt: str | list[str] | None = None,
    height: int | None = None,
    width: int | None = None,
    num_frames: int | None = None,
    frame_rate: float | None = None,
    num_inference_steps: int | None = None,
    sigmas: list[float] | None = None,
    timesteps: list[int] | None = None,
    guidance_scale: float = 4.0,
    noise_scale: float = 0.0,
    num_videos_per_prompt: int | None = 1,
    generator: Generator | list[Generator] | None = None,
    latents: Tensor | None = None,
    audio_latents: Tensor | None = None,
    prompt_embeds: Tensor | None = None,
    negative_prompt_embeds: Tensor | None = None,
    prompt_attention_mask: Tensor | None = None,
    negative_prompt_attention_mask: Tensor | None = None,
    decode_timestep: float | list[float] = 0.0,
    decode_noise_scale: float | list[float] | None = None,
    output_type: str = "np",
    return_dict: bool = True,
    attention_kwargs: dict[str, Any] | None = None,
    max_sequence_length: int | None = None,
) -> list[DiffusionOutput]

prepare_latents ¶

prepare_latents(
    image: Tensor | None = None,
    batch_size: int = 1,
    num_channels_latents: int = 128,
    height: int = 512,
    width: int = 768,
    num_frames: int = 121,
    noise_scale: float = 0.0,
    dtype: dtype | None = None,
    device: device | None = None,
    generator: Generator | list[Generator] | None = None,
    latents: Tensor | None = None,
) -> tuple[Tensor, Tensor]

Prepare I2V latents and the first-frame conditioning mask.

If caller-provided latents are used without an image, the latents must already represent the full video state including the conditioning first frame. Packed 3D latents are assumed to be in transformer token layout.

get_ltx2_post_process_func ¶

get_ltx2_post_process_func(od_config: OmniDiffusionConfig)

Factory for the LTX-2.3 post-process function.

Detects the vocoder output sample rate at factory time and captures it in the closure so that the audio_sample_rate flows through DiffusionEngine -> OmniRequestOutput -> serving_video.