vllm_omni.diffusion.utils.media_utils ¶

Video/audio muxing utilities using PyAV (no ffmpeg binary dependency).

logger `module-attribute` ¶

logger = init_logger(__name__)

ChunkedMP4Encoder ¶

Encode temporal video chunks while the producer is still decoding.

A bounded queue and one muxing worker provide ordered backpressure. What that bounds is pending raw-frame memory: at most max_pending chunks wait to be encoded, so the bound is in chunks rather than bytes and a taller frame or a longer chunk raises it. The muxed container is not bounded -- it accumulates in memory and grows with output duration. Chunks use the same (T, H, W, 3) uint8 contract as :func:mux_video_audio_bytes.

close `class-attribute` `instance-attribute` ¶

close = finish

height `instance-attribute` ¶

height = int(height)

width `instance-attribute` ¶

width = int(width)

abort ¶

abort() -> None

Discard pending chunks and join the worker after cancellation.

Any chunk already being encoded must finish before the worker exits.

finish ¶

finish() -> bytes

Flush the muxer and return complete progressive MP4 bytes.

push ¶

push(
    chunk: ndarray,
    *,
    on_consumed: Callable[[], None] | None = None,
) -> None

Queue a chunk; asynchronous failures surface here or at finish().

on_consumed fires exactly once after this encoder has read chunk, including when the chunk is discarded by an abort or by a failed encode. Ownership transfers on call: whether this returns or raises, the callback is this encoder's responsibility, so a caller lending a pooled host buffer never compensates on the error path.

FragmentedMP4Muxer ¶

Incrementally mux video frames into one fragmented MP4 byte stream.

close ¶

close() -> bytes

Flush delayed encoder packets, close the container, and return final bytes.

mux_video_frames ¶

mux_video_frames(video_frames: ndarray) -> bytes

Mux a batch of uint8 RGB frames and return newly written MP4 bytes.

count_mp4_frames ¶

count_mp4_frames(video_bytes: bytes) -> int | None

Read an MP4's frame count from its sample table, without decoding it.

finalize_streaming_video_bytes ¶

finalize_streaming_video_bytes(
    video_bytes: bytes,
    *,
    input_format: str,
    fps: float = 25.0,
    video_codec_options: dict[str, str] | None = None,
) -> bytes

Convert streamed video bytes into a progressive MP4 for local playback.

mux_av_video_audio_bytes ¶

mux_av_video_audio_bytes(
    video_frames: Iterable[VideoFrame],
    width: int,
    height: int,
    audio_waveform: ndarray | None = None,
    *,
    fps: float = 25.0,
    audio_sample_rate: int | None = None,
    video_codec: str = "h264",
    audio_codec: str = "aac",
    crf: str = "18",
    video_codec_options: dict[str, str] | None = None,
) -> bytes

Mux preconstructed video frames and optional audio into MP4 bytes.

mux_video_audio_bytes ¶

mux_video_audio_bytes(
    video_frames: ndarray,
    audio_waveform: ndarray | None = None,
    *,
    fps: float = 25.0,
    audio_sample_rate: int = 44100,
    video_codec: str = "h264",
    audio_codec: str = "aac",
    crf: str = "18",
    video_codec_options: dict[str, str] | None = None,
) -> bytes

Mux video frames and optional audio waveform into MP4 bytes.

Parameters:

Name	Type	Description	Default
`video_frames`	`ndarray`	uint8 array of shape `(T, H, W, 3)` (RGB).	required
`audio_waveform`	`ndarray \| None`	float32 array – mono `(N,)` or `(N, C)` / `(C, N)`.	`None`
`fps`	`float`	Video frame rate.	`25.0`
`audio_sample_rate`	`int`	Audio sample rate in Hz.	`44100`
`video_codec`	`str`	Video codec name.	`'h264'`
`audio_codec`	`str`	Audio codec name.	`'aac'`
`crf`	`str`	Constant rate factor for the video encoder.	`'18'`

Returns:

Type	Description
`bytes`	Raw MP4 bytes ready to be written to disk or streamed.

normalize_preencode_batch_frames ¶

normalize_preencode_batch_frames(value: Any) -> int

Validate the request's MP4 transfer/encoding batch threshold.

normalize_video_codec_options ¶

normalize_video_codec_options(
    value: Any,
) -> dict[str, str] | None

Coerce a request's video_codec_options into PyAV's str->str contract.

The value reaches a worker straight from extra_params, so reject a non-mapping here rather than letting it fail inside the encoder after the decode has already run.