Skip to content

vllm_omni.diffusion.postprocess.rife_interpolator

RIFE 4.22.lite frame interpolation for vLLM-Omni video generation.

RIFE model code is vendored and adapted from
  • https://github.com/hzwer/ECCV2022-RIFE (MIT License)
  • https://github.com/hzwer/Practical-RIFE (MIT License) Copyright (c) 2021 Zhewei Huang

The FrameInterpolator wrapper and vLLM-Omni integration code are original work.

logger module-attribute 

logger = init_logger(__name__)

FrameInterpolator

Lazy-loaded RIFE 4.22.lite frame interpolator.

interpolate_tensor 

interpolate_tensor(
    video: Tensor, exp: int = 1, scale: float = 1.0
) -> tuple[Tensor, int]

Head

Bases: Module

Feature encoder producing four-channel features at full resolution.

cnn0 instance-attribute 

cnn0 = Conv2d(3, 16, 3, 2, 1)

cnn1 instance-attribute 

cnn1 = Conv2d(16, 16, 3, 1, 1)

cnn2 instance-attribute 

cnn2 = Conv2d(16, 16, 3, 1, 1)

cnn3 instance-attribute 

cnn3 = ConvTranspose2d(16, 4, 4, 2, 1)

relu instance-attribute 

relu = LeakyReLU(0.2, True)

forward 

forward(x: Tensor) -> Tensor

IFBlock

Bases: Module

Single-scale optical flow, mask, and feature block.

conv0 instance-attribute 

conv0 = Sequential(
    _conv(in_planes, c // 2, 3, 2, 1),
    _conv(c // 2, c, 3, 2, 1),
)

convblock instance-attribute 

convblock = Sequential(
    ResConv(c),
    ResConv(c),
    ResConv(c),
    ResConv(c),
    ResConv(c),
    ResConv(c),
    ResConv(c),
    ResConv(c),
)

lastconv instance-attribute 

lastconv = Sequential(
    ConvTranspose2d(c, 4 * 13, 4, 2, 1), PixelShuffle(2)
)

forward 

forward(
    x: Tensor,
    flow: Tensor | None = None,
    scale: float = 1.0,
) -> tuple[Tensor, Tensor, Tensor]

IFNet

Bases: Module

Four-scale IFNet optical flow network.

block0 instance-attribute 

block0 = IFBlock(7 + 8, c=192)

block1 instance-attribute 

block1 = IFBlock(8 + 4 + 8 + 8, c=128)

block2 instance-attribute 

block2 = IFBlock(8 + 4 + 8 + 8, c=64)

block3 instance-attribute 

block3 = IFBlock(8 + 4 + 8 + 8, c=32)

encode instance-attribute 

encode = Head()

forward 

forward(
    x: Tensor,
    timestep: float = 0.5,
    scale_list: list[float] | None = None,
) -> tuple[
    list[Tensor],
    Tensor,
    list[tuple[Tensor, Tensor] | Tensor],
]

Model

Wraps IFNet and exposes RIFE-compatible load/inference helpers.

flownet instance-attribute 

flownet = IFNet()

device 

device() -> device

eval 

eval() -> Model

inference 

inference(
    img0: Tensor,
    img1: Tensor,
    scale: float = 1.0,
    timestep: float = 0.5,
) -> Tensor

load_model 

load_model(path: str) -> None

ResConv

Bases: Module

Residual convolution block with learnable beta scaling.

beta instance-attribute 

beta = Parameter(ones((1, c, 1, 1)), requires_grad=True)

conv instance-attribute 

conv = Conv2d(
    c, c, 3, 1, dilation, dilation=dilation, groups=1
)

relu instance-attribute 

relu = LeakyReLU(0.2, True)

forward 

forward(x: Tensor) -> Tensor

interpolate_video_tensor 

interpolate_video_tensor(
    video: Tensor,
    exp: int = 1,
    scale: float = 1.0,
    model_path: str | None = None,
) -> tuple[Tensor, int]

Interpolate a video tensor and return the FPS multiplier.

warp 

warp(ten_input: Tensor, ten_flow: Tensor) -> Tensor

Warp input tensor by optical flow using grid_sample.