Skip to content

vllm_omni.diffusion.models.soulx_singer.modules.preprocess.asr

Lyric transcription (Paraformer + Parakeet) for SoulX preprocess.

logger module-attribute 

logger = init_logger(__name__)

LyricModel

Bases: Module

device_str instance-attribute 

device_str = device

en_asr instance-attribute 

en_asr: Module | None = None

en_model_path instance-attribute 

en_model_path = en_model_path

hop_size instance-attribute 

hop_size = hop_size

target_sr instance-attribute 

target_sr = target_sr

zh_asr instance-attribute 

zh_asr = ParaformerASR(zh_model_path, device=device)

forward 

forward(
    audio: str | ndarray,
    language: str = "Mandarin",
    *,
    sample_rate: int = 16000,
    f0: ndarray | None = None,
) -> tuple[list[str], list[float]]

ParaformerASR

Bases: Module

Mandarin/Cantonese ASR via FunASR Paraformer.

asr instance-attribute 

asr = AutoModel(
    model=model_path, disable_update=True, device=device
)

forward 

forward(
    audio: str | ndarray, sample_rate: int = 16000
) -> tuple[list[str], list[float]]

ParakeetASR

Bases: Module

English ASR via NeMo Parakeet-TDT.

asr instance-attribute 

asr = nemo_asr.models.ASRModel.restore_from(
    restore_path=model_path, map_location=device
)

forward 

forward(
    audio: str | ndarray, sample_rate: int = 16000
) -> tuple[list[str], list[float]]

build_words_with_gaps 

build_words_with_gaps(
    raw_words, raw_timestamps, audio_duration: float
)

refine_word_durs_with_f0 

refine_word_durs_with_f0(
    words: list[str],
    word_durs: list[float],
    f0: ndarray,
    *,
    sample_rate: int = 24000,
    hop_size: int = 480,
) -> tuple[list[str], list[float]]