Skip to content

vllm_omni.utils.forced_aligner

Shared forced aligner for streaming TTS word timestamps.

Hosts one lazy-loaded, in-process pooling vllm.LLM shared by the TTS frontend. llm.encode is sync/blocking, so :func:align runs it in asyncio.to_thread, serialized on a lock since the offline LLM is not thread-safe. See :func:align for the return-value contract.

logger module-attribute 

logger = logging.getLogger(__name__)

ForcedAlignerConfig dataclass

Plain-data config for constructing the aligner vllm.LLM (lazy-loaded).

architecture class-attribute instance-attribute 

architecture: str | None = None

dtype class-attribute instance-attribute 

dtype: str | None = None

extra_llm_kwargs class-attribute instance-attribute 

extra_llm_kwargs: dict[str, Any] = field(
    default_factory=dict
)

gpu_memory_utilization class-attribute instance-attribute 

gpu_memory_utilization: float | None = None

max_model_len class-attribute instance-attribute 

max_model_len: int | None = None

model instance-attribute 

model: str

pooling_task class-attribute instance-attribute 

pooling_task: str | None = None

runner class-attribute instance-attribute 

runner: str | None = None

trust_remote_code class-attribute instance-attribute 

trust_remote_code: bool | None = None

ForcedAlignerLoadError

Bases: RuntimeError

The aligner model/config could not be loaded (permanent until restart).

Distinguished from a per-request alignment failure so the streaming layer can report a clear reason once rather than degrading every request to timestamps: null.

WordTimestamp dataclass

Internal alignment record. Serialized to a plain JSON object ({"word", "start_ms", "end_ms"}) at the WebSocket boundary.

end_ms instance-attribute 

end_ms: int

start_ms instance-attribute 

start_ms: int

word instance-attribute 

word: str

align async 

align(
    *,
    audio: bytes,
    text: str,
    sample_rate: int,
    config: ForcedAlignerConfig,
    language: str | None = None,
) -> list[WordTimestamp] | None

Run one forced-alignment pass.

Parameters:

Name Type Description Default
audio bytes

Signed-int16 little-endian mono PCM bytes.

 required
text str

Ground-truth text whose words to align.

 required
sample_rate int

Sample rate of audio in Hz.

 required
config ForcedAlignerConfig

Aligner config (same instance for every call across the server's lifetime; reload requires a server restart).

 required
language str | None

Optional language hint for word segmentation. None / "auto" use the space + Chinese-mixed path; "japanese" / "korean" (or their codes) need qwen_asr installed for faithful tokenisation, else they degrade to whitespace splitting.

 None

Returns:

Type Description
list[WordTimestamp] | None

List of :class:WordTimestamp on success (possibly empty for
list[WordTimestamp] | None

silence / no aligned tokens), None on a per-request alignment
list[WordTimestamp] | None

failure (decode error).

Raises:

Type Description
ForcedAlignerLoadError

the model/config could not be loaded. This is permanent until restart, so callers surface it once instead of degrading every request to None.

build_forced_aligner_config 

build_forced_aligner_config(
    args: Any,
) -> ForcedAlignerConfig | None

Build a config from CLI args, or None when the feature is off.

Precedence (lowest to highest): the packaged default YAML (:data:_DEFAULT_CONFIG_PATH, Qwen deploy defaults) -> a user YAML passed via --forced-aligner-config -> the --forced-aligner model path. The feature is off (returns None) unless a model resolves from this chain. Per-field overrides such as gpu_memory_utilization live in the YAML.