vllm.models.hy_v4.nvidia.mrope
¶
Shared per-forward M-RoPE rows for HY4 fused rotary kernels.
Functions:
-
apply_mrope–Rotate strided Q/K in place using this forward's shared token rows.
-
fused_mrope_norm–Normalize Q/KV and rotate K using shared per-token rows.
-
rope_into–Write interleaved-RoPE'd
src(tokens, heads, rot_dim) intodst. -
set_token_rope–Build this forward's per-token M-RoPE cos/sin rows once, before the
-
token_rope–(positions, cos_sin_cache)for the 1-D fused RoPE kernels.
_bind_token_rope(rotary_emb, rows)
¶
Keep this graph's rows alive and restore them before eager indexers.
apply_mrope(rotary_emb, positions, query, key)
¶
Rotate strided Q/K in place using this forward's shared token rows.
Source code in vllm/models/hy_v4/nvidia/mrope.py
fused_mrope_norm(rotary_emb, positions, q_c, kv_lora, q_norm, kv_norm)
¶
Normalize Q/KV and rotate K using shared per-token rows.
Source code in vllm/models/hy_v4/nvidia/mrope.py
rope_into(positions, src, cos_sin_cache, dst)
¶
Write interleaved-RoPE'd src (tokens, heads, rot_dim) into dst.
cos_sin_cache is (positions, rot_dim): cos then sin halves.
Source code in vllm/models/hy_v4/nvidia/mrope.py
set_token_rope(rotary_emb, positions, x)
¶
Build this forward's per-token M-RoPE cos/sin rows once, before the
layers run; token_rope reads them. x sets the cache dtype.
Source code in vllm/models/hy_v4/nvidia/mrope.py
token_rope(rotary_emb, positions, cos_sin_cache)
¶
(positions, cos_sin_cache) for the 1-D fused RoPE kernels.
[3, T] M-RoPE positions become arange(T) into the per-token cos/sin
rows built by set_token_rope for this forward.