vllm.models.minimax_m3.amd.mono.dispatch
¶
Which MiniMax-M3 forwards take the mono path (--monokernel).
The deployment is checked once (config_refusal); a step is routed to the
kernels only when it is a pure decode of 1..MAX_TOKENS tokens whose tables the
kernels can read (_step_refusal). Every per-step condition is decided from
TP-replicated state: the kernels all-reduce in-kernel, so ranks that disagreed
would wait on each other forever.
The runner is built on the first eligible step, which under cudagraphs is the warmup run before the first capture (its allocations and IPC handshake cannot be captured), and rebuilt when vLLM re-binds the KV cache after its memory-profiling capture round.
Classes:
-
MonoDecode–Routes supported decode steps of one
MiniMaxM3Modelto the kernels.
Functions:
-
config_refusal–Why this deployment cannot use the kernels at all, or None.
MonoDecode
¶
Routes supported decode steps of one MiniMaxM3Model to the kernels.
Built only for a deployment config_refusal accepts.
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 | |
_ensure_runner()
¶
Build (or rebuild after a KV cache re-bind) the runner outside capture.
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
_slots_shared(fwd, n)
¶
The kernels insert K/V and the index key through one page-128 slot mapping, so the index cache must share the main cache's block ids. Checked once (a device sync).
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
_mxfp8_attention_ok(layers)
¶
The sparse layers' qkv and o projections are MXFP8. The weight dtype and scale shape are set when the layer is built, so this works before the weights load.
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
_separate_shared_ok(layers)
¶
Every MoE layer keeps its shared expert as a separate MLP (the kernels then
run it from that MLP's MXFP8 weights, weights.mxfp8_a_tiles).
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
_sparse_layers(layers)
¶
The layers the kernels run, after the dense prefix; empty for a layer mix they cannot serve.
_step_refusal(fwd, layer_name, n, positions)
¶
Why this step cannot run on the kernels, or None when it can.
Source code in vllm/models/minimax_m3/amd/mono/dispatch.py
config_refusal(vllm_config, model)
¶
Why this deployment cannot use the kernels at all, or None.