vllm.models.mistral.mistral_large4.amd.specialized_decode.runtime.guards
¶
Loaded-model, device and KV-cache refusals (each list holds every reason).
Functions:
-
cache_refusals–At bind: every kernel layer's latent and indexer caches.
-
model_refusals–After load: the layer mix, each kernel layer's weights, the device.
cache_refusals(model, max_model_len)
¶
At bind: every kernel layer's latent and indexer caches.
Source code in vllm/models/mistral/mistral_large4/amd/specialized_decode/runtime/guards.py
model_refusals(model, config)
¶
After load: the layer mix, each kernel layer's weights, the device.