llmcompressor.transformers.compression.mtp
Checkpoint handling for MTP layers supported by Transformers.
Functions:
-
has_mtp_in_checkpoint–Whether the model config declares MTP layers.
-
load_with_mtp_model–Attach MTP before the surrounding loaders convert and offload the model.
-
save_mtp_tensors–Preserve unloaded MTP using an already validated source.
-
validate_mtp_copy_source–Return source MTP weights and patterns only if the weights are unquantized.
has_mtp_in_checkpoint
Whether the model config declares MTP layers.
Source code in src/llmcompressor/transformers/compression/mtp.py
load_with_mtp_model
Attach MTP before the surrounding loaders convert and offload the model.
Source code in src/llmcompressor/transformers/compression/mtp.py
save_mtp_tensors
save_mtp_tensors(
model: PreTrainedModel,
destination: str,
*,
source_mtp: tuple[
dict[str, tuple[str, str | None]], list[str]
]
| None,
) -> None
Preserve unloaded MTP using an already validated source.
Source code in src/llmcompressor/transformers/compression/mtp.py
validate_mtp_copy_source
validate_mtp_copy_source(
model: PreTrainedModel,
) -> (
tuple[dict[str, tuple[str, str | None]], list[str]]
| None
)
Return source MTP weights and patterns only if the weights are unquantized.
Return None when the model config does not declare MTP. If MTP is declared but no weights are found, return an empty weight map with the patterns; save_mtp_tensors warns when saving this source.
Raise ValueError for source-quantized MTP weights, which cannot be copied without preserving their serving scheme.