llmcompressor.entrypoints.model_free.converter
Classes:
-
ModelFreePtqConverter–Converter that quantizes and compresses safetensors checkpoints without
ModelFreePtqConverter
Bases: Converter
Converter that quantizes and compresses safetensors checkpoints without
loading a model. Implements the compressed-tensors Converter protocol so
it can be chained with other converters (e.g. a dequantizer) and used
with convert_checkpoint. For microscale schemes, it needs the checkpoint's
weight_names to resolve fused weights; model_free_ptq passes them.
Parameters:
-
config(QuantizationConfig) –quantization config to apply
-
weight_names(Iterable[str] | None, default:None) –names of all tensors in the checkpoint. Required to resolve fused weights for microscale schemes, because a layer's fused group depends on its sibling layers. For example, Kimi
g_projis fused with delta attention q/k/v/b/f_a projections on some layers and with MLA q_a/kv_a projections on others
Methods:
-
get_dependencies–For microscale schemes, return fused partner tensor names that must be
-
process–Quantize and compress all tensors. Device is inferred from the tensors.
-
update_config–Build or merge the final QuantizationConfig in COMPRESSED status.
-
validate–Validate tensor shapes and simulate the compressed state-dict layout.
Source code in src/llmcompressor/entrypoints/model_free/converter.py
get_dependencies
For microscale schemes, return fused partner tensor names that must be loaded together to compute a shared global scale. Only the primary weight of a fused group has dependencies, so the shard owning the primary loads the whole group. Standard schemes have no cross-tensor dependencies.
Source code in src/llmcompressor/entrypoints/model_free/converter.py
process
Quantize and compress all tensors. Device is inferred from the tensors.
Source code in src/llmcompressor/entrypoints/model_free/converter.py
update_config
Build or merge the final QuantizationConfig in COMPRESSED status. When chained after a dequantizer (config=None), creates from scratch. When seeded with a pre-existing checkpoint config, merges into it.
Source code in src/llmcompressor/entrypoints/model_free/converter.py
validate
Validate tensor shapes and simulate the compressed state-dict layout.
Converter validation runs on meta tensors before conversion. Some
calibration observers need tensor values, so validation initializes each
quantized module with meta tensors and compresses it without calibration.
This verifies the same shape and compression compatibility checks as
:meth:process while leaving observers to run only during the real
conversion passes.
Source code in src/llmcompressor/entrypoints/model_free/converter.py
split_fused_moe_experts
Find fused MoE experts (with gate_up_proj/down_proj). Split them from 3D tensors into individual 2D expert tensors.
Args: tensors: Dictionary of loaded tensors from safetensors file
Returns: split_tensors: New dictionary with split expert weights