llmcompressor.observers.imatrix
Classes:
-
IMatrixMSEObserver–Search candidate min/max ranges using per-input-channel activation
IMatrixMSEObserver
Bases: Observer
Search candidate min/max ranges using per-input-channel activation importance (E[x²]) to weight QDQ error.
Supports CHANNEL, GROUP, and TENSOR_GROUP for weight-only Linear modules. Falls back to uniform MSE when importance data is unavailable.
Importance is accumulated on the observer as raw _imatrix_sum /
_imatrix_count and synced across DDP ranks via _act_sync_dict
before observation.
The base objective is importance * abs(QDQ(w) - w) ** norm, summed over
each quantization group. If magnitude_normalization is enabled, it also
multiplies each element by sqrt(w**2 + group_absmax**2). Importance is
accumulated from observer inputs as E[x²].
Observer kwargs are read from QuantizationArgs.observer_kwargs:
maxshrink (float): Maximum fraction of range to shrink. Defaults to
0.95.
patience (int): Number of consecutive candidates without a strict
improvement before early stopping in the eager implementation.
Triton always evaluates the full grid. 0 runs the full eager
search. Defaults to 5.
grid (float): Linear grid resolution or exponential candidate budget.
Defaults to 100.
norm (float): Power applied to absolute QDQ error before importance
weighting. Defaults to 2.8.
expand (float): Multiplier applied to observed min/max before the
shrink search; must be at least 1. Defaults to 1.0.
strict (bool): Raise an error if importance is unavailable or has an
unsupported shape. If false, warn and fall back to uniform MSE.
Defaults to False.
grid_type (str or None): "linear" or "exponential". None
selects exponential spacing when MX scales are generated (E8M0),
and linear spacing otherwise. Exponential candidates are powers
of two starting at 1. Defaults to None.
magnitude_normalization (bool): Multiply each error by
sqrt(w**2 + group_absmax**2). Defaults to True.
Methods:
-
attach–Attach a forward-pre hook to accumulate E[x²] per input channel.
-
detach–Remove the activation collection hook.
Source code in src/llmcompressor/observers/imatrix.py
attach
Attach a forward-pre hook to accumulate E[x²] per input channel.
Source code in src/llmcompressor/observers/imatrix.py
detach
NVFP4ExpandedIMatrixObserver
Bases: IMatrixMSEObserver
IMatrix observer with candidate range and grid defaults tuned for NVFP4.
Covers 1.8x down to ~0.8x of the per-group range, matching the candidate
range of :class:NVFP4ExpandedMSEObserver. It retains imatrix's default
magnitude normalization.
Usage::
QuantizationArgs(
...
observer="nvfp4_expanded_imatrix",
)