compressed_tensors.quantization.quant_args

Bases: Aliasable, str, Enum

Enum storing strategies for activation ordering during GPTQ calibration

Weight: Changes the way calibration occurs but doesn't change the quantization format compared to no activation ordering (normal latency). Compared to Group, it has lower latency and slightly worse accuracy. Compared to no activation ordering during calibration it has slightly better accuracy.

Static: alias for Weight

Bases: str, Enum

Enum storing potential dynamic types.

1. If dynamic is True, all quantization parameters are generated on the fly.
2. If dynamic is False, all quantization parameters generated are static.
3. If "local" is provided, only local quantization parameters are dynamic.

Note: "local" requires the TENSOR_GROUP strategy, which currently only applies to NVFP4.

Bases: FloatArgs

Methods:

• cast_to_fp4 –

  Round float values to the nearest E2M1 representable value.

cast_to_fp4(x: Tensor) -> torch.Tensor

54
55
56
57
58
59
60
61
62
63
64
65
66

@staticmethod
def cast_to_fp4(x: torch.Tensor) -> torch.Tensor:
    """Round float values to the nearest E2M1 representable value.

    Uses Triton for CUDA/XPU tensors when available, falls back to
    torch.compile for CPU tensors.

    :param x: input tensor to quantize
    :return: input tensor after rounding to fp4 (maintains same dtype)
    """
    from compressed_tensors.quantization.utils.fp4_utils import cast_to_fp4

    return cast_to_fp4(x)

Bases: BaseModel

User facing arguments used to define a quantization config for weights or activations

Parameters:

• num_bits –

  quantization bit depth

• type –

  dtype to quantize to, either int or float

• symmetric –

  whether or not quantization scale is symmetric about zero-point

• strategy –

  string id determining the scope of scale/zero-point to apply

• group_size –

  group length to use for the group strategy

• block_structure –

  2d block structure to use for the block strategy; must be a list of two ints [rows, cols] like [128, 128].

• dynamic –

  set True to perform dynamic quantization - values will not be calibrated during calibration phase, instead during inference new quantization ranges will be observed with every sample. Defaults to False for static quantization. Note that enabling dynamic quantization will change the default observer to a memoryless one

• actorder –

  activation ordering strategy for GPTQ calibration. Options are GROUP (reorder by activation with g_idx mapping, higher accuracy but higher latency -- removed 2026/08/27), WEIGHT (reorder columns by activation magnitude during calibration only, normal latency with slight accuracy improvement) or None (no activation ordering).

Bases: str, Enum

Enum storing quantization strategy options

Bases: str, Enum

Enum storing quantization type options