llmcompressor.utils.triton_utils
Low-level Triton helpers shared by observer kernels.
Classes:
-
TritonDType–Shared Triton dtype codes; FP8 and E8M0 are used for scales only.
Functions:
-
calculate_candidate_scale–Derive and round a candidate scale from candidate min/max bounds.
-
calculate_observer_error–Calculate MSE or importance-weighted QDQ error for observer search.
-
calculate_qdq_error–Compute QDQ powers in FP32 after rounding the difference to input dtype.
-
is_xpu_target–Report a non-XPU target when Triton has no constexpr functions.
-
observer_triton_req–Check whether observer inputs use a format supported by the shared kernels.
-
quantize_dequantize–GPTQ-style QDQ shared by MSE and importance-weighted observer kernels.
-
round_candidate_scale–Round a floating candidate scale as compressed-tensors qparams do.
-
round_fp32_as_dtype–Round an FP32 value to the requested dtype and return it as FP32.
-
scale_round_config–Return the Triton scale rounding mode and minimum valid scale.
TritonDType
Bases: IntEnum
Shared Triton dtype codes; FP8 and E8M0 are used for scales only.
calculate_candidate_scale
calculate_candidate_scale(
min_value,
max_value,
q_min,
q_max,
SCALE_ROUND_TYPE: constexpr,
QUANT_TYPE: constexpr,
SYMMETRIC: constexpr,
COMPUTE_DTYPE: constexpr,
)
Derive and round a candidate scale from candidate min/max bounds.
Source code in src/llmcompressor/utils/triton_utils.py
calculate_observer_error
calculate_observer_error(
values,
quantized,
importance,
group_absmax,
norm: constexpr,
HAS_IMPORTANCE: constexpr,
HAS_MAGNITUDE_NORMALIZATION: constexpr,
COMPUTE_DTYPE: constexpr,
)
Calculate MSE or importance-weighted QDQ error for observer search.
Source code in src/llmcompressor/utils/triton_utils.py
calculate_qdq_error
Compute QDQ powers in FP32 after rounding the difference to input dtype.
Source code in src/llmcompressor/utils/triton_utils.py
is_xpu_target
observer_triton_req
Check whether observer inputs use a format supported by the shared kernels.
Source code in src/llmcompressor/utils/triton_utils.py
quantize_dequantize
quantize_dequantize(
values,
scale,
zero_point,
q_min,
q_max,
QUANT_TYPE: constexpr,
NUM_BITS: constexpr,
HAS_ZP: constexpr,
COMPUTE_DTYPE: constexpr,
USE_FP8_E4B15: constexpr = False,
)
GPTQ-style QDQ shared by MSE and importance-weighted observer kernels.
Source code in src/llmcompressor/utils/triton_utils.py
round_candidate_scale
Round a floating candidate scale as compressed-tensors qparams do.
Source code in src/llmcompressor/utils/triton_utils.py
round_fp32_as_dtype
Round an FP32 value to the requested dtype and return it as FP32.
Source code in src/llmcompressor/utils/triton_utils.py
scale_round_config
Return the Triton scale rounding mode and minimum valid scale.