Skip to content

compressed_tensors.quantization.quant_scheme

Classes:

  • QuantizationScheme –

    Set of QuantizationArgs defining how the weights, inputs and outputs of target list

Functions:

QuantizationScheme

Bases: BaseModel

Set of QuantizationArgs defining how the weights, inputs and outputs of target list of modules should be quantized

Parameters:

  • targets –

    list of modules to apply the QuantizationArgs to, can be layer names, layer types or a regular expression, typically ["Linear"]

  • weights –

    quantization config for layer weights

  • input_activations –

    quantization config for layer inputs

  • output_activations –

    quantization config for layer outputs

  • format –

    CompressionFormat for the layer

is_preset_scheme 

is_preset_scheme(name: str) -> bool

Parameters:

  • name (str) –

    preset quantization settings name

Returns:

  • bool –

    True if the name is a preset scheme name

Source code in src/compressed_tensors/quantization/quant_scheme.py 
156
157
158
159
160
161
def is_preset_scheme(name: str) -> bool:
    """
    :param name: preset quantization settings name
    :return: True if the name is a preset scheme name
    """
    return name.upper() in PRESET_SCHEMES

preset_name_to_scheme 

preset_name_to_scheme(name: str, targets: list[str]) -> QuantizationScheme

Parameters:

  • name (str) –

    preset quantization settings name. must exist in upper case in PRESET_SCHEMES

  • targets (list[str]) –

    list of quantization targets to be passed to the Scheme

Returns:

Source code in src/compressed_tensors/quantization/quant_scheme.py 
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
def preset_name_to_scheme(name: str, targets: list[str]) -> QuantizationScheme:
    """
    :param name: preset quantization settings name. must exist in upper case in
        PRESET_SCHEMES
    :param targets: list of quantization targets to be passed to the Scheme
    :return: new QuantizationScheme for a given name with the given targets
    """
    name = name.upper()

    if name not in PRESET_SCHEMES:
        raise KeyError(
            f"Unknown preset scheme name {name}, "
            f"available names: {list(PRESET_SCHEMES.keys())}"
        )

    scheme_args = deepcopy(PRESET_SCHEMES[name])  # deepcopy to avoid args references
    return QuantizationScheme(
        targets=targets,
        **scheme_args,
    )