Skip to content

llmcompressor.modifiers.awq

Backwards compatibility shim for AWQModifier.

This module has been moved to llmcompressor.modifiers.transform.awq. This shim will be removed in a future version.

Classes:

  • AWQMapping –

    Dataclass storing config of activation mappings to smooth

Functions:

AWQMapping dataclass 

AWQMapping(
    smooth_layer: str,
    balance_layers: list[str],
    activation_hook_target: str | None = None,
)

Dataclass storing config of activation mappings to smooth The output activations of smooth_layer are input activations into the balance_layers

AWQMappings are resolved into ResolvedMappings, which retain pointers to the actual torch.nn.Modules and additional metadata at runtime

Parameters:

  • smooth_layer (str) –

    regex or name of the activation layer to smooth

  • balance_layers (list[str]) –

    list of regex or names of weight layers that must be balanced to offset the smoothing

  • activation_hook_target (str | None, default: None ) –

    optional dotted attribute path relative to the parent module (lowest common ancestor of balance_layers) specifying which submodule to hook for activation caching. Useful for parallel transformer blocks (e.g. Cohere, Gemma 3) where the first balance layer is not the correct place to capture activations. When None (default), the hook is placed on balance_layers[0].

get_layer_mappings_from_model 

get_layer_mappings_from_model(
    model: Module,
) -> list[AWQMapping]

Infer AWQ mappings from a model. Checks the dynamic mapping registry first (for models needing runtime-generated mappings), then falls back to the static registry, then to default mappings.

Parameters:

  • model (Module) –

    the model to infer mappings for

Returns:

  • list[AWQMapping] –

    list of AWQMapping for the model

Source code in src/llmcompressor/modifiers/transform/awq/dynamic_mappings.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
def get_layer_mappings_from_model(model: Module) -> list[AWQMapping]:
    """
    Infer AWQ mappings from a model. Checks the dynamic mapping registry
    first (for models needing runtime-generated mappings), then falls back
    to the static registry, then to default mappings.

    :param model: the model to infer mappings for
    :return: list of AWQMapping for the model
    """
    model_name = model.__class__.__name__

    if model_name in AWQ_DYNAMIC_MAPPING_REGISTRY:
        mappings = AWQ_DYNAMIC_MAPPING_REGISTRY[model_name](model)
        if mappings is not None:
            return mappings

    if model_name in AWQ_MAPPING_REGISTRY:
        return AWQ_MAPPING_REGISTRY[model_name]

    logger.info(
        f"Architecture {model_name} not found in mappings. "
        f"Using default mappings: {default_mappings}"
    )
    return default_mappings