Skip to content

llmcompressor.entrypoints.model_free.reindex_fused_weights

Functions:

reindex_fused_weights 

reindex_fused_weights(
    model_stub: str,
    save_directory: str,
    num_workers: int = 5,
)

This function is deprecated. The reindexing step is no longer necessary for model_free_ptq with microscale schemes (NVFP4A16, MXFP4A16).

Previously, this script was used to reindex safetensors files so that all fused modules (gate_up, qkv) were in the same safetensors file. This is now handled automatically by model_free_ptq.

Parameters:

  • model_stub (str) –

    huggingface model hub or path to local weights files

  • save_directory (str) –

    output directory for reindexed weights files

  • num_workers (int, default: 5 ) –

    number of worker threads to save files with

Source code in src/llmcompressor/entrypoints/model_free/reindex_fused_weights.py 
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
@deprecated(
    "The reindex_fused_weights preprocessing step is no longer necessary. "
    "model_free_ptq now handles fused weights directly without requiring reindexing."
)
def reindex_fused_weights(
    model_stub: str,
    save_directory: str,
    num_workers: int = 5,
):
    """
    This function is deprecated. The reindexing step is no longer necessary for
    model_free_ptq with microscale schemes (NVFP4A16, MXFP4A16).

    Previously, this script was used to reindex safetensors files so that all fused
    modules (gate_up, qkv) were in the same safetensors file. This is now handled
    automatically by model_free_ptq.

    :param model_stub: huggingface model hub or path to local weights files
    :param save_directory: output directory for reindexed weights files
    :param num_workers: number of worker threads to save files with
    """
    logger.warning(
        "reindex_fused_weights is deprecated and does nothing. "
        "The reindexing step is no longer necessary - model_free_ptq now handles "
        "fused weights directly."
    )