Skip to content

llmcompressor.entrypoints.model_free.scheduler

Functions:

estimate_job_memory 

estimate_job_memory(
    inverse_weight_map: InverseWeightMap,
) -> int

Rough memory estimate for a shard job, based on source file sizes.

Source code in src/llmcompressor/entrypoints/model_free/scheduler.py 
14
15
16
17
def estimate_job_memory(inverse_weight_map: InverseWeightMap) -> int:
    """Rough memory estimate for a shard job, based on source file sizes."""
    total = sum(os.path.getsize(p) for p in inverse_weight_map)
    return int(total * _MEMORY_MULTIPLIER)