llmcompressor.core.events

LLM Compressor Core Events Package

This package provides the core components and lifecycle management for events used in the LLM Compressor framework. It includes definitions for various event types and lifecycles that are critical for managing the state and execution flow of the model compression and training processes.

Modules:

event –

Module for defining and managing events in the LLM Compressor.

Classes:

Event –

A class for defining an event that can be triggered during sparsification.
EventType –

An Enum for defining the different types of events that can be triggered

Event `dataclass`

Event(
    type_: Optional[EventType] = None,
    steps_per_epoch: Optional[int] = None,
    batches_per_step: Optional[int] = None,
    invocations_per_step: int = 1,
    global_step: int = 0,
    global_batch: int = 0,
)

A class for defining an event that can be triggered during sparsification.

Parameters:

type_ (Optional[EventType], default: None ) –

The type of event.
steps_per_epoch (Optional[int], default: None ) –

The number of steps per epoch.
batches_per_step (Optional[int], default: None ) –

The number of batches per step where step is an optimizer step invocation. For most pathways, these are the same. See the invocations_per_step parameter for more details when they are not.
invocations_per_step (int, default: 1 ) –

The number of invocations of the step wrapper before optimizer.step was called. Generally can be left as 1 (default). For older amp pathways, this is the number of times the scaler wrapper was invoked before the wrapped optimizer step function was called to handle accumulation in fp16.
global_step (int, default: 0 ) –

The current global step.
global_batch (int, default: 0 ) –

The current global batch.

Methods:

new_instance –

Creates a new instance of the event with the provided keyword arguments.
should_update –

Determines if the event should trigger an update.

Attributes:

current_index (float) –

Calculates the current index of the event.
epoch (int) –

Calculates the current epoch.
epoch_based (bool) –

Determines if the event is based on epochs.
epoch_batch (int) –

Calculates the current batch within the current epoch.
epoch_full (float) –

Calculates the current epoch with the fraction of the current step.
epoch_step (int) –

Calculates the current step within the current epoch.

current_index `property` `writable`

current_index: float

Calculates the current index of the event.

Returns:

float –

The current index of the event, which is either the global step or the epoch with the fraction of the current step.

Raises:

ValueError –

if the event is not epoch based or if the steps per epoch are too many.

epoch `property`

epoch: int

Calculates the current epoch.

Returns:

int –

The current epoch.

Raises:

ValueError –

if the event is not epoch based.

epoch_based `property`

epoch_based: bool

Determines if the event is based on epochs.

Returns:

bool –

True if the event is based on epochs, False otherwise.

epoch_batch `property`

epoch_batch: int

Calculates the current batch within the current epoch.

Returns:

int –

The current batch within the current epoch.

Raises:

ValueError –

if the event is not epoch based.

epoch_full `property`

epoch_full: float

Calculates the current epoch with the fraction of the current step.

Returns:

float –

The current epoch with the fraction of the current step.

Raises:

ValueError –

if the event is not epoch based.

epoch_step `property`

epoch_step: int

Calculates the current step within the current epoch.

Returns:

int –

The current step within the current epoch.

Raises:

ValueError –

if the event is not epoch based.

new_instance

new_instance(**kwargs) -> Event

Creates a new instance of the event with the provided keyword arguments.

Parameters:

kwargs –

Keyword arguments to set in the new instance.

Returns:

Event –

A new instance of the event with the provided kwargs.

Source code in src/llmcompressor/core/events/event.py

def new_instance(self, **kwargs) -> "Event":
    """
    Creates a new instance of the event with the provided keyword arguments.

    :param kwargs: Keyword arguments to set in the new instance.
    :return: A new instance of the event with the provided kwargs.
    :rtype: Event
    """
    logger.debug("Creating new instance of event with kwargs: {}", kwargs)
    instance = deepcopy(self)
    for key, value in kwargs.items():
        setattr(instance, key, value)
    return instance

should_update

should_update(
    start: Optional[float],
    end: Optional[float],
    update: Optional[float],
) -> bool

Determines if the event should trigger an update.

Parameters:

start (Optional[float]) –

The start index to check against, set to None to ignore start.
end (Optional[float]) –

The end index to check against, set to None to ignore end.
update (Optional[float]) –

The update interval, set to None or 0.0 to always update, otherwise must be greater than 0.0, defaults to None.

Returns:

bool –

True if the event should trigger an update, False otherwise.

Source code in src/llmcompressor/core/events/event.py

def should_update(
    self, start: Optional[float], end: Optional[float], update: Optional[float]
) -> bool:
    """
    Determines if the event should trigger an update.

    :param start: The start index to check against, set to None to ignore start.
    :type start: Optional[float]
    :param end: The end index to check against, set to None to ignore end.
    :type end: Optional[float]
    :param update: The update interval, set to None or 0.0 to always update,
        otherwise must be greater than 0.0, defaults to None.
    :type update: Optional[float]
    :return: True if the event should trigger an update, False otherwise.
    :rtype: bool
    """
    current = self.current_index
    logger.debug(
        "Checking if event should update: "
        "current_index={}, start={}, end={}, update={}",
        current,
        start,
        end,
        update,
    )
    if start is not None and current < start:
        return False
    if end is not None and current > end:
        return False
    return update is None or update <= 0.0 or current % update < 1e-10

EventType

Bases: Enum

An Enum for defining the different types of events that can be triggered during model compression lifecycles. The purpose of each EventType is to trigger the corresponding modifier callback during training or post training pipelines.

Parameters:

INITIALIZE –

Event type for initialization.
FINALIZE –

Event type for finalization.
BATCH_START –

Event type for the start of a batch.
LOSS_CALCULATED –

Event type for when loss is calculated.
BATCH_END –

Event type for the end of a batch.
CALIBRATION_EPOCH_START –

Event type for the start of a calibration epoch.
SEQUENTIAL_EPOCH_END –

Event type for the end of a layer calibration epoch, specifically used by src/llmcompressor/pipelines/sequential/pipeline.py
CALIBRATION_EPOCH_END –

Event type for the end of a calibration epoch.
OPTIM_PRE_STEP –

Event type for pre-optimization step.
OPTIM_POST_STEP –

Event type for post-optimization step.

llmcompressor.core.events

Event dataclass

current_index property writable

epoch property

epoch_based property

epoch_batch property

epoch_full property

epoch_step property

new_instance

should_update

EventType

Event `dataclass`

current_index `property` `writable`

epoch `property`

epoch_based `property`

epoch_batch `property`

epoch_full `property`

epoch_step `property`