llmcompressor.modifiers.pruning.reap.base
REAP (Router-weighted Expert Activation Pruning) modifier for MoE models.
See: https://arxiv.org/abs/2510.13999
Classes:
-
REAPPruningModifier–Prunes experts from MoE layers using the REAP saliency metric. For each
REAPPruningModifier
Bases: ExpertPruningModifierBase
Prunes experts from MoE layers using the REAP saliency metric. For each
expert j the saliency is
``S_j = mean(g_j * ||f_j||_2)``
averaged over the tokens routed to expert j, where:
g_jis the router gate weight assigned to expertj(the coefficient that multiplies the expert's output when combining experts), andf_jis expertj's output activation for that token, so||f_j||_2is its L2 norm.
Parameters:
-
sparsity–fraction of experts to remove per layer, in
[0, 1). When0.0, no experts are pruned; saliency is still computed so that areport_pathcan be written, but the model weights and config are never modified. -
ignore–module name patterns to skip during MoE layer pruning.
-
report_path–optional path to a
.jsonfile where the per-expert statistics are written after calibration completes. The report is a JSON object mappingcountandsaliencyto one inner list per MoE layer (ordered by layer). Usetools/plot_reap_report.pyto generate a saliency visualization. Required whensparsityis0.0, since the report is then the modifier's only output.Example recipe::
REAPPruningModifier: sparsity: 0.25
Methods:
-
expert_hook–Record the per-token L2 norm of the expert's output.
-
experts_block_hook–Add each routed token's contribution to the layer's statistics.
-
init_moe_stats–Allocate the REAP accumulators for a layer.
-
reduce_moe_stats–Reduce a layer's statistics across ranks, in place.
-
saliency–Return the mean saliency
S_jper expert.
expert_hook
Record the per-token L2 norm of the expert's output.
Upcasts to the modifier's precision and stores into the layer's
per-batch norm buffer, for :meth:experts_block_hook to consume at
the end of the batch.
Parameters:
-
moe_name(str) –Name of the MoE layer the expert belongs to
-
expert_idx(int) –Index of the expert within its layer
-
module(ExpertMLP) –The expert module the hook is attached to
-
args(tuple) –The expert's forward arguments
-
output(Tensor) –The expert's output for the current batch
Source code in src/llmcompressor/modifiers/pruning/reap/base.py
experts_block_hook
Add each routed token's contribution to the layer's statistics.
Accumulates topk_weight * ||f_j|| into the per-expert saliency
sums and bumps the per-expert routed-token counts. Two token layouts
are handled: under the calibrate-all-experts flag every expert is
evaluated for every token, so norms are gathered by top-k index;
otherwise indices are flattened in (slot, token) order to match the
torch.where scatter order of LinearExperts2D. Resets the
per-batch norm buffers afterwards.
Parameters:
-
moe_name(str) –Name of the MoE layer the block belongs to
-
module(Module) –The experts block the hook is attached to
-
args(tuple) –The block's forward arguments, where
args[1]andargs[2]are the router's top-k indices and weights -
output(Tensor) –The block's output for the current batch
Source code in src/llmcompressor/modifiers/pruning/reap/base.py
129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 | |
init_moe_stats
Allocate the REAP accumulators for a layer.
Creates the per-expert saliency sums and routed-token counts, plus
the per-batch expert-norm buffers filled by :meth:expert_hook.
Parameters:
-
moe_name(str) –Name of the MoE layer
-
num_experts(int) –Number of experts in the layer
-
hidden_dim(int) –Unused by REAP
-
device(device) –The layer's execution device
Source code in src/llmcompressor/modifiers/pruning/reap/base.py
reduce_moe_stats
Reduce a layer's statistics across ranks, in place.
Reduces the saliency sums to the source rank, which alone computes the pruning decision and report from them, and all-reduces the per-expert counts so every rank agrees on whether the layer saw calibration data.
Parameters:
-
moe_name(str) –Name of the MoE layer whose stats to reduce
Source code in src/llmcompressor/modifiers/pruning/reap/base.py
saliency
Return the mean saliency S_j per expert.
The router-weighted mean of the expert's output norms over the tokens routed to it.
Parameters:
-
moe_name(str) –Name of the MoE layer
Returns:
-
Tensor–Per-expert saliency, indexed by expert