vllm.model_executor.layers.quantization.utils.humming.priority

Humming kernel selection priority.

Functions:

• prefers_humming –

  Whether Humming outranks Marlin. A missing compute capability uses the

• prioritize_humming –

  Move Humming directly ahead of Marlin where Humming is preferred.

prefers_humming(compute_capability=None)

Whether Humming outranks Marlin. A missing compute capability uses the current device.

Source code in vllm/model_executor/layers/quantization/utils/humming/priority.py

def prefers_humming(compute_capability: int | None = None) -> bool:
    """Whether Humming outranks Marlin. A missing compute capability uses the
    current device."""
    if compute_capability is None:
        from vllm.platforms import current_platform

        if current_platform.is_cuda():
            cc = current_platform.get_device_capability()
            compute_capability = cc.to_int() if cc is not None else None
    return compute_capability in _HUMMING_PREFERRED_CAPABILITIES

prioritize_humming(kernels, compute_capability=None)

Move Humming directly ahead of Marlin where Humming is preferred.

Every other entry keeps its relative order and the input list is not modified. Match kernel class names or MoE backend enum names.

Source code in vllm/model_executor/layers/quantization/utils/humming/priority.py

def prioritize_humming(
    kernels: list[_KernelT],
    compute_capability: int | None = None,
) -> list[_KernelT]:
    """Move Humming directly ahead of Marlin where Humming is preferred.

    Every other entry keeps its relative order and the input list is not
    modified. Match kernel class names or MoE backend enum names.
    """
    if not prefers_humming(compute_capability):
        return kernels

    names = [
        (kernel.name if isinstance(kernel, Enum) else kernel.__name__).lower()
        for kernel in kernels
    ]
    humming = next((i for i, name in enumerate(names) if "humming" in name), None)
    marlin = next((i for i, name in enumerate(names) if "marlin" in name), None)
    if humming is None or marlin is None or humming < marlin:
        return kernels
    kernels = kernels.copy()
    kernels.insert(marlin, kernels.pop(humming))
    return kernels