CPU 绑定¶

Starting from vllm-ascend v0.18.0rc1, CPU binding is enabled by default on ARM-based Ascend servers.

You usually do not need to configure it manually. Set enable_cpu_binding only when you want to disable it or make the default explicit.

CPU 绑定的优势¶

CPU 绑定 improves host-side scheduling for multi-socket ARM servers with Ascend NPUs. It is designed to solve three common host-side inference performance issues:

Lower cross-NUMA traffic. Worker processes stay closer to the CPU and memory resources selected for their active NPU, reducing remote NUMA access.
Lower context-switch overhead from thread preemption. Key runtime threads run on stable CPU ranges, reducing scheduler movement and CPU contention on busy hosts.
Better latency stability and multi-worker isolation. Independent workers avoid sharing the same CPU/NUMA resources, which helps reduce tail-latency jitter and makes throughput more predictable during multi-NPU serving.

This feature is a host-side performance optimization. It does not change model execution logic or numerical outputs. When memory migration support is unavailable, CPU affinity still works, but memory locality may be worse and latency or throughput may degrade.

使用方法¶

在线服务¶

默认行为：

vllm serve Qwen/Qwen2.5-7B-Instruct

禁用 CPU 绑定：

vllm serve Qwen/Qwen2.5-7B-Instruct \
  --additional-config '{"enable_cpu_binding": false}'

离线推理¶

默认行为：

from vllm import LLM

llm = LLM(model="Qwen/Qwen2.5-7B-Instruct")

禁用 CPU 绑定：

from vllm import LLM

llm = LLM(
    model="Qwen/Qwen2.5-7B-Instruct",
    additional_config={"enable_cpu_binding": False},
)

要求¶

Official vllm-ascend images have already included util-linux and procps / procps-ng in v0.18.0rc1 and earlier releases. Starting from v0.18.0rc1, the official images also include numactl.

如果您未使用官方镜像，请手动安装主机工具：

# Ubuntu/Debian
sudo apt-get install -y util-linux numactl procps

# RHEL/CentOS/Alma/Rocky
sudo yum install -y util-linux numactl procps-ng

# openEuler
sudo dnf install -y util-linux numactl procps-ng

Without numactl / migratepages, vLLM Ascend skips only memory migration. The worker process and runtime threads are still pinned, but pages already placed on remote NUMA nodes are not migrated, which can reduce locality and degrade latency or throughput.

For optimal locality, use a cpuset that is evenly distributed across NUMA nodes. Unbalanced cpusets may reduce the locality benefit of CPU binding.

On Ascend 950, CPU binding uses deterministic global CPU slicing because Ascend 950 does not report NPU-to-CPU affinity in npu-smi info -t topo. Ascend 950 still pins worker, ACL, and release threads and can still migrate memory pages when migratepages is available. Because Ascend 950 skips IRQ binding, it does not reserve the first two CPUs in each NPU pool for IRQ handling. Those CPUs are assigned to the main worker instead.

For IRQ binding, the process also needs permission to read /proc/interrupts and write /proc/irq/*/smp_affinity. If irqbalance is running and the process can use systemctl, vLLM Ascend stops it before applying IRQ affinity. In containers where systemctl is unavailable, stop irqbalance on the host when IRQ affinity matters.

Ascend 950 does not apply IRQ binding. When running on Ascend 950, the log contains [irq] IRQ binding skipped on Ascend 950. and no /proc/irq/*/smp_affinity files are written by this feature.

On the host, stop irqbalance before starting vLLM when you need stable IRQ affinity:

sudo systemctl stop irqbalance

After the vLLM service exits, restart it if the host should return to the default IRQ balancing policy:

sudo systemctl start irqbalance

常见问题与故障排查¶

消息	含义	操作
`CPU binding skipped: non-ARM CPU detected.`	CPU 绑定仅在 ARM 上运行。	在 x86_64 上无需任何操作。
`Can not get running npu info.`	未找到正在运行的 NPU，或者 `ASCEND_RT_VISIBLE_DEVICES` 过滤掉了所有 NPU。	检查可见的 NPU ID 和 `npu-smi info`。
`Insufficient CPUs for binding...`	可用 CPU 少于角色拆分所需的数量。具有 IRQ 绑定的设备每个逻辑 NPU 至少需要 5 个 CPU；Ascend 950 至少需要 3 个。	扩大 cpuset 或减少可见的 NPU。
`NPU topo affinity not found...`	拓扑亲和性不可用。	vLLM Ascend 回退到 `global_slice`；仅当此设备上预期有拓扑亲和性时，才检查 `npu-smi info -t topo`。
`The 'migratepages' command is not available...`	内存迁移被跳过，但 CPU 线程绑定仍会继续。	如果 NUMA 局部性或性能受到影响，请安装 `numactl`。
`[irq] IRQ binding skipped on Ascend 950.`	Ascend 950 不使用 IRQ 绑定步骤。	无需任何操作。Worker、ACL、release 线程绑定和内存迁移仍会继续。
`Bind cpus failed in rank...`	某个绑定步骤失败，该 rank 的 CPU 绑定被跳过。	检查 `taskset`、`lscpu`、`npu-smi`、cpuset 大小和 `/proc/irq` 权限。