跳转至

CPU 绑定

概述

CPU binding is an Ascend-native host-side optimization for vLLM workers on ARM servers. Starting from vllm-ascend v0.18.0rc1, it is enabled by default through enable_cpu_binding=True.

The feature does not change model execution logic or numerical results. It only controls CPU placement for the worker process, key runtime threads, memory pages, and NPU IRQs when the host environment allows it. By keeping the main worker, ACL, and release threads on dedicated CPU ranges, it helps reduce context-switch overhead from scheduler preemption on busy hosts.

为什么需要 CPU 绑定？

On multi-socket ARM systems, the Linux scheduler may place worker threads on CPU far from the NPU that the worker drives. This can increase cross-NUMA traffic, increase thread preemption, and introduce latency jitter. The Ascend backend therefore owns a CPU allocation policy to reduce cross-NUMA traffic, reduce thread preemption, and improve latency stability instead of relying on upstream GPU NUMA binding flags.

这也是上游 NUMA 标志在 Ascend 上被适配的原因：

  • --numa-bind 被转换为 additional_config={"enable_cpu_binding": true}。
  • --numa-bind-nodes 和 --numa-bind-cpus 被忽略，因为 Ascend 根据 NPU 拓扑或全局逻辑 NPU ID 计算 CPU 池。

工作原理

分配器根据运行时主机状态推导其计划：

输入 来源 用途
允许的 CPU /proc/self/status Cpus_allowed_list 唯一有资格绑定的 CPU。容器 cpuset 会被尊重。
逻辑 NPU 映射 npu-smi info -m 将卡/芯片 ID 映射到全局逻辑 NPU ID，并提供 total_logic_npus。在 Ascend 950 上，Chip Logic ID 未报告，因此使用 NPU ID 作为逻辑 ID。
运行中的 NPU npu-smi info 进程表，经 ASCEND_RT_VISIBLE_DEVICES 过滤 识别此工作进程使用的逻辑 NPU。A2/A3 进程行使用 NPU Chip；Ascend 950 进程行使用 NPU ID。
拓扑亲和性 npu-smi info -t topo 为 topo_affinity 模式提供 NPU 到 CPU 的亲和性。
CPU NUMA 映射 lscpu -e=CPU,NODE 用于将单 NUMA 亲和性池扩展到下一个 NUMA 节点。

策略选择

绑定策略根据 Ascend 设备类型选择：

设备类型 策略 原因
A3 global_slice A3 使用 HCCS 卡间互联。每个 NPU 与所有 NUMA 节点的距离几乎相等，因此没有强烈的 NPU 到 NUMA 亲和性信号。基于全局逻辑 NPU ID 的分片提供了确定性的、不重叠的 CPU 池，以及工作进程间的 CPU/NUMA 隔离。
Ascend 950 global_slice Ascend 950 报告 NPU 到 NPU/NIC 的拓扑，但不在 npu-smi info -t topo 中报告 NPU 到 CPU 的亲和性。它还通过 NPU ID 而非 NPU Chip 报告进程行。基于全局逻辑 NPU ID 的分片在不依赖缺失亲和性数据的情况下保持 CPU 池的确定性。
A2 和 Atlas 300 推理产品 topo_affinity A2 和 Atlas 300 推理产品通过 npu-smi info -t topo 提供 NPU 到 CPU 的亲和性信息，因此它们在可用时使用此拓扑信号。

如果选择了 topo_affinity 但拓扑亲和性不可用，分配器将回退到 global_slice。

CPU 池构建

global_slice

global_slice is designed for devices without a useful NPU-to-CPU affinity signal, including A3 and Ascend 950. Because A3's HCCS interconnect makes the distance from each NPU to each NUMA node nearly the same, topology affinity is not a useful placement signal. Ascend 950 similarly exposes UB/NIC topology but not CPU affinity. The allocator therefore partitions the sorted allowed_cpus list by global logical NPU ID.

  1. 按以下顺序确定 total_npus：
  2. 来自 total_logic_npus 的 npu-smi info -m
  3. 拓扑亲和性条目数量
  4. 运行中 NPU 数量
  5. 计算：
  6. base = len(allowed_cpus) // total_npus
  7. extra = len(allowed_cpus) % total_npus
  8. 每个逻辑 NPU 获得一个确定性的分片：
  9. NPU ID < extra 获得 base + 1 个 CPU。
  10. 其余 NPU ID 获得 base 个 CPU。
  11. 只有运行中的 NPU 会被实例化到 npu_cpu_pool 中。

This is the key property: two independent worker processes with the same cpuset but different visible NPU IDs still get non-overlapping CPU池s because both processes slice against the same global NPU ID space. With a NUMA-aligned cpuset, this also provides CPU/NUMA isolation between workers, so one worker does not share the same CPU or NUMA slice with another worker.

global_slice 需要足够的 CPU 来满足所选设备的角色拆分：

  • Devices with IRQ binding require base >= 5: 2 CPU for SQ/CQ中断 binding, at least 1 CPU for the main worker, 1 CPU for ACL线程, and 1 CPU for release thread.
  • Ascend 950 skips IRQ binding and does not reserve SQ/CQ中断 CPU, so it requires base >= 3: at least 1 CPU for the main worker, 1 CPU for ACL线程, and 1 CPU for release thread.

topo_affinity

topo_affinity is designed for A2, Atlas 300 inference products, and other non-A3 device types. A2 和 Atlas 300 推理产品 expose meaningful NPU-to-CPU affinity information, so the allocator starts from NPU topology affinity when it is available and then avoids overlap for shared affinity groups.

  1. 从所有逻辑NPU构建候选NPU：
  2. 始终包含运行中的NPU
  3. 仅当非运行中NPU的亲和性与该进程的允许cpuset重叠时才包含它们
  4. 对于每个候选NPU，将拓扑亲和性与allowed_cpus取交集。
  5. 如果某个候选NPU的交集为空，则此rank的绑定失败。
  6. 如果亲和性CPU都在一个NUMA节点上，则使用来自下一个NUMA节点的CPU扩展池，受allowed_cpus约束。
  7. 将具有相同扩展池的NPU分组，并在该组内均匀分割每个共享池。
  8. 在最终的npu_cpu_pool中仅保留运行中的NPU。

The non-running candidate step is intentional. It prevents two independent single-card workers from selecting the same CPU range when their visible NPUs share the same topology affinity.

角色拆分

构建CPU池后，分配器按角色进行拆分：

对于支持IRQ绑定的设备：

角色 CPU
SQ/CQ中断 pool[0]、pool[1]
主工作进程及子线程 pool[2:-2]
ACL线程 pool[-2]
释放线程 pool[-1]

对于Ascend 950：

角色 CPU
主工作进程及子线程 pool[:-2]
ACL线程 pool[-2]
释放线程 pool[-1]

If a final pool has fewer CPU than the selected role split requires, binding fails for this rank and the worker logs a warning from the caller. The minimum is 5 CPU per NPU for devices with IRQ binding, and 3 CPU per NPU for Ascend 950.

条件性主机调优

After CPU affinity is applied, CPU binding can also apply two host-side tuning steps when the environment supports them:

  • Memory migration uses migratepages to move the worker process's existing pages to the selected NUMA node. This keeps the worker closer to the memory it reads and reduces remote-NUMA memory read latency.
  • IRQ binding places NPU IRQ handling on the CPU reserved for the corresponding NPU when /proc/irq is writable and IRQ files can be resolved. Ascend 950 skips this step and gives those CPU to the main worker instead.

These are conditional parts of CPU binding, not separate feature switches. If a host prerequisite is missing, that step is skipped while CPU thread binding still proceeds. Missing migratepages can still leave pages on remote NUMA nodes, so latency or throughput may regress compared with a full CPU binding setup.

示例

具有640个CPU和16个NPU的A3推理服务器

输入：

  • allowed_cpus = [0..639]
  • total_logic_npus = 16
  • running_npu_list = [0..15]

计算：

  • base = 640 // 16 = 40
  • extra = 0
  • 工作进程 i driving logical NPU i receives CPU slice [i * 40 .. i * 40 + 39].

全局切片视图：

CPU range: 0                                                             639
           |-- worker0/NPU0 --|-- worker1/NPU1 --| ... |-- worker15/NPU15 --|
           |      0-39        |      40-79       | ... |      600-639       |

每个工作进程切片内的角色拆分：

40-CPU worker slice
| IRQ CPUs | main worker process and subthreads | ACL thread | release thread |
|  c0-c1   |              c2-c37                |    c38     |      c39       |

具体示例：

工作进程 逻辑NPU CPU池 中断CPU 主CPU ACL CPU 释放CPU
0 0 0-39 0-1 2-37 38 39
1 1 40-79 40-41 42-77 78 79
... ... ... ... ... ... ...
15 15 600-639 600-601 602-637 638 639

This layout remains deterministic even when different worker processes share the same cpuset, because slicing is based on the global logical NPU ID.

具有隐藏相同亲和性NPU的A2拓扑亲和性

来自A2拓扑的输入：

  • NPU0亲和性：144-167
  • NPU2亲和性：144-167
  • 进程A仅看到NPU0
  • 进程B仅看到NPU2
  • 两个进程都有allowed_cpus = [144..191]

The allocator includes the hidden same-affinity NPU as a candidate in each process, splits the shared extended pool, and then keeps only the visible NPU in the final pool.

最终池：

进程 可见NPU 最终CPU池
A 0 144-167
B 2 168-191

即使两个工作进程作为独立的单卡服务启动，这也能避免CPU池重叠。

日志

分配器记录所选模式和分配计划：

[cpu_bind_mode] mode=topo_affinity rank=0 visible_npus=[0]
The CPU allocation plan is as follows:
NPU0: main=[...] acl=[...] release=[...]

限制

  • CPU绑定仅在ARM上运行，在x86_64上跳过。
  • Each final NPU pool must have enough CPU for its role split: at least 5 CPU for devices with IRQ binding, and at least 3 CPU for Ascend 950.
  • global_slice is deterministic and provides CPU/NUMA isolation when the cpuset is NUMA-aligned, but it cannot guarantee NUMA-local pools when CPU numbering or cpuset layout crosses NUMA boundaries.
  • topo_affinity依赖于npu-smi info -t topo的可用输出。
  • IRQ binding requires writable /proc/irq and resolvable PCI/IRQ information. Ascend 950 skips IRQ binding even when /proc/irq is writable, and does not reserve 中断CPU in its role split.
  • Memory migration requires migratepages; otherwise only memory migration is skipped. CPU affinity still applies, but performance may degrade because existing pages are not moved to the target NUMA node and may be read through higher-latency remote NUMA access.
  • 如果异常逃逸出绑定流程，NPUWorker会记录警告并跳过该rank的CPU绑定。

参考

  • 实现：vllm_ascend/cpu_binding.py
  • 工作进程集成：vllm_ascend/worker/worker.py
  • 配置：vllm_ascend/ascend_config.py 和 docs/source/user_guide/configuration/additional_config.md
  • 测试：tests/ut/device_allocator/test_cpu_binding.py