Design Documents

This section provides an overview of the features implemented in vLLM Ascend. Developers can refer to this guide to understand how vLLM Ascend works.

Design Documents

• Patch in vLLM Ascend
• CPU Binding
• Prepare inputs for model forwarding
• Disaggregated-prefill
• Expert Parallelism Load Balancer (EPLB)
• ACL Graph
• KV Cache Pool
• Adding a custom aclnn operation
• Context Parallel (CP)
• Dynamic Chunked Pipeline Parallel (CPP)
• Quantization Adaptation Guide
• Npugraph_ex