GLM-5.3 (Experimental)

1 Introduction

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks.

This document will show the main verification steps of the model, including supported features, feature configuration, environment preparation, multi-node deployment, accuracy and performance evaluation.

Warning

Current status and constraints

• GLM-5.3 has only been tested on the official Docker image quay.io/ascend/vllm-ascend:v0.23.0-a3 and quay.io/ascend/vllm-ascend:v0.23.0 only on multi-node co-located scenario.
• The features listed in Supported Features are only those enabled by the verified deployment commands in this document, and do not imply that all features are supported for GLM-5.3. This is an early-access version; performance optimization and reliability validation are still in progress (see Declaration).
• All the scripts below is based on v0.23.0, so some params are not supported in main code. If you are using the main branch of vllm-Ascend, please make sure to check it.

2 Supported Features

Refer to Supported Features List to get the model's supported feature matrix.

Refer to Feature Guide to get the feature's configuration.

3 Prerequisites

3.1 Model Weight

• GLM-5.3-w8a8c8: requires 2 Atlas 800 A3 (128GB × 8) node or 4 Atlas 800 A2 (64G × 32).Download model weight.
• You can use msmodelslim to quantize the model directly.

It is recommended to download the model weight to the shared directory of multiple nodes, such as /root/.cache/

3.2 Verify Multi-node Communication (Optional)

If you want to deploy multi-node environment, you need to verify multi-node communication according to verify multi-node communication environment.

4 Installation

4.1 Docker Image Installation

• You can use our official docker image to run GLM-5.3 directly.

A3 seriesA2 series

Start the docker image on each node.

export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0-a3
export NAME=vllm-ascend

# Run the container using the defined variables
# Note: If you are running bridge network with docker, please expose available ports for multiple nodes communication in advance
docker run --rm \
--name $NAME \
--net=host \
--shm-size=1g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci4 \
--device /dev/davinci5 \
--device /dev/davinci6 \
--device /dev/davinci7 \
--device /dev/davinci8 \
--device /dev/davinci9 \
--device /dev/davinci10 \
--device /dev/davinci11 \
--device /dev/davinci12 \
--device /dev/davinci13 \
--device /dev/davinci14 \
--device /dev/davinci15 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \
-it $IMAGE bash

Start the docker image on each of your nodes.

export IMAGE=quay.io/ascend/vllm-ascend:v0.23.0
docker run --rm \
    --name vllm-ascend \
    --shm-size=1g \
    --net=host \
    --device /dev/davinci0 \
    --device /dev/davinci1 \
    --device /dev/davinci2 \
    --device /dev/davinci3 \
    --device /dev/davinci4 \
    --device /dev/davinci5 \
    --device /dev/davinci6 \
    --device /dev/davinci7 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -v /root/.cache:/root/.cache \
    -it $IMAGE bash

If you want to deploy multi-node environment, you need to set up environment on each node.

4.2 Source Code Installation

If you don't want to use the docker image as above, you can also build all from source:

• Install vllm-ascend from source, refer to installation.

5 Online Service Deployment

The deployment scenarios validated for this release are organized by context window size (below 1M), hardware (Atlas 800 A3 / A2), and deployment mode (multi-node co-located). All startup scripts below are the verified reference commands; key parameters are explained after each scenario.

Warning

• The scripts below is tested on v0.23.0, some params may have changed in main branch.

5.1 Multi-node Deployment

If you want to deploy multi-node environment, you need to verify multi-node communication according to verify multi-node communication environment.

Common Issues Tip: If you encounter issues, Refer to Public FAQs.

5.1.1 Context Below 1M

A3 seriesA2 series

• GLM-5.3-w8a8c8: can be deployed on 2 Atlas 800 A3 (64GB × 16).

Run the following scripts on two nodes respectively.

node 0

# this obtained through ifconfig
# nic_name is the network interface name corresponding to local_ip of the current node
nic_name="xxx"
local_ip="xxx"

# The value of node0_ip must be consistent with the value of local_ip set in node0 (master node)
node0_ip="xxxx"

export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export HCCL_TRANSFER_TIMEOUT=600
export HCCL_EXEC_TIMEOUT=3600
export HCCL_CONNECT_TIMEOUT=3600
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=1
export HCCL_BUFFSIZE=400
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export VLLM_ASCEND_ENABLE_MLAPO=1

vllm serve /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.3-w8a8c8 \
    --host 0.0.0.0 \
    --port 8077 \
    --safetensors-load-strategy prefetch \
    --api-server-count 1 \
    --data-parallel-size 8 \
    --data-parallel-start-rank 0 \
    --data-parallel-size-local 4 \
    --data-parallel-address $node0_ip \
    --data-parallel-rpc-port 12980 \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --seed 1024 \
    --served-model-name glm-5 \
    --tool-call-parser glm47 \
    --reasoning-parser glm45 \
    --enable-auto-tool-choice \
    --max-num-seqs 6 \
    --max-model-len 202752 \
    --max-num-batched-tokens 4096 \
    --trust-remote-code \
    --gpu-memory-utilization 0.90 \
    --quantization ascend \
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
    --additional-config '{"enable_dsa_cp": true, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_balance_scheduling": true, "enable_fused_mc2": 1, "enable_flashcomm1": true}'  \
    --speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}'

node 1

# this obtained through ifconfig
# nic_name is the network interface name corresponding to local_ip of the current node
nic_name="xxx"
local_ip="xxx"

# The value of node0_ip must be consistent with the value of local_ip set in node0 (master node)
node0_ip="xxxx"

export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export HCCL_TRANSFER_TIMEOUT=600
export HCCL_EXEC_TIMEOUT=3600
export HCCL_CONNECT_TIMEOUT=3600
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=1
export HCCL_BUFFSIZE=400
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export VLLM_ASCEND_ENABLE_MLAPO=1

vllm serve /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.3-w8a8c8 \
    --host 0.0.0.0 \
    --port 8077 \
    --headless \
    --data-parallel-size 8 \
    --data-parallel-start-rank 4 \
    --data-parallel-size-local 4 \
    --data-parallel-address $node0_ip \
    --data-parallel-rpc-port 12980 \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --seed 1024 \
    --served-model-name glm-5 \
    --tool-call-parser glm47 \
    --reasoning-parser glm45 \
    --enable-auto-tool-choice \
    --max-num-seqs 6 \
    --max-model-len 202752 \
    --max-num-batched-tokens 4096 \
    --trust-remote-code \
    --gpu-memory-utilization 0.92 \
    --quantization ascend \
    --enable-chunked-prefill \
    --enable-prefix-caching \
    --async-scheduling \
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
    --additional-config '{"enable_dsa_cp": true, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_balance_scheduling": true, "enable_fused_mc2": 1, "enable_flashcomm1": true}' \
    --speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}'

• GLM-5.3-w8a8c8: can be deployed on 4 Atlas 800 A2 (64GB × 32).

Run the following scripts on four nodes respectively.

node 0

# this obtained through ifconfig
# nic_name is the network interface name corresponding to local_ip of the current node
nic_name="xxx"
local_ip="xxx"

# The value of node0_ip must be consistent with the value of local_ip set in node0 (master node)
node0_ip="xxx"

export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export VLLM_RPC_TIMEOUT=360000
export VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=30000
export HCCL_EXEC_TIMEOUT=200
export HCCL_CONNECT_TIMEOUT=120
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=10
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export ACL_OP_INIT_MODE=1
export CPU_AFFINITY_CONF=1
export VLLM_ASCEND_ENABLE_MLAPO=1
export VLLM_ENGINE_READY_TIMEOUT_S=1200

vllm serve /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.3-w8a8c8 \
    --host 0.0.0.0 \
    --port 8077 \
    --max-model-len 135000 \
    --data-parallel-size 4 \
    --data-parallel-size-local 1 \
    --data-parallel-start-rank 0 \
    --data-parallel-address "${node0_ip}" \
    --data-parallel-rpc-port 12980 \
    --tensor-parallel-size 8 \
    --enable-expert-parallel \
    --seed 1024 \
    --served-model-name glm-5 \
    --safetensors-load-strategy prefetch \
    --max-num-seqs 128 \
    --max-num-batched-tokens 8192 \
    --trust-remote-code \
    --quantization ascend \
    --gpu-memory-utilization 0.92 \
    --speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
    --additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true, "fuse_muls_add": true, "multistream_overlap_shared_expert": true, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_flashcomm1": true}' \
    --enable-prefix-caching \
    --async-scheduling \
    --api-server-count 1

node 1-3

# this obtained through ifconfig
# nic_name is the network interface name corresponding to local_ip of the current node
nic_name="xxx"
local_ip="xxx"

# The value of node0_ip must be consistent with the value of local_ip set in node0 (master node)
node0_ip="xxx"

# node1: dp_start_rank=1, node2: dp_start_rank=2, node3: dp_start_rank=3
dp_start_rank=1

export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export VLLM_RPC_TIMEOUT=360000
export VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=30000
export HCCL_EXEC_TIMEOUT=200
export HCCL_CONNECT_TIMEOUT=120
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=10
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export ACL_OP_INIT_MODE=1
export CPU_AFFINITY_CONF=1
export VLLM_ASCEND_ENABLE_MLAPO=1
export VLLM_ENGINE_READY_TIMEOUT_S=1200

vllm serve /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.3-w8a8c8 \
    --host 0.0.0.0 \
    --port 8077 \
    --headless \
    --max-model-len 135000 \
    --data-parallel-size 4 \
    --data-parallel-size-local 1 \
    --data-parallel-start-rank ${dp_start_rank} \
    --data-parallel-address "${node0_ip}" \
    --data-parallel-rpc-port 12980 \
    --tensor-parallel-size 8 \
    --enable-expert-parallel \
    --seed 1024 \
    --served-model-name glm-5 \
    --safetensors-load-strategy prefetch \
    --max-num-seqs 128 \
    --max-num-batched-tokens 8192 \
    --trust-remote-code \
    --quantization ascend \
    --gpu-memory-utilization 0.92 \
    --speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
    --additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true,"fuse_muls_add": true, "multistream_overlap_shared_expert": true, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_flashcomm1": true}' \
    --enable-prefix-caching \
    --async-scheduling

Key Parameter Descriptions:

Only the key parameters specific to this model/scenario are described below. max-model-len and max-num-seqs need to be set according to the actual usage scenario.

Multi-node network and data parallel configuration:

• HCCL_IF_IP, GLOO_SOCKET_IFNAME, TP_SOCKET_IFNAME, HCCL_SOCKET_IFNAME: Network interface configuration for multi-node communication. Set nic_name to the network interface name (obtained via ifconfig) and local_ip to the current node's IP address. These must be correctly configured on each node for successful multi-node communication.
• --data-parallel-size 8: Total number of data parallel ranks across all nodes (4 ranks per node in this scenario).
• --data-parallel-size-local 4: Number of data parallel ranks on the current node.
• --data-parallel-start-rank: Starting rank offset for data parallel ranks on this node. Node 0 uses 0, node 1 uses 4.
• --data-parallel-address: IP address of the data parallel master node (node 0). Must match the local_ip of the master node.
• --data-parallel-rpc-port 12980: RPC port for data parallel master communication. Must be the same across all nodes.
• --headless: Indicates a non-master node (used on node 1-3). Do not use on node 0.

A2-specific environment variables:

• CPU_AFFINITY_CONF=1: Enables CPU core affinity binding for worker processes.
• ACL_OP_INIT_MODE=1: ACL operator initialization mode to speed up operator compilation.
• VLLM_RPC_TIMEOUT=360000 / VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=3000 / HCCL_EXEC_TIMEOUT=200 / HCCL_CONNECT_TIMEOUT=120 / VLLM_ENGINE_READY_TIMEOUT_S=1200: Timeout settings for multi-node startup and model execution on the slower A2 platform. Increase them if the engine fails to become ready in time.

Notice: This scenario enables additional_config.enable_fused_mc2=1 (fused dispatch_ffn_combine/mega_moe operators). Fused MC2 conflicts with multistream_overlap_shared_expert — the two optimizations must not be enabled at the same time (the runtime forcibly disables multistream_overlap_shared_expert when fused MC2 is on).

5.1.2 1M Context Deployment

The 1M context scenarios have not yet tested for GLM-5.3. If you want to deploy, please refer to scripts in GLM-5.2 1M Context Deployment.

5.2 Prefill-Decode Disaggregation

Prefill-Decode disaggregation scenarios have not yet tested for GLM-5.3. If you want to deploy prefill-decode disaggregation, you can refer to scripts in GLM-5.2 Prefill-Decode Disaggregation.

6 Functional Verification

Once your server is started, you can query the model with input prompts:

curl http://<node0_ip>:<port>/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "glm-5",
        "messages":[
            {
                "role": "user",
                "content": "Who are you?",
            }
        ],
        "temperature": 0
    }'

Expected result should be have this:

"message":{"role":"assistant","content":"I'm GLM, a large language model developed by Z.ai. I'm designed to understand and generate human-like text based on the conversations we have together. My tvolves processing diverse text data to help answer questions and provide assistance across many topics.\n\nI don't store your personal data, and I'm contiarning to improve my capabilities. Is there something specific I can help you with today?","refusal":null,"annotations":null,"audio":null,"function_call":oning":"Let me analyze this question about my identity. First, I should acknowledge that this is a fundamental question about who and what I am. The key pover are my identity as GLM, a large language model by Z.ai, and my core capabilities. I should explain my primary function of text processing and generate being transparent about my nature as an AI system. It's also important to clarify my role in helping users and my ability to engage with various topics. ention my text processing abilities and learning from diverse datasets, but avoid making claims about consciousness or emotions. The response should be stogically, starting with my basic identity and moving on to my capabilities and purpose. I'll organize this information in a clear, straightforward manner sses the user's query directly."}

7 Accuracy Evaluation

Here are two accuracy evaluation methods.

7.1 Using AISBench

1. Refer to Using AISBench for details.

2. After execution, you can get the result. Here are the results of GLM-5.3-w8a8c8 in vllm-ascend:v0.23.0 for reference only.

dataset	model	hardware	metric	mode	vllm-api-general-chat
GPQA Diamond	GLM-5.3-w8a8c8	A3	accuracy	gen	92.42
GPQA Diamond	GLM-5.3-w8a8c8	A2	accuracy	gen	90.40

7.2 Using Language Model Evaluation Harness

Not tested yet.

8 Performance Evaluation

8.1 Using AISBench

Refer to Using AISBench for performance evaluation for details.

8.2 Using vLLM Benchmark

Refer to vllm benchmark for more details.

9 FAQ

• Q: How to enable function calling for GLM-5.3?

A: Please add following configurations in vLLM startup command

--tool-call-parser glm47 \
--reasoning-parser glm45 \
--enable-auto-tool-choice \

10 Declaration

• The current version is only for early experience, and performance optimization is still in progress.
• The service reliability has not been fully validated, and it is not recommended for direct use in production environments.