跳转至

文档编写指南

模型教程文档编写指南

docs/source/_templates/Model-Deployment-Tutorial-Template.md 是编写模型部署教程的模板。您可以复制并修改它以创建新文档。

可测试文档代码块生成（model-code)

  • For documentation authors: how to insert testable command blocks into docs
  • For developers: how to add a new converter

Built-in supported converter_tag values:

converter_tag Renders YAML source
single_node A single node's env exports + vllm serve script test_cases[case_index]
multi_node One host's env exports + vllm serve script deployment[host_index]
external_dp_template One external-DP node's env exports + vllm serve command templates[host_index]
external_dp_launch One launch_online_dp.py line per node config[]
external_dp_proxy The load-balance proxy launch command config[] + routing

For authors: add a block

Warning

By default, the generator scans only .md files under docs/source/tutorials/models/ and produces artifacts. If you put model-code blocks in other directories, Sphinx builds will not automatically generate the corresponding scripts.

All model-code blocks need:

Option Required Description
block_name Yes Block name; must be unique within the current document
converter_tag Yes Selects one of the built-in converters
test_case_path Yes Repository-relative YAML path that stays within the repo; the file must exist

Use the body of the block to add shell wrapper lines such as set -eux. Always place the {{ generated }} placeholder where the converter output should be inserted.

converter_tag: single_node

single_node reads one item from test_cases. The optional case_index metadata selects the item; when omitted, it defaults to 0.

Only the fields read by this converter are expanded below. Other test metadata can be left in the YAML and is ignored by this converter.

test_cases:
  - name: qwen3-8b-single
    model: Qwen/Qwen3-8B
    envs:
      HCCL_BUFFSIZE: "1024"
      SERVER_PORT: DEFAULT_PORT
    server_cmd:
      - --tensor-parallel-size
      - "1"
      - --port
      - $SERVER_PORT
      - --trust-remote-code
    server_cmd_extra:
      - --enable-expert-parallel
    benchmarks: ...

envs is rendered as export lines. SERVER_PORT: DEFAULT_PORT is resolved to the default single-node port 8000. model becomes vllm serve <model>, and server_cmd plus optional server_cmd_extra become command arguments. Both command fields can be either a shell string or a flat token list.

按如下方式编写文档块：

```model-code
:block_name: qwen3_8b_single_node
:converter_tag: single_node
:test_case_path: tests/e2e/nightly/single_node/models/configs/your_model.yaml
:case_index: 0

set -eux
{{ generated }}
```

生成的 shell 脚本：

set -eux
export HCCL_BUFFSIZE=1024
export SERVER_PORT=8000

vllm serve Qwen/Qwen3-8B \
  --tensor-parallel-size 1 \
  --port $SERVER_PORT \
  --trust-remote-code \
  --enable-expert-parallel

converter_tag: multi_node

multi_node reads one item from deployment. The required host_index metadata selects which host to render.

deployment:
  - envs:
      SERVER_PORT: "8000"
    server_cmd: >
      vllm serve Qwen/Qwen3-235B-A22B
      --host 0.0.0.0
      --port $SERVER_PORT
      --data-parallel-size 2
      --tensor-parallel-size 8
      --data-parallel-address $LOCAL_IP
  - envs:
      SERVER_PORT: "8000"
    server_cmd: >
      vllm serve Qwen/Qwen3-235B-A22B
      --headless
      --port $SERVER_PORT
      --data-parallel-size 2
      --tensor-parallel-size 8
      --data-parallel-start-rank 1
      --data-parallel-address $MASTER_IP
benchmarks: ...

server_cmd must be a complete command starting with vllm serve <model>. It can be written as a shell string or a flat token list.

按如下方式编写文档块：

```model-code
:block_name: qwen3_235b_worker_1
:converter_tag: multi_node
:test_case_path: tests/e2e/nightly/multi_node/internal_dp/config/your_model.yaml
:host_index: 1

set -eux
{{ generated }}
```

host_index: 1 生成的 shell 脚本：

set -eux
export MASTER_IP=192.168.1.10
export SERVER_PORT=8000

vllm serve Qwen/Qwen3-235B-A22B \
  --headless \
  --port $SERVER_PORT \
  --data-parallel-size 2 \
  --tensor-parallel-size 8 \
  --data-parallel-start-rank 1 \
  --data-parallel-address $MASTER_IP

converter_tag: external_dp_template

external_dp_template reads one item from templates. The required host_index metadata selects which template to render. The top-level model field is also required because the converter builds vllm serve <model>.

model: Eco-Tech/GLM-Test
templates:
  - node_index: 0
    envs:
      HCCL_BUFFSIZE: "1024"
      ASCEND_RT_VISIBLE_DEVICES: "${VISIBLE_DEVICES}"
    server_cmd_template:
      - --host
      - 0.0.0.0
      - --port
      - ${PORT}
      - --data-parallel-size
      - ${DP_SIZE}
      - --data-parallel-rank
      - ${DP_RANK}
      - --data-parallel-address
      - ${DP_ADDRESS}
      - --data-parallel-rpc-port
      - ${DP_RPC_PORT}
      - --tensor-parallel-size
      - ${TP_SIZE}
      - --trust-remote-code
config: ...
routing: ...

Known braced template variables are rewritten to the positional shell arguments that run_dp_template.sh receives from launch_online_dp.py:

模板变量 渲染后的位置参数
${VISIBLE_DEVICES} $1
${PORT} $2
${DP_SIZE} $3
${DP_RANK} $4
${DP_ADDRESS} $5
${DP_RPC_PORT} $6
${TP_SIZE} $7

Unknown braced variables and unbraced shell references such as $SERVER_PORT are left unchanged.

按如下方式编写文档块：

```model-code
:block_name: glm_external_dp_template_node0
:converter_tag: external_dp_template
:test_case_path: tests/e2e/nightly/multi_node/external_dp/config/your_model.yaml
:host_index: 0

set -eux
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name

{{ generated }}
```

host_index: 0 生成的 shell 脚本：

set -eux
export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name

export HCCL_BUFFSIZE=1024
export ASCEND_RT_VISIBLE_DEVICES=$1

vllm serve Eco-Tech/GLM-Test \
  --host 0.0.0.0 \
  --port $2 \
  --data-parallel-size $3 \
  --data-parallel-rank $4 \
  --data-parallel-address $5 \
  --data-parallel-rpc-port $6 \
  --tensor-parallel-size $7 \
  --trust-remote-code

converter_tag: external_dp_launch

external_dp_launch reads the full config list and renders one launch_online_dp.py command per node. It does not take an index option.

config:
  - node_index: 0
    port_start: 7100
    dp_rpc_port: 12321
    dp_size: 2
    dp_size_local: 2
    dp_rank_start: 0
    tp_size: 8
    dp_address: "${NODE_0_IP}"
  - node_index: 1
    port_start: 7200
    dp_rpc_port: 12321
    dp_size: 4
    dp_size_local: 4
    dp_rank_start: 0
    tp_size: 4
    dp_address: "${NODE_1_IP}"
templates: ...
routing: ...

按如下方式编写文档块：

```model-code
:block_name: glm_external_dp_launch
:converter_tag: external_dp_launch
:test_case_path: tests/e2e/nightly/multi_node/external_dp/config/your_model.yaml

set -eux
{{ generated }}
```

生成的 shell 脚本：

set -eux
python launch_online_dp.py --dp-size 2 --tp-size 8 --dp-size-local 2 --dp-rank-start 0 --dp-address ${NODE_0_IP} --dp-rpc-port 12321 --vllm-start-port 7100

python launch_online_dp.py --dp-size 4 --tp-size 4 --dp-size-local 4 --dp-rank-start 0 --dp-address ${NODE_1_IP} --dp-rpc-port 12321 --vllm-start-port 7200

converter_tag: external_dp_proxy

external_dp_proxy reads config and routing. It renders the load_balance_proxy_server_example.py command for routing.type: disaggregated_prefill. It does not take an index option.

routing:
  type: disaggregated_prefill
  groups:
    prefiller: [0]
    decoder: [1]
config:
  - node_index: 0
    port_start: 7100
    dp_size_local: 2
    dp_rpc_port: 12321
    dp_size: 2
    dp_rank_start: 0
    tp_size: 8
    dp_address: "${NODE_0_IP}"
  - node_index: 1
    port_start: 7200
    dp_size_local: 4
    dp_rpc_port: 12321
    dp_size: 4
    dp_rank_start: 0
    tp_size: 4
    dp_address: "${NODE_1_IP}"
templates: ...

routing.groups.prefiller and routing.groups.decoder contain indices into config. Each referenced node expands to dp_size_local host and port entries. The proxy itself is rendered on ${NODE_0_IP}:1999.

按如下方式编写文档块：

```model-code
:block_name: glm_external_dp_proxy
:converter_tag: external_dp_proxy
:test_case_path: tests/e2e/nightly/multi_node/external_dp/config/your_model.yaml

set -eux
{{ generated }}
```

生成的 shell 脚本：

set -eux
python load_balance_proxy_server_example.py \
  --host ${NODE_0_IP} \
  --port 1999 \
  --prefiller-hosts \
    ${NODE_0_IP} \
    ${NODE_0_IP} \
  --prefiller-ports \
    7100 \
    7101 \
  --decoder-hosts \
    ${NODE_1_IP} \
    ${NODE_1_IP} \
    ${NODE_1_IP} \
    ${NODE_1_IP} \
  --decoder-ports \
    7200 \
    7201 \
    7202 \
    7203

本地调试与生成

仅生成（不构建完整站点）

# Generate all model-code artifacts under docs/source/tutorials/models/
python3 tools/docs_codegen/cli.py

# Generate artifacts for a single document
python3 tools/docs_codegen/cli.py --doc docs/source/tutorials/models/Kimi-K2-Thinking.md

# Generate a single block and print it (no files written)
python3 tools/docs_codegen/cli.py \
  --block docs/source/tutorials/models/Kimi-K2-Thinking.md::kimi_k2_thinking_single_node \
  --dry-run --stdout

默认情况下，生成物写入：docs/_build/doc_codegen/<doc_stem>/<block_name>.sh。

Note

脚本生成后，请务必检查生成内容是否可运行，特别是环境变量和命令行参数等关键部分。

构建站点并在本地预览

# Install documentation build dependencies
python3 -m pip install -r docs/requirements-docs.txt

# (Optional) Clean previous builds
make -C docs clean

# Build the English site
make -C docs html

# (Optional) Build the Chinese site
make -C docs intl

# Preview locally
python3 -m http.server -d docs/_build/html 8000

# Then open in a browser:
# http://localhost:8000

面向开发者：添加新的转换器

A converter turns one loaded YAML file plus one parsed ModelCodeBlock into a GeneratedScript. The current pipeline is:

  1. BlockScanner parses model-code fences and accepts only options listed in MODEL_CODE_OPTION_NAMES.
  2. YamlLoader loads test_case_path.
  3. get_converter() looks up block.converter_tag from build_default_converters().
  4. The selected converter returns GeneratedScript(content=..., language="shell").
  5. GeneratorService replaces {{ generated }} in the block body, validates that the final script is non-empty, and writes docs/_build/doc_codegen/<doc_stem>/<block_name>.sh.

To add a converter:

  1. In tools/docs_codegen/converters.py, add a BaseConverter subclass with a unique name. That name is the value authors put in :converter_tag:.
  2. Implement convert(self, loaded_yaml, *, block) -> GeneratedScript. Use make_docs_codegen_error(..., block=block) for user-facing validation errors so the CLI and Sphinx output include document context.
  3. Reuse helpers from tools/docs_codegen/utils.py, such as require_mapping, require_mapping_list, require_scalar_mapping, require_indexed_mapping, require_node_field, parse_command_tokens, substitute_template_positionals, and render_cli_command.
  4. Register the converter in build_default_converters(). If it is not registered, get_converter() will reject the new converter_tag.
  5. If the converter needs new directive metadata, add the option name to MODEL_CODE_OPTION_NAMES in tools/docs_codegen/scanner.py and to ModelCodeDirective.option_spec in tools/docs_codegen/sphinx_extension.py. Read the option with block.get_option("<option_name>").
  6. Add or update tests in tests/ut/tools/test_docs_codegen.py. Cover the successful render path, required option validation, YAML shape validation, and any CLI/Sphinx scanner behavior affected by new metadata.
  7. Add a real model-code example in a model tutorial, preferably under docs/source/tutorials/models/, and point it to an existing YAML file under tests/.
  8. Validate with the CLI:
python3 tools/docs_codegen/cli.py --doc <your_doc> --dry-run
python3 tools/docs_codegen/cli.py --block <your_doc>::<block_name> --dry-run --stdout

If a converter should render something other than shell, set GeneratedScript.language accordingly so Sphinx can highlight the generated literal block correctly.