vllm bench sweep startup¶

JSON CLI Arguments¶

When passing JSON CLI arguments, the following sets of arguments are equivalent:

• --json-arg '{"key1": "value1", "key2": {"key3": "value2"}}'
• --json-arg.key1 value1 --json-arg.key2.key3 value2

Additionally, list elements can be passed individually using +:

• --json-arg '{"key4": ["value3", "value4", "value5"]}'
• --json-arg.key4+ value3 --json-arg.key4+='value4,value5'

Arguments¶

--startup-cmd¶

The command used to run the startup benchmark.

Default: vllm bench startup

--serve-params¶

Path to JSON file containing parameter combinations for the vllm serve command. Only parameters supported by vllm bench startup will be applied.

--startup-params¶

Path to JSON file containing parameter combinations for the vllm bench startup command.

--strict-params¶

If set, unknown parameters in sweep files raise an error instead of being ignored.

Default: False

-o, --output-dir¶

The main directory to which results are written.

Default: results

-e, --experiment-name¶

The name of this experiment (defaults to current timestamp). Results will be stored under output_dir/experiment_name.

--num-runs¶

Number of runs per parameter combination.

Default: 1

--show-stdout¶

If set, logs the standard output of subcommands.

Default: False

--dry-run¶

If set, prints the commands to run, then exits without executing them.

Default: False

--resume¶

Resume a previous execution of this script, i.e., only run parameter combinations for which there are still no output files under output_dir/experiment_name.

Default: False