vllm bench sweep¶
Overview¶
Benchmark for a parameter sweep.
Subcommands¶
| Command | Description |
|---|---|
serve | Run vLLM server benchmark under multiple settings. |
serve_workload | Explore the latency-throughput tradeoff for different workload levels. |
startup | Benchmark vLLM startup time over parameter combinations. |
plot | Plot performance curves from parameter sweep results. |
plot_pareto | Plot Pareto frontier between tokens/s/user and tokens/s/GPU from parameter sweep results. |