Skip to content

vllm.v1.worker.gpu.spec_decode.dspark.speculator

DSpark speculator: semi-autoregressive parallel drafting.

DSpark drafts a block of num_speculative_tokens tokens in one parallel pass (reusing the DFlash machinery: context-KV precompute + a query-block forward), then injects intra-block dependency with a lightweight sequential Markov head.

Differences from DFlash
  • Anchor-as-first-prediction: each request emits exactly N = num_speculative_tokens query tokens (anchor + N-1 noise), NOT 1 + N. Every query position is a prediction (the anchor predicts the first draft token), so we sample at all N positions and sample_pos = query_pos + 1 (standard next-token), whereas DFlash's masks sit AT the predicted position. This is the sample_from_anchor path in the shared prepare-inputs kernel. Speculators-format checkpoints instead use the DFlash 1 + N fill-in layout (anchor is the bonus token).
  • Sequential Markov sampling: instead of DFlash's single parallel sample, we sample left-to-right, adding a prefix-dependent Markov bias derived from the previously sampled token at each step.

CUDA graphs (FULL, mirroring DFlash) cover the whole draft step: the parallel backbone forward AND the sequential Markov sampling.