/ Paper arXiv 2610.27815
In short
In shortReport a mistake
Key numbers
From the paperCopied from the abstract, not written by AI.
Abstract
From the paperThe authors' own words.
Speculative decoding accelerates autoregressive (AR) language models by letting a small model draft tokens serially and a large model verify them in parallel. We reverse these roles. Starting from a standard AR checkpoint, we fine-tune the large model into a proposer that predicts a block of future tokens in a single forward pass, then use one additional forward pass to verify the proposed block. This gives ProVe, Proposer-Verifier decoding. The verifier can be a smaller AR model or the proposer itself; the latter gives ProVe-lossless, which reproduces the proposer's greedy AR decoding. Block proposals already provide substantial room for acceleration: with a block size of 16, the proposed block has an average AR-matching prefix of 6.6 tokens on GSM8K. Across six benchmarks with a 32B proposer, ProVe-lossless reaches 2.5x average AR throughput without training a separate draft head. With a small 3B verifier, ProVe reaches 2.7x average throughput at 2.6 accuracy points below AR on the six-task average, providing a controllable tradeoff between speed and accuracy.
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
Where it fits
The most cited papers it builds on, and the papers that build on it.
/ Builds on 3 most cited
- 20245.8k citedQwen2.5 Technical Report ↗
- 20244.4k citedDeepSeek-V3 Technical Report
- 20222.3k citedFast Inference from Transformers via Speculative Decoding ↗
/ Built on by None yet
No papers cite it yet. We check again every week.
From Semantic Scholar, updated weekly.
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.
