AI PAPERS/DAILY
Subscribe free Open sourceView on GitHub Star This project is open sourceRead the code or use it in your own project. If you like it, don’t forget to give it a ★ star on GitHub.

All papersEfficient AI & Edge

Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps in a Small Local Model

/ Paper arXiv 2611.14418

Key numbers

15%At a 15% escalation budget it catches 78% of flips and the margin gate 94%
78%At a 15% escalation budget it catches 78% of flips and the margin gate 94%
94%At a 15% escalation budget it catches 78% of flips and the margin gate 94%

From the paperCopied from the abstract, not written by AI.

Why this ranked

MeasuredPublic numbers, not AIHow ranking works

7 / 100#132 of 1200 papers that day, from GitHub, Hugging Face, Hacker News and Semantic Scholar, looked up the morning it was ranked.

Builder evidence0 / 40

  • No public code found

Community attention7 / 30

  • ▲ 4 upvotes on Hugging Face

Author track record0 / 15

  • Nothing found

What the paper shows0 / 15

  • Nothing found

Abstract

From the paperThe authors' own words.

Local decoding usually fixes one weight precision for the whole generation. We study a different allocation: store 8-bit weights, decode with their top 4 bits, and redo a step at 8 bits only when the low-precision output distribution looks uncertain (entropy above a threshold or a small top-two margin). Quantization is simulated, the model is Qwen3-0.6B, and the benchmark is GSM8K. Cost is counted in weight bytes read per token. On 36724 replayed steps the entropy of the low pass ranks argmax flips with AUC 0.93. At a 15% escalation budget it catches 78% of flips and the margin gate 94%, against the 15% expected for a random gate of equal rate (a computed expectation, not a run). In 150 live problems all-high scores 64.0% and all-low 55.3% (McNemar p=0.053). An entropy gate with a 16% escalation rate scores 62.7% at 0.87 of the all-high weight bytes. Its paired difference is -1.3 (-8.0 to +5.3) points to all-high and +7.3 (+0.0 to +14.7) points to all-low (p=0.080), so the live data do not separate the gate from either end. Verified speculation reaches zero KL at similar bytes in the same cost model. A live random control, a thinking-mode phase rule, a second model and live KL were not run.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      / Read next, same idea

      Read the PDF