/ Paper arXiv 2610.36671
Curator's notes
LOWN skips image processing during diffusion decoding unless needed, cutting FLOPs while keeping performance.
Curator's notesWritten by AI from the paperEvery number here appears in the abstractReport a mistake
What's new
Introduces certificate‑guided token unmasking that decides per step whether to use visual tokens, unlike prior static pruning methods.
Key result
Retains 99.6% of LLaDA‑V's score on ten benchmarks while using only 35.3% of decoding FLOPs on average.
Why it matters
Reduces compute cost for vision‑language diffusion models, enabling faster inference on limited hardware.
Key numbers
From the paperCopied from the abstract, not written by AI.
Abstract
From the paperThe authors' own words.
Diffusion vision-language models (DVLMs) process their entire input at every denoising step. Since visual tokens make up 89-97% of this input on the benchmarks we study, most of the decoding computation is spent repeatedly processing the same image. Visual token pruning methods partially relieve this burden by removing part of the visual tokens, but they lose the removed information and still process the remaining image at every step. To measure how often the image is actually needed, we compare LLaDA-V's predictions with and without the image at every decoding step and find that removing the image changes a token being unmasked at only 22.8% of steps on average and at as few as 7.9%. We therefore propose LOWN, a decoding method that decides at every step whether to process the image and how many tokens to unmask. LOWN predicts the masked tokens without the image and processes the image only when this prediction disagrees with the last image-conditioned prediction. Agreement certifies the earlier prediction for reuse, and the number of certified tokens sets how many are unmasked in that step. We further observe that the need for the image differs across benchmarks, so LOWN's adaptive decision is needed rather than a fixed schedule. Without training or a tuned threshold, LOWN retains 99.6% of LLaDA-V's score on ten benchmarks at 35.3% of the decoding FLOPs on average. LOWN also generalizes across LLaDA-based DVLMs, composes orthogonally with visual token pruning, and runs under key-value caching for further efficiency gains. Code and project page are available at https://hyeon9698.github.io/LOWN/
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
What you could build
Written by AIIntermediateRequires understanding of diffusion VLM decoding and integration of certificate logic
- Implement a real‑time image captioning service that uses LOWN to lower latency
- Create a mobile app that generates visual stories with diffusion VLMs using LOWN for efficiency
- Build a benchmark suite comparing static token pruning vs. LOWN on various VLM tasks
Where it fits
The most cited papers it builds on, and the papers that build on it.
/ Builds on 3 most cited
- 202312k citedVisual Instruction Tuning ↗
- 20193.7k citedGQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
- 20222.9k citedLearn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering ↗
/ Built on by None yet
No papers cite it yet. We check again every week.
From Semantic Scholar, updated weekly.
Try it
Code, models and demos that use this paper.
- Code:
- ★ 22 on GitHub ↗
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Quiz yourself
How often does removing the image change an unmasked token during decoding according to the authors?
Only 22.8% of steps on average, and as few as 7.9% on some benchmarks.
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.