# GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

**Authors:** Chanseul Cho, Noori Bae, Dongheon Lee, Youngmin Ro  
**Lab:** University of Seoul · **Published:** 30 Oct 2026  
**Links:** [arXiv](https://arxiv.org/abs/2610.41629) · [PDF](https://arxiv.org/pdf/2610.41629) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.41629.html)

## Abstract

Reinforcement learning with verifiable rewards (RLVR) has recently been extended to vision-language models (VLMs), yet its effectiveness remains limited for text-rich reasoning tasks. We highlight a key factor contributing to this limitation: models frequently arrive at correct final answers despite misperceiving visual information during intermediate steps. We define this phenomenon as Correct-but-Visually-Misgrounded Reasoning (CVMR). Because standard RLVR rewards only final-answer correctness, it fails to differentiate between properly grounded and misgrounded reasoning, reinforcing both equally. Through token-level analysis, we reveal that CVMR tokens exhibit distinct attention patterns that differ systematically from well-grounded ones. Based on this observation, we propose GRIT, a training framework that incorporates visual grounding into RLVR to mitigate the impact of CVMR. Furthermore, to address the lack of text-rich reasoning benchmarks that allow for systematic evaluation across text length and reasoning depth, we introduce GRIT-Bench. Experiments demonstrate that GRIT significantly improves both reasoning accuracy and grounding fidelity over existing RLVR methods.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.