All papersRLHF & AlignmentTop pick on Nov 5JD.com

Turns Matter: Advancing Streaming Video RL via Turn-Level Credit

/ Paper arXiv 2611.06728

Curator's notes

TurnWise adds turn-level credit assignment to streaming video RL, improving decision quality without extra critics.

Curator's notesWritten by AI from the paperReport a mistake

What's new

Introduces turn-level GRPO with silence-completed prefix scoring and turn-mean policy optimization, unlike prior single trajectory-level advantage methods.

Key result

Extensive experiments show TurnWise outperforms prior RL approaches for streaming video understanding.

Why it matters

It yields more informative, less redundant streaming responses, benefiting real-time video applications.

Abstract

From the paperThe authors' own words.

Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

What you could build

Written by AIIntermediateRequires RL implementation and integration with streaming video models

  1. Implement a live video captioning system using TurnWise for adaptive response timing
  2. Create a streaming video QA bot that decides when to answer versus wait using turn-level credit
  3. Develop a real‑time video summarizer that leverages TurnWise to balance silence and output

Try it

Code, models and demos that use this paper.

Models:
shuai22/turns-matter-rl-ckpt ↗1 model on Hugging Face

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Quiz yourself

    How does TurnWise assign credit to each decision unlike previous methods?

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      / Next pickThink Less, Locally: Local Preference Learning for Reasoning Compression
      Read the PDF