All papersGenerative AI & Diffusion

PhyRubric: Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation

/ Paper arXiv 2611.13576

Key numbers

34which spans 34 categories across five physical domains
3,788and scale it up to PhyRubric-Train with 3,788 videos

From the paperCopied from the abstract, not written by AI.

Why this ranked

MeasuredPublic numbers, not AIHow ranking works

18 / 100#19 of 955 papers that day, from GitHub, Hugging Face, Hacker News and Semantic Scholar, looked up the morning it was ranked.

Builder evidence8 / 40

  • Code released

Community attention10.4 / 30

  • ▲ 10 upvotes on Hugging Face

Author track record0 / 15

  • Nothing found

What the paper shows0 / 15

  • Nothing found

Abstract

From the paperThe authors' own words.

As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense. However, existing benchmarks rely on annotators who rate each video with a single score based on intuition, yielding ratings that are subjective and do not tell which physical principle is violated. We introduce PhyRubric, a rubric-grounded diagnostic framework for evaluating physical commonsense in T2V generation. For each caption, PhyRubric constructs a caption-specific rubric before any video is seen, decomposing the caption into semantic and physical items, and derives scores from item-level verdicts grounded in timestamped evidence. We instantiate this framework as PhyRubric-Bench, which spans 34 categories across five physical domains, and scale it up to PhyRubric-Train with 3,788 videos, on which we fine-tune the open-source PhyRubric-Judge. Empirical results across ten T2V models show that physical commonsense remains far from solved, and that PhyRubric-Judge correlates more closely with human judgments than existing open-source judges while matching the frontier proprietary Gemini 4 Argon. Further analysis provides actionable insights for T2V models.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Try it

Code, models and demos that use this paper.

Models:
kittyhy/PhyRubric-Judge ↗1 model on Hugging Face
Datasets:
kittyhy/PhyRubric-Bench ↗2 datasets on Hugging Face

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      Read the PDF