All papersReasoning & AgentsGoogle

ReproBridge: Building Reproducibility into ML Research Artifacts

/ Paper arXiv 2611.08452

Key numbers

76.4% → 92.6%refined artifacts raise mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by…
19.7%…mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by 19.7% on average.
90%with all four models achieving mean success rates above 90%

From the paperCopied from the abstract, not written by AI.

Abstract

From the paperThe authors' own words.

Reproducing machine learning experiments requires reconstructing procedures from information scattered across papers, released code, and supporting artifacts. For coding agents, reproduction success can therefore depend on their ability to fill in missing details, rather than on the released artifacts alone. We ask whether preserving recovered procedures and decisions improves subsequent independent reproduction. We define a standardized artifact design comprising a refined codebase, environment setup, experiment launchers, and a reproduction guide, and introduce ReproBridge, an agent workflow for constructing these refined reproducible artifacts. ReproBridge-Bench provides 1,029 tasks from 387 experiments across 36 papers, anchored by five human-verified reference scores per task. Agents attempt tasks in fresh sessions with raw code or refined artifacts; a separate Auditor checks execution evidence. Across four evaluated models, refined artifacts raise mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by 19.7% on average. The cross-model success-rate spread narrows from 21.6 to 3.9 percentage points, with all four models achieving mean success rates above 90%. Removing any component reduces success in the ablation study, with the largest losses from code and environment removal. These results suggest that preserving execution knowledge helps even models with lower raw-code success reproduce experiments more reliably and at lower API cost.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Where it fits

The most cited papers it builds on, and the papers that build on it.

/ Builds on 3 most cited

  1. 20236.7k citedReflexion: language agents with verbal reinforcement learning ↗
  2. 20173.7k citedGuidelines for Evaluating and Expressing the Uncertainty of Nist Measurement Results
  3. 20172.9k citedA manifesto for reproducible science

/ Built on by None yet

No papers cite it yet. We check again every week.

From Semantic Scholar, updated weekly.

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      Read the PDF