AI PAPERS/DAILY
Subscribe free Open sourceView on GitHub Star This project is open sourceRead the code or use it in your own project. If you like it, don’t forget to give it a ★ star on GitHub.

All papersGenerative AI & DiffusionKuaishou

LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis

/ Paper arXiv 2611.06856

Key numbers

15%Reviewed expert labels include a 15% double-labeled subset with 89.4% exact agreement and 0.83 quadratic weighted…
89.4%…expert labels include a 15% double-labeled subset with 89.4% exact agreement and 0.83 quadratic weighted kappa.
90LongBanana contains 90 carefully designed tasks organized by a 3-by-3 taxonomy

From the paperCopied from the abstract, not written by AI.

Abstract

From the paperThe authors' own words.

We introduce LongBanana, to our knowledge the first expert-designed and expert-verified benchmark for long-context multi-reference image synthesis in the age of agent-driven image generation. The task requires retrieving dispersed visual evidence, rejecting distractors, binding entities and relations, and composing one coherent final image from multiple references. LongBanana contains 90 carefully designed tasks organized by a 3-by-3 taxonomy; each task includes about 32 reference images, 20 constraints, and 50 evaluation items, yielding 4,670 expert-specified atomic evaluation items in total. Its graph-grounded evaluation design instantiates each task with reference images and an entity-relation graph, then converts node, edge, and whole-graph obligations into checklist items for Content Fidelity, Relational Binding, and Global Constraint Satisfaction. Reviewed expert labels include a 15% double-labeled subset with 89.4% exact agreement and 0.83 quadratic weighted kappa. We further introduce LongBanana-Gen, an agent-harness framework for long-horizon image generation; experiments show that minimal agent harnesses improve over direct generation, while a ReAct-Plan-Exec-Reflect (RPR) workflow yields harness-dependent gains by strengthening evidence grounding and repair. Together, LongBanana provides an auditable testbed for measuring dense visual evidence use and a shared target for developing stronger agent-driven image-generation systems. Code and data are available at the project page: https://long-banana.com/.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Try it

Code, models and demos that use this paper.

Datasets:
Kwai-Kolors/LongBanana ↗1 dataset on Hugging Face

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      / Read next, same idea

      Read the PDF