All papersGenerative AI & DiffusionTop pick on Nov 3FrameX-AI

FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation

/ Paper arXiv 2610.36545

Curator's notes

FrameForge uses adaptive virtual temporal relations from video models to improve unified conditional image generation.

Curator's notesWritten by AI from the paperEvery number here appears in the abstractReport a mistake

What's new

It introduces Mixture of Temporal Experts and Chain of Temporal Reasoning to dynamically model condition-target distances and intermediate transformations.

Key result

FrameForge achieves superior performance on DreamBench, COCO2017, and GEdit-Bench across subject-driven, canny-to-image, and image editing tasks.

Why it matters

Builders can leverage video priors to handle diverse image generation conditions with a single model.

Abstract

From the paperThe authors' own words.

Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

What you could build

Written by AIIntermediateRequires understanding of video model embeddings and attention masking

  1. Create a text‑guided photo editor that edits images via latent intermediate states
  2. Develop a sketch‑to‑photo generator using adaptive temporal intervals
  3. Build a subject‑driven image synthesis tool that swaps objects based on textual prompts

Try it

Code, models and demos that use this paper.

Demos:
akhaliq/FrameForge ↗1 demo on Hugging Face

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Quiz yourself

    How does FrameForge model the distance between condition and target images?

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      / Next pickR^2Orch: Recursive Self-Improvement for Agent Orchestration via Self-Play within a Skill-Internalization Loop
      Read the PDF