/ Paper arXiv 2611.12885
Curator's notes
Robots learn manipulation from a single demo by rehearsing in simulation, writing feedback to context, achieving 83.4% success without gradient updates.
A robot can learn a new manipulation task from one example by practicing in a simulated world, without changing its internal code.Robots learn manipulation from a single demo by rehearsing in simulation, writing feedback to context, achieving 83.4% success without gradient updates.World In-Context Learningin-context learningLearning where the model uses examples provided at inference time without updating its weights. (WICL) lets a frozen model rehearse tasks in simulation, achieving 83.4% success on RoboTwin 2.0 with single-demo, no gradient updates.Curator's notesWritten by AI from the paperEvery number here appears in the abstractReport a mistake
What's new
Proposes World In-Context Learningin-context learningLearning where the model uses examples provided at inference time without updating its weights. where a frozen model rehearses tasks in a simulated world and stores feedback in context, surpassing trained policies with one demonstration.
Key result
RoboScript reaches 83.4% mean success on all 50 RoboTwin 2.0 tasks, exceeding the strongest trained action policy by over 20 points.
Why it matters
Allows rapid skill acquisition for robots without costly retraining, leveraging simulation to adapt to new scenes.
Key numbers
From the paperCopied from the abstract, not written by AI.
Abstract
From the paperThe authors' own words.
Trained action policies write manipulation skill into network weights, tying what is learned to the distribution it is learned from. In-context learning (ICL) avoids this coupling but has not matched trained policies, because a demonstration recorded in another scene carries no feedback from the world the robot currently faces. We observe that what an agent learns in context is semantic, recording task steps and object roles rather than visual appearance, so experience gathered in one world transfers to another. Based on this insight, we propose World In-Context Learning (WICL), a paradigm in which a frozen model starts from a demonstration, rehearses the task in a world such as a simulator, and writes the feedback into its context rather than its weights. We build RoboScript, a modular infrastructure that implements WICL for agentic manipulation. From a single demonstration, RoboScript learns each task through rehearsal in simulation, freezes the learned knowledge as a Task MemoryTask MemoryA frozen representation that stores the knowledge acquired from rehearsing a task in simulation., and deploys the frozen memory on new scenes with data isolation that ensures every deployment input is available on a real robot. On all 50 tasks of RoboTwin 2.0, RoboScript reaches 83.4% mean success across clean and randomized settings with one demonstration per task and no gradient updates, exceeding the strongest trained action policy by over 20 points. The method generalizes across visual distributions, improves through reflection, scales with the foundation model, and transfers to a physical dual-arm robot through cross-embodiment memory adaptationcross-embodiment memory adaptationTransferring a learned Task Memory to a robot with a different physical configuration.. With a world in the loop, a robot can learn a manipulation task in its context as well as it learns one in its weights. All code is available at https://github.com/ArcheOne-AI/RoboScript
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
What you could build
Written by AIIntermediateNeeds simulator integration and a large frozen model but no training loops.
- Create a demo-to-simulation pipeline that lets a robot learn new pick-and-place tasks from a single video.
- Develop a cross-embodiment memory adapter to transfer learned Task Memories between different robot arms.
- Build an interactive UI for users to provide one-shot demonstrations and instantly deploy them on a real robot.
Try it
Code, models and demos that use this paper.
- Code:
- ★ 34 on GitHub ↗
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Quiz yourself
What mean success rate did RoboScript achieve on the 50 RoboTwin 2.0 tasks with one demonstration per task?
It achieved an 83.4% mean success rate, which is more than 20 points higher than the best trained action policy.
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.
