AI PAPERS/DAILY
Subscribe free Open sourceView on GitHub Star This project is open sourceRead the code or use it in your own project. If you like it, don’t forget to give it a ★ star on GitHub.

All papersComputer VisionNVIDIA

Interactive World Simulation with Cosmos for Physical AI

/ Paper arXiv 2611.06627

Abstract

From the paperThe authors' own words.

We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework enables users and embodied agents to interact with evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical state maintained by an external simulator. We demonstrate this approach across camera navigation and robot manipulation, controllable video transfer, and multi-view autonomous vehicle (AV) simulation. These examples provide a shared foundation for developing interactive virtual environments for Physical AI training and evaluation. Our code and model checkpoints are available at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3-examples. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Try it

Code, models and demos that use this paper.

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      / Read next, same idea

      Read the PDF