/ Paper arXiv 2611.08373
Curator's notes
SLA continually pre‑trains a vision‑language‑action model to align visual embeddings with 3D spatial representations, improving transferable spatial understanding for robots.
A robot model learns to match its camera view with 3‑D space while training, so it can handle new tasks better.SLA continually pre‑trains a vision‑language‑action model to align visual embeddings with 3D spatial representations, improving transferable spatial understanding for robots.SLA continually pre‑trains a VLA model with joint action prediction and alignment to frozen 3D perception embeddings, using feature‑dependent gating and probabilistic state dropoutprobabilistic state dropoutA training technique that randomly omits state information to force reliance on visual cues..Curator's notesWritten by AI from the paperEvery number here appears in the abstractReport a mistake
What's new
Introduces continual pre‑training with implicit spatial alignment and feature‑dependent gates to fuse embeddings across heterogeneous robot datasets, using only training‑time auxiliary components.
Why it matters
Better spatial grounding lets robots generalize manipulation across tasks without redesigning perception.
Abstract
From the paperThe authors' own words.
Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropoutprobabilistic state dropoutA training technique that randomly omits state information to force reliance on visual cues. encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
What you could build
Written by AIIntermediateneeds access to robot datasets and compute for continual pre‑training
- Create a robot pick‑and‑place system that uses SLA to plan grasps in unseen scenes.
- Develop a simulation‑to‑real transfer pipeline leveraging SLA’s spatial embeddings.
- Build an interactive demo where a robot follows natural language commands to navigate cluttered environments.
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Quiz yourself
What auxiliary components does SLA use only during training to preserve the original inference architecture?
Feature‑dependent gates, probabilistic state dropout, and a teacher that exploits depth and camera calibration are used only during training.
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.