← Back to arXiv Robotics

arXiv Robotics briefings

ResearchOfficialarXiv Robotics

Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control

A new preprint explores how mechanistic interpretability can reveal low-dimensional, robustness-critical features in World Action Models (WAMs), enabling targeted interventions. The authors introduce contrastive activation directions for training-free steering and propose the World-Action Linear Quadratic Regulator (WA-LQR), a feedback control method that leverages local linearity in WAM activation dynamics. Experiments show that these approaches improve robustness to various perturbations in Cosmos-Policy and DiT4DiT models, outperforming baseline methods.

Why it matters: This work offers a novel, training-free approach to enhance the robustness of world action models, which could improve reliability in robotics and autonomous systems.

ResearchOfficialarXiv Robotics

Reflex: Real-Time VLA Control Through Streaming Inference

Reflex is a framework that enables real-time streaming inference for flow matching Vision-Language-Action (VLA) models by leveraging a Timestep-Invariance Property. It partitions attention context into static, sliding, and dynamic regions, allowing O(1) incremental cache updates, and introduces AdaRMSNorm to prevent numerical instability. Reflex achieves a 2.58× inference speedup and stable 50Hz streaming on benchmarks, reducing reaction latency by up to 54%.

Why it matters: This work addresses a key latency bottleneck in flow matching VLA models, enabling real-time robotic control without sacrificing performance.

ResearchOfficialarXiv Robotics

RoboTTT: Scaling Robot Policy Context to 8K Timesteps

Researchers present RoboTTT, a robot policy model that scales visuomotor context to 8,000 timesteps—three orders of magnitude beyond prior approaches—without increasing inference latency. RoboTTT enables new capabilities such as one-shot in-context imitation from human video, robust long-horizon task completion, and on-the-fly policy improvement. The model achieves an 87% performance improvement over single-step baselines and fully completes complex, multi-stage real-robot manipulation tasks.

Why it matters: This work establishes context length as a powerful new scaling axis for robot foundation models, unlocking significant advances in long-horizon and in-context robotic capabilities.

ResearchOfficialarXiv Robotics

SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents

Researchers have introduced SafeRelBench, a benchmark comprising 507 samples designed to evaluate process-level safety in vision-language-model-driven embodied agents. The benchmark focuses on spatial relations such as support and containment, which are critical for safe interaction in household environments. Testing seven different models revealed that agents frequently achieve task completion while violating safety constraints during multi-step actions, highlighting a significant gap between task success and safety compliance.

Why it matters: SafeRelBench exposes the need for embodied agents to reason more effectively about spatial relations to ensure safety, rather than prioritizing task completion alone.

ResearchOfficialarXiv Robotics

Acc-CBF-QP: Acceleration-Based Safety Filter for RL Robotic Control

Researchers present Acc-CBF-QP, an acceleration-based Quadratic Program safety filter leveraging Control Barrier Functions to enforce safety constraints on reinforcement learning (RL) policies in real time, without altering the training process. The method is demonstrated on both a Kinova Gen3 manipulator and a Unitree H1 humanoid robot, achieving up to 92% reduction in constraint violations on hardware and fully eliminating violations on the Kinova Gen3. The approach maintains nominal RL task performance in safe regimes and prevents constraint-induced shutdowns under aggressive commands.

Why it matters: This work offers a practical and effective solution for safely deploying RL policies on real-world robots, addressing a major challenge in robotic control.

ModelsOfficialarXiv Robotics

OASIS-Map: New System for Object-Level Change Detection in Multi-Session Robotic Mapping

Researchers have introduced OASIS-Map, a multi-session mapping system that maintains spatio-temporally consistent object-level maps for robots in semi-static environments. The system leverages dense patch-level semantic correspondences to detect scene changes and associate objects across repeated visits, addressing challenges such as partial views and occlusion. OASIS-Map was evaluated in real-world scenarios, achieving F1 scores of 0.783 in car replacement detection and 0.667 in moved object association tasks.

Why it matters: This work advances long-term robotic inspection by improving change detection and object association in dynamic environments.

ResearchOfficialarXiv Robotics

Interventional Causal Circuits Enable Safer and More Efficient Robot Action Testing

A new framework combines a Joint Probability Tree with a Causal Circuit to diagnose and correct robot action failures efficiently. In simulation experiments, the method reduced failed attempts by up to 37% under degraded conditions and provided interpretable causal reports for each failure. The approach operates without retraining or additional data collection, supporting both autonomous recovery and operator oversight.

Why it matters: This work introduces a tractable, interpretable method for safer robot action testing and failure recovery, potentially improving the reliability and deployment of physical AI systems.

ResearchOfficialarXiv Robotics

SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation

SoftNav introduces a method for injecting entity-level 3D continuous representations as soft tokens into the hidden space of vision-language models (VLMs), enabling goal-directed navigation with minimal training. The approach achieves state-of-the-art results on the HM3D-OVON benchmark and demonstrates zero-shot transfer to other navigation benchmarks and real-world robot deployment, all without retraining or architectural changes.

Why it matters: This work bridges the gap between 3D scene understanding and VLMs, enabling efficient and transferable embodied navigation with minimal data and parameters.

ResearchOfficialarXiv Robotics

AHEAD: Anticipatory Hand-Driven Teleoperation via Human Intent Prediction

AHEAD is a real-time VR teleoperation system that predicts operator intent from hand and head signals to enable proactive robot control. In user studies, it reduced robot reaction latency by 0.6–1.4 seconds compared to baselines and lowered reported operator workload during pick-and-place tasks. The system uses an attention-based classifier to anticipate grasp and placement intentions, allowing the robot to begin actions earlier while maintaining stability.

Why it matters: This work shows that intent prediction can make teleoperation more efficient and less fatiguing, which could improve productivity in repetitive robotic tasks.

ResearchOfficialarXiv Robotics

Open-AoE: Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning

Open-AoE is an open, community-oriented dataset and toolchain for egocentric manipulation, featuring approximately 2,000 hours of video collected by over 500 contributors using more than 400 smartphones. The dataset includes structured annotations such as text labels, MANO-based hand poses, camera trajectories, and temporally localized atomic actions. It also provides a full pipeline for data processing, including temporal action segmentation, semantic annotation, hand and camera trajectory reconstruction, as well as downstream tools for visualization, cross-embodiment retargeting, and model training. This resource is designed to facilitate embodied AI research and human-to-robot transfer by lowering barriers to data contribution and reuse.

Why it matters: Open-AoE provides a large-scale, practical infrastructure for embodied AI research, potentially accelerating advances in manipulation tasks and world modeling.

ResearchOfficialarXiv Robotics

Semantic Audio-driven Understanding for Dynamic Humanoid Whole Body Control

Researchers introduce a multi-modal framework that enables humanoid robots to autonomously select and execute motion skills in real time based on audio input. The system processes both music and speech, using audio fingerprinting and semantic embeddings for music, and a discrete skill library for speech, to guide motion policy selection. Validation is performed both in simulation and on a Unitree G1 humanoid robot, demonstrating robust transfer from simulation to real-world operation.

Why it matters: This work represents a meaningful advance in humanoid robot autonomy, enabling more natural and responsive interactions with dynamic audio cues.

ResearchOfficialarXiv Robotics

Kepler-Encoder-v0.1: A Multimodal Embedding Model That Fuses Robot State and Vision

Researchers present Kepler-Encoder-v0.1, a multimodal encoder that fuses vision, proprioception, and force/torque data into a shared latent space using cross-attention and self-supervised learning. At inference, only vision is used, yet the latent representation recovers force and end-effector state information better than vision-only baselines, particularly in scenarios where visual input is limited. The encoder generalizes across four different robots, and its latent can serve as a training-free invalid-state monitor.

Why it matters: This work demonstrates that incorporating robot state during training enables vision-only representations to capture information about force and contact, potentially improving robot perception and safety without extra sensors at runtime.

ResearchOfficialarXiv Robotics

Agile perceptive multi-skill locomotion for quadrupedal robots in the wild

Researchers introduce APT-RL, a unified framework that enables quadrupedal robots to perform agile, multi-skill locomotion in complex environments using only onboard sensors and computation. The system allows robots to autonomously transition between different gaits and traverse obstacles such as stairs, hurdles, gaps, and uneven terrain, achieving peak speeds of up to 6 meters per second in real-world tests. The approach leverages large-scale motion datasets and reinforcement learning to train robust, transferable locomotion skills.

Why it matters: This work demonstrates a significant advance in quadrupedal robot autonomy, enabling high-speed, versatile navigation of unstructured environments without reliance on external computation.

ResearchOfficialarXiv Robotics

JOP-VLN: Joint On-and-Off Policy Learning for Vision-and-Language Navigation

Researchers introduce JOP-VLN, a framework that integrates imitation learning and reinforcement learning for vision-and-language navigation tasks. The method employs a three-stage training pipeline, combining off-policy imitation learning, DAgger-based exploration, and joint on-and-off policy learning. JOP-VLN achieves success rates of 69.9% on the VLN-CE R2R benchmark and 68.0% on RxR, setting a new state-of-the-art on R2R.

Why it matters: This work demonstrates a significant advance in vision-and-language navigation by effectively bridging imitation and reinforcement learning paradigms, resulting in improved navigation performance.

ResearchOfficialarXiv Robotics

COLMAR: Cooperative View Policy Learning for Multi-Agent Active 3D Reconstruction

COLMAR is a cooperative view policy learning framework designed for multi-agent active 3D reconstruction. It introduces shared policy optimization and reconstruction-aware objectives to improve coordination among agents, reducing redundant observations and enhancing coverage. Experiments on the GLEAM and Replica datasets show that COLMAR achieves up to 54% higher reconstruction accuracy and 49% greater coverage compared to baseline methods.

Why it matters: Improving coordination in multi-agent 3D reconstruction can significantly enhance the efficiency and quality of autonomous exploration and mapping systems.

ResearchOfficialarXiv Robotics

Agentic Reinforcement Learning Framework Improves Robotic Manipulation Robustness

A new agentic reinforcement learning framework enables robots to restore effective execution in manipulation tasks by making high-level decisions based on recent execution history. The approach introduces runtime metrics to assess execution quality and triggers recovery mechanisms when deviations occur. Evaluated on the LIBERO benchmark, the method achieves up to 13.7% higher success rates under standard conditions and up to 39.2% under disturbance conditions, demonstrating improved robustness.

Why it matters: Improving execution robustness in robotic manipulation is crucial for reliable performance in uncertain and dynamic environments.

ResearchOfficialarXiv Robotics

EgoHTR: Egocentric 4D Dataset for Human Terrain Traversal

Researchers have introduced EgoHTR, a dataset comprising 55 egocentric 4D human motion sequences recorded in complex, real-world terrains using wearable sensors and a portable 3D scanner. The dataset includes over 150,000 frames and is evaluated against motion-capture ground truth, demonstrating high accuracy. The authors also show that the dataset can be used to train perceptive locomotion policies, with successful hardware deployment on a Unitree G1 humanoid robot for reconstructed reference motions.

Why it matters: EgoHTR provides a novel resource for developing and benchmarking context-aware locomotion in humanoid robots navigating unstructured environments.

ResearchOfficialarXiv Robotics

Ordinal Stage-State Interface Outperforms Text in VLA Continuation Fine-Tuning

A preprint studying the LIBERO-10 robotics benchmark compares three ways of providing stage information to a VLA (Vision-Language-Action) policy: full-task instruction, current-stage text, and ordinal stage-state. Results show that while full-task instruction achieves the highest mean success rate (57.45%) under direct fine-tuning, the ordinal stage-state interface outperforms both alternatives under continuation fine-tuning, achieving 53.75% mean success rate and exceeding the others in all paired runs.

Why it matters: This work demonstrates that the effectiveness of explicit stage information in VLA policy fine-tuning depends on both the interface representation and the training setup.

ResearchOfficialarXiv Robotics

Topology-Agnostic Mesh Reconstruction of Deformable Objects from Sparse Touch

Researchers have developed a topology-agnostic estimator that reconstructs the full mesh of deformable objects—such as rope, cloth, and soft bodies—using only a few touch inputs and no visual data. The method employs a permutation-invariant cross-attention architecture and achieves a roughly two-thirds reduction in reconstruction error compared to non-learned baselines. Additionally, the approach leverages deep-ensemble uncertainty to guide the selection of subsequent touch points, further improving reconstruction accuracy, especially in challenging scenarios with self-occlusion.

Why it matters: This work advances robotic perception in vision-denied environments, enabling more reliable manipulation of deformable objects using sparse tactile information.

ResearchOfficialarXiv Robotics

VAMP-MR: Vector-Accelerated Motion Planning for Multi-Robot Arms

Researchers have introduced VAMP-MR, a suite of motion planners for multi-robot-arm systems that combine classical planning algorithms with vectorized collision-checking using CPU SIMD instructions. The approach achieves up to two orders of magnitude speedup in both motion planning and execution postprocessing for multi-arm manipulation tasks. The implementation is open-sourced to facilitate further research and development in the field.

Why it matters: This work enables near real-time, collision-free motion planning for multi-robot-arm systems, which could significantly impact industrial automation and robotics research.