arXiv's cs.RO (Robotics) category covers research on physical and simulated robots, including how AI models are used for perception, planning, and control in embodied systems.
A new arXiv preprint introduces HiFi-UMI, a portable system for collecting high-fidelity robot manipulation data without using real robots or external tracking. The study demonstrates that policies trained solely on HiFi-UMI data can be deployed directly on real robots, achieving performance comparable to teleoperation baselines across several policy architectures. The authors also release a large, open-source dataset of synchronized, wide-field demonstrations to support further research.
Why it matters: This work suggests that high-fidelity, robot-free data collection could significantly reduce the cost and logistical barriers to training deployable robotic manipulation policies.
A new arXiv preprint introduces Pigey, a closed-loop orchestrator that decomposes complex robotic tasks into subgoals, directs low-level policies, and recovers from failures, all without additional training. In experiments, Pigey improved performance on the LIBERO-PRO benchmark from 12.8% to 53.3% and increased real-world policy success rates from near-zero to over 90% on tasks requiring reasoning, using existing vision-language-action policies. The approach highlights the potential of separating high-level planning from low-level control in generalist robots.
Why it matters: This result suggests a practical path to dramatically improving generalist robot performance without retraining, which could accelerate real-world deployment and flexibility of robotic systems.
Researchers introduce a method for representing both human and robot hands as sets of 3D particles, with actions defined as end-effector particle displacement fields, to facilitate cross-embodiment learning. They train a graph-based world model on diverse simulated and real hand data, and show that this model, combined with model-predictive control, enables effective dexterous manipulation on previously unseen robotic hands with different kinematics. The study demonstrates that increasing the diversity of training embodiments improves generalization, and that mixing simulated and real data yields better performance than using either alone.
Why it matters: This work provides a shared geometric representation that could accelerate the development of generalist robots capable of learning manipulation skills across a wide range of hand morphologies.
A new preprint report envisions a future where drones transport goods and medical supplies at a scale comparable to highways and the electric grid, but highlights a significant 'capability gap' between current hardware and the software needed for safe, large-scale operations. The report details twelve technical challenges, including scaling to millions of drones, AI assurance, edge-cloud coordination, and regulatory issues, outlining a roadmap for overcoming these barriers.
Why it matters: Addressing these challenges is crucial for enabling transformative applications of drone technology in logistics, disaster response, and infrastructure inspection.
Researchers have introduced the Open Ant, a physical robot platform modeled after the popular Gymnasium Ant environment, to facilitate reinforcement learning (RL) research beyond simulation. The platform enables learning walking policies from scratch in about one hour using either SARSA(λ) or Soft Actor-Critic algorithms, and supports successful transfer of policies trained in simulation to the real robot. Open-source hardware and software are provided to encourage broader adoption and experimentation in real-world RL.
Why it matters: The Open Ant platform makes it significantly easier for RL researchers to test and validate algorithms on real robots, helping to bridge the gap between simulation and real-world deployment.
A new preprint introduces RoboInspector, a pipeline designed to systematically identify and characterize unreliable behaviors in policy code generated by large language models (LLMs) for robotic manipulation. The study analyzes 216 combinations of tasks, instructions, and LLMs, uncovering four main types of unreliable behaviors that can lead to manipulation failures. The authors also propose a refinement approach, guided by feedback from failed policy code, which improves reliability by up to 35% in both simulated and real-world robotic environments.
Why it matters: This work exposes critical reliability challenges in LLM-driven robotic manipulation and demonstrates a practical method to significantly reduce failures, which is important for real-world deployment.
Researchers introduce a hierarchical framework that employs Signal Temporal Logic (STL) as a shared representation to bridge high-level language understanding and low-level robot execution. The system uses a vision-language model to decompose instructions into subtasks with STL specifications, enabling enforcement of precise spatial, temporal, and logical constraints. Real-world tabletop experiments demonstrate that this approach improves the precision, reliability, and interpretability of language-conditioned robot planning compared to standard vision-language-action models.
Why it matters: This work demonstrates that integrating formal specifications like STL can address key limitations of vision-language-action models, enabling more reliable and interpretable robot planning from natural language instructions.
Researchers introduce FARO, a nested kino-dynamic framework designed for rapid feasibility checking and trajectory generation in humanoid loco-manipulation tasks. The system combines a feasibility-guided tree search with a large language model (LLM)-based contact plan sampling strategy to improve search efficiency. Generated trajectories are shown to be trackable by a reinforcement learning (RL)-based controller, enabling execution in real-world scenarios.
Why it matters: This work demonstrates a novel integration of LLM-based sampling, feasibility checking, and RL control to advance fast and practical planning for humanoid robots in previously unseen environments.
Fly0 introduces a framework that separates semantic reasoning from geometric planning for aerial vision-language navigation (VLN). It uses a multimodal large language model (MLLM) to interpret instructions into 2D coordinates, then leverages depth data for 3D localization and generates collision-free trajectories. Experiments show Fly0 improves success rate by over 20% and reduces navigation error by about 50% compared to state-of-the-art baselines in unstructured environments.
Why it matters: Fly0 enables more robust and efficient zero-shot aerial navigation by reducing reliance on continuous MLLM inference, which lowers latency and enhances stability for real-world drone operations.
Researchers introduce Agentic Real2Sim, a framework that leverages vision-language agents to automate the conversion of real-world recordings of object-robot interactions into simulatable digital twins. The system is demonstrated on tasks involving rigid objects, deformable objects, and humanoid motions, domains typically requiring separate pipelines. Agentic Real2Sim achieves conversion success rates comparable to leading models but at a lower computational cost, supporting downstream robotics tasks such as policy learning and evaluation.
Why it matters: This framework could streamline and scale the creation of realistic physics simulations from real-world robot interactions, potentially accelerating robotics research and development.
Researchers introduce ImplicitRDP, an end-to-end diffusion policy that unifies visual planning and reactive force control for contact-rich robotic manipulation. The approach leverages Structural Slow-Fast Learning to process asynchronous visual and force signals, and Virtual-target-based Representation Regularization to improve modality integration. Experiments show that ImplicitRDP outperforms vision-only and hierarchical baselines in terms of reactivity and success rates on challenging manipulation tasks.
Why it matters: This work demonstrates a significant advance in integrating visual and force modalities for robust, reactive robotic manipulation in contact-rich environments.
Researchers propose Masked Visual Actions, a pixel-space control interface that represents robotic actions as partially revealed trajectories in video. A single model, finetuned on 15 hours of real and simulated data, demonstrates strong visual fidelity and controllability across various scenes and robot embodiments. The approach enables forward dynamics prediction, model-based planning, and inverse modeling within a unified framework.
Why it matters: This method offers a new way to connect video models and robotic control without explicit action tokens, potentially simplifying and unifying world modeling for manipulation tasks.
A new policy architecture called LANav, based on linear attention, has been proposed for open-vocabulary object goal navigation. LANav achieves a 36.4% success rate on the HM3D-OVON benchmark, outperforming Transformer-based baselines by 6.3 percentage points. The method also demonstrates an 82% success rate in real-world tests on a Unitree Go2 robot, indicating strong sim-to-real transfer and practical feasibility.
Why it matters: This work demonstrates that structured linear attention mechanisms can surpass Transformer architectures for navigation tasks, potentially leading to more efficient and effective robotic navigation systems.
A new dual-process planning framework for autonomous driving leverages a structured chain-of-thought (S-CoT) schema to represent scene knowledge. The system uses a lightweight Arbiter to route routine scenes to fast meta-action prediction and complex scenes to slower, structured reasoning, with a rule-based validator ensuring consistency between reasoning and actions. On the NAVSIM benchmark, the approach achieves 80.14% planning accuracy, 97.20% logical consistency, and reduces latency by 17.39% compared to always using slow reasoning. The framework also demonstrates strong annotation quality and identifies scenarios where performance may degrade.
Why it matters: This work demonstrates a practical advance in autonomous driving planning by enabling adaptive, verifiable reasoning with reduced manual annotation and improved efficiency.
Researchers introduce RAPT, a lightweight, self-supervised deployment monitor operating at 50Hz that learns nominal execution from simulation and detects out-of-distribution (OOD) states in real-time during sim-to-real transfer for humanoid robots. In simulation, RAPT improves true positive rate by 37% over the strongest baseline at a 0.5% false positive rate, and on hardware, it achieves 89% TPR with 75% semantic failure diagnosis accuracy across 21 categories. The system also provides interpretable, per-dimension predictive deviation signals and post-hoc semantic failure diagnosis using LLM-based reasoning.
Why it matters: This work advances safety and reliability in deploying learned control policies on humanoid robots by enabling high-frequency, accurate OOD detection and interpretable failure diagnosis during real-world operation.
Seg2Grasp is a modular pipeline for suction-based grasping in cluttered bin picking scenarios. It combines a Transformer-based segmentation module, surface normal-based grasp planning, and open-vocabulary classification using Mask-CLIP. Real-world experiments indicate that Seg2Grasp achieves higher success rates and greater adaptability compared to existing methods.
Why it matters: This work demonstrates a significant advance in robotic bin picking by introducing a modular approach that improves robustness and adaptability over end-to-end learning methods.
Researchers introduce a training-free, selective test-time scaling framework for World Action Models (WAMs) in robotics. Their method uses cross-view depth reprojection consistency to rank sampled rollouts, improving task success rates across several benchmarks. With a gating mechanism, the approach recovers 74.8% of the performance gain achieved by always-on scaling, while only using extra computation at 26.2% of decision points.
Why it matters: This work offers a practical way for robots to allocate computational resources more efficiently during inference, enhancing task performance without requiring task-specific labels.
Researchers introduce VLN-AVP, a zero-shot navigation framework for autonomous valet parking that integrates a Bird's-Eye-View model with vision-language models, removing the need for pre-built maps. The system features a hybrid memory mechanism combining short-term perception and long-term topological memory. In simulation, VLN-AVP achieves over 25% higher success rates than prior vision-language navigation methods and demonstrates leading performance in real-world vehicle experiments.
Why it matters: This work advances autonomous vehicle navigation in parking garages by enabling map-free operation guided by natural language, improving scalability and practical deployment.
GeoWorldAD is a geometry-based action model for autonomous driving that grounds trajectory planning in ego-aligned 3D space and anticipates short-horizon scene evolution using latent future geometry tokens. The model achieves state-of-the-art performance on the NAVSIM v1 and v2 benchmarks, effectively balancing collision avoidance with driving progress by leveraging explicit 3D geometry and future scene modeling.
Why it matters: This work shows that incorporating explicit 3D geometry and future scene evolution into planning can significantly improve both safety and efficiency in autonomous driving systems.
Researchers have developed a shared-autonomy framework for robotic manipulation that uses a single RGB-D camera to track operator arm motion and hand gestures without wearables or calibration. The system grounds free-form text prompts for target specification using a vision-language model, and employs a GPU-accelerated model-predictive controller for collision avoidance and a potential field for assisted grasping. Validation on a quadruped mobile manipulator in industrial tasks showed improved safety and task success compared to ablated versions.
Why it matters: This work demonstrates a novel integration of vision-language models and real-time control, enabling safer and more intuitive teleoperation in complex industrial environments.