Multimodal AI news — Page 2

AI systems that understand or generate combinations of text, images, audio, video, and other types of data.

ResearchOfficialarXiv Cryptography and Security

Sarus: Privacy-Preserving Multi-Vendor Perception Fusion via Homomorphic Encryption

Researchers introduce Sarus, a framework that leverages homomorphic encryption to enable privacy-preserving fusion of perception outputs from multiple autonomous vehicle vendors. The system aggregates encrypted detection data, protecting both proprietary model details and sensitive environmental information. Experiments on the KITTI dataset demonstrate that Sarus achieves improved scene-level coverage by combining complementary detections from different modalities, with linear computational scaling and bounded overhead, indicating feasibility for real-time deployment.

Why it matters: This work demonstrates a practical approach to secure, privacy-preserving cooperative perception in autonomous vehicles, addressing key barriers to multi-vendor collaboration without exposing sensitive data.

ResearchOfficialarXiv Computer Vision

VLMs Confuse Anomalies with Hazards, Study Finds

A new study introduces a distinction between hazard and anomaly in evaluating Vision-Language Models (VLMs) for safety reasoning. The researchers found that VLMs often misinterpret anomalous situations as hazardous, relying too heavily on contextual irregularity as an indicator of danger. The study also provides a public dataset to facilitate further research in this area.

Why it matters: This work exposes a critical failure mode in VLM safety reasoning that could impact their deployment in safety-critical systems.

ResearchOfficialarXiv Computation and Language

Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning

A new preprint introduces Stochastic Meta-Unlearning (SMU), a bilevel framework designed to improve machine unlearning in vision-language models (VLMs). SMU leverages VLM-level feedback to optimize the language backbone for unlearning, addressing the challenge that text-only unlearning is insufficient when image information is present. Experimental results on two VLMs and multiple datasets show that SMU achieves superior forget-retain trade-offs compared to existing baselines and demonstrates transferability to new unlearning targets and methods.

Why it matters: This work advances the field of machine unlearning by demonstrating that multimodal feedback is essential for effective and transferable unlearning in vision-language models.

ResearchOfficialarXiv Computation and Language

Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio

Researchers present Fusion Embedding, a family of models that extend a frozen vision-language embedding base to include audio without updating its parameters. The first generation uses a 16.4M-parameter connector for audio, while the second adds modality-gated deep adapters (44.2M parameters) that activate only for audio inputs. The models enable zero-shot audio-image retrieval by aligning audio to text, and both generations can be trained in hours on a single GPU.

Why it matters: This work demonstrates a practical method for unifying text, image, video, and audio embeddings in a single space, enabling cross-modal retrieval without requiring paired audio-visual training data.

ResearchOfficialarXiv AI/ML

Attacking Graph Foundation Models Through Their Shared Representation

Researchers have identified a novel attack surface in graph foundation models: the alignment layer that maps diverse graph inputs into a shared representation. They demonstrate that targeted perturbations in this representation space can severely degrade model performance, with some models—such as OpenGraph—being particularly vulnerable. Additionally, input-space attacks that modify edges, features, or text can eliminate at least half of correct predictions in three out of six tested models.

Why it matters: This work exposes a previously unexamined vulnerability in graph foundation models that can be exploited at inference time without access to training data, raising new security concerns for real-world deployments.

ResearchOfficialarXiv AI/ML

MAGE: Human-Like Macro Placement via Agentic Multimodal Reasoning

MAGE is a multimodal, multi-agent framework for macro placement refinement in chip design, combining structured floorplanning rules, visual checks, and iterative refinement. In tests across nine designs, MAGE achieved geometric-mean improvements of 11.1%-19.3% in Worst Negative Slack (WNS) and 70.0%-74.0% in Total Negative Slack (TNS) over commercial macro placers. On three designs with human-expert baselines, MAGE outperformed human experts by 18.3% in WNS and 72.5% in TNS, and also improved human-likeness metrics by 6%-48% over all baselines.

Why it matters: This work demonstrates a significant advance in automated chip floorplanning, showing that a multimodal, agentic approach can outperform both commercial tools and human experts in macro placement.

ResearchOfficialarXiv AI/ML

From Agent Failure Paths to Quantified Residual Risk: A Compositional Framework for Resilient Agentic AI

A new framework, CPSAINT, is introduced for agentic AI, offering a seven-layer integrity decomposition that spans physical state, sensors, data, compute, actuators, environment, and time. Paired with the FRIESA-K residual-risk functional, the approach maps specific failure paths to quantified risk instances using a controlled absorbing Markov model, enabling formal, composable risk assessment. The framework is demonstrated on both a warehouse robot and a financial-services agent, maintaining consistent semantics and structure across domains.

Why it matters: This work provides a formal and composable method to quantify residual risk in agentic AI systems, addressing a key challenge in AI safety and governance.

ResearchOfficialarXiv Statistical ML

Semi-Supervised Conditional Diffusion via Label Augmentation

Researchers propose label-augmented conditional diffusion (LACD), a method that incorporates unlabeled data into conditional diffusion models by assigning a designated trivial label and performing joint denoising score matching. Theoretical analysis provides sufficient conditions for identifiability and shows that, with enough unlabeled data, LACD achieves faster convergence in total variation distance compared to purely supervised approaches. Experiments on synthetic, image, and tabular data demonstrate improved sample efficiency and generative performance.

Why it matters: This work provides a principled approach to leveraging unlabeled data for conditional diffusion models, potentially reducing reliance on costly labeled datasets.

ResearchOfficialarXiv Software Engineering

CLOSER-Bench: A Benchmark for Budgeted Cross-Stage Hardware Design Closure by AI Agents

CLOSER-Bench is a new benchmark designed to evaluate AI agents on the complex task of hardware design closure across multiple stages, including RTL, synthesis, and physical design. The benchmark introduces a controlled protocol that measures agent performance on tasks spanning from specification to GDS, capturing both final quality and intermediate progress. Initial experiments reveal a significant gap between task completion and achieving full design closure, highlighting the limitations of current approaches and motivating the need for sequential decision-making strategies.

Why it matters: CLOSER-Bench provides a unified and rigorous framework for assessing AI agents on long-horizon, multi-stage hardware engineering tasks, addressing a key challenge in automating hardware design workflows.

ResearchOfficialarXiv Software Engineering

Teach it to stop, not just to click: Reproducibility and repair in agentic computer-use RL

A preprint investigates reproducibility in reinforcement learning for agentic computer-use, focusing on a 35B-parameter agent. The study finds that single-run evaluations are unreliable due to high variance from data sampling and nondeterminism, with evaluation variance itself being negligible. Repairability is shown to be two-tiered: fixed-token interventions are reliably effective, while open-ended corrections are only partially successful. The authors also release a library to facilitate routine multi-seed evaluation reporting.

Why it matters: This work exposes critical reproducibility challenges in agentic RL and introduces practical tools and analysis for more reliable evaluation and repair of computer-use agents.

ResearchOfficialarXiv Robotics

SinD 2.0: Multi-City UAV Dataset for Autonomous Driving Safety Validation

SinD 2.0 is a large-scale drone-based dataset capturing six signalized intersections across four Chinese cities, featuring 32,682 safety-critical events and hierarchical semantic risk annotations. It includes a full-stack testing toolchain for scenario extraction and closed-loop testing, supporting cross-domain safety analysis of autonomous driving systems. The dataset addresses limitations of existing resources by providing greater geographical diversity and detailed risk labeling.

Why it matters: SinD 2.0 offers a standardized, diverse benchmark with rich risk annotations, enabling more robust and generalizable safety validation for autonomous driving systems at intersections.

ResearchOfficialarXiv Multiagent Systems

SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategies Refinement in E-Commerce Recommendation

SR-Agent is a new agentic framework designed to automate the refinement of post-ranking strategies in industrial e-commerce recommender systems. It integrates three specialized agents to identify user-perceived issues, diagnose recurring problems, and apply constrained refinements with rollback capability. In a one-month A/B test on the Kuaishou platform, SR-Agent increased order volume by 0.71%, browsing depth by 0.34%, and clicked-category diversity by 0.48%, while reducing manual effort and operational costs.

Why it matters: SR-Agent represents the first deployed framework to fully automate the loop of refining post-ranking strategies in industrial recommender systems, demonstrating measurable improvements in key e-commerce metrics and operational efficiency.

ResearchOfficialarXiv Machine Learning

AI Swarm System Autonomously Identifies IGF2 as Colorectal Cancer Vulnerability

A new preprint introduces Octopus, a neuro-symbolic architecture that integrates local LLM swarms with algorithmic physics engines to autonomously generate and test biomedical hypotheses. In a fully unsupervised analysis of colorectal cancer data, the system identified IGF2 as a vulnerability linked to 5-Fluorouracil resistance, with findings validated across in vitro, in silico, and in vivo models. This demonstrates an end-to-end, automated approach to mechanistic biomedical discovery.

Why it matters: This work represents a significant advance in autonomous scientific discovery, showing that AI systems can bridge reasoning and mechanistic validation to accelerate translational cancer research.

ResearchOfficialarXiv Multiagent Systems

Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer

A new preprint presents an AI research agent for materials science that rigorously evaluates modeling changes using held-out data, aiming to ensure real-world transferability of its discoveries. In seven separate searches covering 701 modeling changes across ten Matbench endpoints, nine out of ten selected interventions outperformed baselines on previously unseen data, with notable improvements such as 17.4% for band gap prediction and 18.6% for steel strength. The study introduces a robust evaluation framework for closed-loop AI scientists, emphasizing code reusability and transfer across tasks.

Why it matters: This work provides evidence that closed-loop AI agents can generate scientific advances that generalize to new data, addressing concerns about overfitting and reproducibility in automated research.

ResearchOfficialarXiv Multiagent Systems

FOCAL: On-Device Multi-Agent System for Efficient Desktop Activity Summarization

FOCAL is a privacy-first, on-device multi-agent system designed to summarize desktop interaction streams into task-organized personal logs. By employing a filter-plan-log architecture, FOCAL reduces token consumption by 60.4% and Vision-Language Model (VLM) calls by 72.3% compared to baselines, while improving key information recall from 0.38 to 0.61. The system also maintains high accuracy and recall even during task interruptions, outperforming baseline approaches.

Why it matters: FOCAL demonstrates a significant advance in efficient, privacy-preserving summarization of continuous desktop activity, enabling practical on-device personal logging.

ResearchOfficialarXiv Machine Learning

Building2Building: A Large-Scale Benchmark for Generalizable Real-World Reinforcement Learning

Researchers have introduced Building2Building (B2B), a large-scale benchmark suite for reinforcement learning (RL) focused on realistic HVAC control environments using the EnergyPlus simulator. B2B features a parametric building generator that creates diverse building configurations and defines tasks addressing key RL challenges such as goal adaptation and cross-domain transfer. The benchmark is designed to systematically study generalization and transfer in continuous control settings.

Why it matters: B2B provides a physically grounded and diverse testbed for advancing RL research on generalization, with potential real-world impact on energy efficiency in building HVAC systems.

ResearchOfficialarXiv Machine Learning

Compact CNNs for AI-Based Drone Detection Using RF Signals

Researchers have developed lightweight convolutional neural networks (CNNs) that detect drone video transmitter signals from radio-frequency (RF) emissions. By using rasterized time-domain images as input, these models eliminate the need for frequency-domain preprocessing and achieve high detection accuracy with low computational cost. The approach was validated both offline and in real-time using a GNU Radio signal processing chain.

Why it matters: This work demonstrates a practical advance in efficient, real-time drone detection suitable for embedded electronic warfare and RF monitoring systems.

ResearchOfficialarXiv Machine Learning

RobustMAD: Benchmark Reveals Robustness Gaps in Multimodal Small Language Models for Industrial Anomaly Detection

Researchers have introduced RobustMAD, the first benchmark specifically designed to evaluate the real-world robustness of multimodal small language models (MSLMs) for industrial anomaly detection. The study finds that while top-performing MSLMs can outperform larger models such as GPT-5 Nano, they still exhibit critical failure modes, including fragile multimodal grounding, insufficiently comprehensive responses, and hallucinations on unanswerable queries. The benchmark provides actionable guidance for improving the deployment of MSLMs in safety-critical industrial environments.

Why it matters: RobustMAD exposes key robustness gaps in compact MSLMs, highlighting operational risks that must be addressed before these models can be safely deployed in smart factories.

ResearchOfficialarXiv Machine Learning

Self-Evolving Just-In-Time Memory Enables Proactive Safety in Embodied Agents

A new framework called Self-Evolving Just-In-Time Memory has been proposed to improve safety in embodied agents, such as household robots, by shifting from traditional guardrails that can stall progress to proactive hazard mitigation. The framework introduces components like a Risk-Sufficient Topological Belief Graph, Agency-Grounded Factual Memory, and Experience Memory with procedural Meta-Skills. Experiments on the IS-Bench benchmark show a 30.3% increase in Safe-Success rate for the Qwen3-VL-8B model, indicating more effective hazard mitigation without sacrificing task completion.

Why it matters: This work offers a significant advance in embodied agent safety by enabling proactive hazard mitigation that maintains task efficiency, addressing a key limitation of existing safety approaches.

ResearchOfficialarXiv Machine Learning

NeoST: First Spatio-Temporal Foundation Model Pre-Trained on Pure Synthetic Data

Researchers introduce NeoST, a spatio-temporal foundation model pre-trained exclusively on procedurally generated synthetic data. NeoST features a latent-space reasoning architecture that generates and refines multiple future trajectories, aiming to avoid sequential error accumulation. Experimental results show that NeoST outperforms existing spatio-temporal foundation models on a range of real-world benchmarks, with improved long-horizon stability and inference efficiency.

Why it matters: This work suggests that synthetic data pre-training can address distributional bias in spatio-temporal modeling, potentially leading to more robust and generalizable AI systems for applications such as weather, climate, and physical simulations.