Researchers introduce a probabilistic model for CLIP's latent space using mixtures of von Mises-Fisher distributions on the unit hypersphere, replacing traditional Gaussian assumptions. This approach enables more accurate and interpretable density estimation, leading to significant improvements in long-tailed and out-of-distribution detection, as well as providing a natural semantic decomposition of embeddings.
Why it matters: The work establishes a geometrically consistent framework for modeling and understanding multimodal representations, potentially enhancing reliability in downstream tasks.
A new framework called LAMaS is proposed for orchestrating multi-agent systems with a focus on reducing end-to-end latency. LAMaS combines constrained optimization and critical-path-aware credit assignment during training with a lightweight controller at inference time to adaptively eliminate redundant agent interactions. Experiments across four benchmarks show that LAMaS reduces latency by over 50% compared to existing learning-based baselines, while maintaining competitive or better accuracy. The approach is modular and transfers easily to other multi-agent systems.
Why it matters: This work addresses the significant challenge of inference latency in multi-agent systems, enabling faster and more efficient coordination without sacrificing accuracy.
DevicesWorld is a new benchmark designed to evaluate LLM-based agents on tasks that require collaboration across mobile, desktop, and IoT devices. The benchmark features 6,140 tasks and a unified evaluation framework, revealing that current leading agents achieve only a 12.5% success rate. Analysis of agent failures highlights common issues such as difficulties in information acquisition and confusion between source and output devices.
Why it matters: DevicesWorld fills a critical gap by enabling systematic evaluation of agents' abilities to operate across heterogeneous device environments, which is essential for real-world applications.
Researchers have introduced Samba, a hybrid Mamba-based model for audio-visual navigation that replaces conventional GRUs with a Mamba State Encoder and incorporates an Audio Mamba Encoder to better capture global time-frequency dependencies. On the Matterport3D dataset, Samba achieves an 11.3% improvement in navigation success rate over state-of-the-art models, with even greater gains reported on the Replica dataset. The model demonstrates strong generalization to unheard sound sources and unseen scenes.
Why it matters: Samba modernizes the core architecture for audio-visual navigation, offering improved performance and efficiency, and sets a new direction for future research in embodied AI navigation.
A new framework, Self-Correcting Coupled Markov Jump Processes (SC-CMJP), is introduced to enable concurrent image and text generation by coupling masked diffusion models across modalities. The associated training-free sampler, CO₂Jump, demonstrates state-of-the-art performance on joint multimodal tasks such as image editing and visual reasoning, as shown on newly released large-scale benchmarks.
Why it matters: This work represents a significant advance in multimodal AI by enabling real-time, cross-modal correction and coherent joint outputs, addressing limitations of previous systems that treated modalities separately.
A new framework, FM$^2$, is introduced for federated learning of medical foundation models across institutions with heterogeneous imaging modalities. FM$^2$ features dual Mixture-of-Experts modules and a Heterogeneous Modality Alignment regularizer to address both overlapped and non-overlapped modality distributions. It also leverages caption-enhanced learning using GPT-4o-generated captions to facilitate cross-client representation transfer. Experiments on classification, caption learning, and medical VQA tasks show FM$^2$ consistently outperforms existing federated baselines.
Why it matters: This work advances privacy-preserving medical AI by enabling collaborative training of multimodal foundation models across hospitals, even when participating sites have entirely different imaging modalities.
Researchers have introduced a plug-and-play token reduction framework for vision-language models that eliminates the need for attention maps and pairwise similarity comparisons. By leveraging entropy-based importance estimation and transformation-induced consistency signals, the method selects a compact and diverse set of visual tokens. Experiments across multiple benchmarks show that the approach maintains competitive accuracy even under aggressive token compression, offering a favorable accuracy-efficiency trade-off.
Why it matters: This work could make vision-language models more practical for deployment on resource-constrained edge devices by reducing computational overhead without significant loss in performance.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers have identified a novel cross-application 'Action Rebinding' attack that targets Android GUI agents powered by large multimodal models. This attack allows a malicious app with zero permissions to hijack the agent's execution, enabling privileged operations such as file deletion, SMS transmission, and app uninstallation. The attack exploits the observation-action gap in the agent's reasoning process and achieves a 100% success rate for atomic hijacking, while evading detection by commercial malware scanners.
Why it matters: This work exposes a fundamental security vulnerability in emerging high-privilege GUI agents on Android, revealing that current sandboxing and malware detection mechanisms are insufficient to prevent such attacks.
Researchers introduce the Foundation Model Deployment Portfolio (FMDP) problem, formulating it as a mixed-integer program to minimize the total cost of ownership for deploying large language and vision-language models across transportation management center (TMC) tasks. Their case study demonstrates that a mixed deployment strategy—using open-source APIs for most functions and a closed API only where necessary—can reduce costs by 97% compared to the cheapest all-closed-API baseline, achieving a monthly cost of $34. The study also analyzes when on-premise GPU investment becomes cost-effective.
Why it matters: This work offers a formal, practical framework for optimizing the deployment of foundation models in cost-sensitive, multi-function environments like transportation management centers.
Researchers have developed Mycelium, an active shared workspace that connects human scientists and AI agents by capturing and routing observations and hypotheses to relevant team members. In an empirical test during a biological multi-omics campaign, Mycelium enabled a local analytical finding to inform cross-expert constraints and experimental design. The system provides a computational framework for networked intelligence, modeling scientific collaboration as sparse conditional computation over distributed contexts.
Why it matters: This work advances the field by enabling scalable, networked collaboration between humans and AI, addressing the challenge of coordinating diverse expertise for complex scientific problems.
UrbanAgent is a new framework that approaches urban region profiling as a reasoning-driven inference task, using multiple agents—each dedicated to a specific data modality—to address inconsistencies across data types. The system incorporates tool-augmented evidence retrieval and reinforcement learning, enabling agents to actively acquire and verify information. Experiments on global datasets for carbon emissions, GDP, and population estimation show UrbanAgent achieves an average 8.1% improvement in R2 over existing methods and demonstrates strong generalization to unseen cities.
Why it matters: This work presents a novel agent-based approach that improves the robustness and generalization of urban region profiling by explicitly reasoning over multimodal data, moving beyond traditional correlation-based methods.
Thinking Machines Lab has released its first open model, Inkling, a 975-billion-parameter multimodal AI trained to understand video and audio. The model is available on Together AI's platform from day one, serving as the company's first public demonstration after a year and a half of developing AI infrastructure largely out of public view.
Why it matters: Inkling could help position Thinking Machines Lab as a competitor to Anthropic and OpenAI in the open model space.
A study of 22 Triton and TileLang GPU kernels reveals that correctness-based evaluation can overlook severe performance issues—one TileLang LayerNorm kernel, for example, passes correctness checks but is over 300× slower than the PyTorch baseline. The underlying causes of inefficiency vary by kernel family, with some due to repairable authoring defects and others stemming from code-generation or autotuning limitations. The authors propose two lightweight screening methods—library-relative efficiency and roofline utilization—that successfully identify all functionally correct but inefficient kernels in their evaluation.
Why it matters: This work exposes a critical flaw in current GPU kernel evaluation practices and offers practical tools to ensure that functionally correct kernels also meet performance expectations.
RESOURCE2SKILL is a framework that extracts and organizes executable skills for software agents from multimodal resources such as tutorial videos, code repositories, and articles. These skills are structured in a hierarchical Skill Wiki that integrates text, code, visual examples, and metadata, enabling agents to retrieve and compose relevant skills for complex tasks. In evaluations across seven authoring domains, RESOURCE2SKILL improved agent performance by an average of 11.9 percentage points compared to agents without skill libraries, and outperformed strong baselines in most tested scenarios.
Why it matters: This work demonstrates a significant advance in enabling software agents to autonomously acquire and utilize diverse, human-created resources as reusable skills, enhancing their adaptability and effectiveness in complex tasks.
Researchers have introduced RoboDesign1M, a dataset containing 1 million multimodal samples sourced from scientific literature across various robotics domains. The dataset is designed to support tasks such as automated design generation, text-based design retrieval, and AI-powered design assistants. Experiments demonstrate that RoboDesign1M provides a challenging benchmark for design image generation, visual question answering, and design image retrieval.
Why it matters: RoboDesign1M addresses the scarcity of large-scale robot design datasets, potentially advancing research and development in AI-driven robotic design automation.
NetForge RL is a multi-agent simulation environment designed for cyber defense research, featuring procedurally generated enterprise and operational technology (OT) networks. It supports red-blue team self-play under partial observability, with actions mapped to the MITRE ATT&CK framework and a JAX backend capable of 250,000 steps per second. The environment includes reference baselines, diagnostic probes for defensive skills, and an evaluation runner for reproducible benchmarking.
Why it matters: NetForge RL offers a realistic and reproducible testbed for developing and evaluating multi-agent reinforcement learning systems in cyber defense scenarios with adaptive adversaries and noisy, partial observations.
SinAE is a flow-matching autoencoder that uses a single vanilla Transformer architecture to handle molecules, crystals, and proteins, without relying on domain-specific operators. It achieves near-lossless reconstruction across these domains and demonstrates strong generative performance, with joint training on molecules and crystals leading to improvements in both domains.
Why it matters: This work provides a unified approach for generative modeling across diverse atomic systems, enabling cross-domain transfer and potentially alleviating data scarcity issues.
A new framework for Composed Image Retrieval (CIR) achieves state-of-the-art zero-shot performance on the CIRR dataset (44.04% R@1, an improvement of 8.79%) without using visual features. The approach combines attribute-augmented hybrid scoring to address the loss of visual detail and LLM-based reranking to ensure semantic consistency. Ablation studies confirm that both components contribute to the performance gains.
Why it matters: This work shows that vision-free methods can outperform traditional approaches on complex multimodal retrieval tasks, suggesting new directions for efficient and simplified retrieval systems.
A new framework called PFAdapter introduces hierarchical LoRA decomposition to separate global-shared and local-private parameters for federated fine-tuning of multimodal large language models (MLLMs). By synchronizing only the global-shared components and keeping local adaptations private, PFAdapter reduces communication costs by nearly 50% and achieves accuracy improvements of 2.4% to 4.8% on several medical and multimodal datasets. The approach also uses orthogonality regularization to enforce strict separation between parameter types, preventing redundant feature learning.
Why it matters: This work offers a practical advance for deploying personalized, communication-efficient AI at network edges, addressing key challenges in federated learning for multimodal models.
SlimPer introduces a new approach to personalized ranking in recommendation systems by iteratively refining a compact knowledge base, rather than relying on large intermediate representations that scale with user history length. This design achieves O(N) per-layer computational cost and fixed-size intermediate representations, allowing the model to handle over 10,000 user history events efficiently. Deployed on Instagram Reels and Feed, SlimPer has demonstrated measurable improvements in user engagement while unifying sparse, dense, and sequence features within a single model backbone.
Why it matters: SlimPer enables deeper and more efficient personalization in large-scale recommendation systems by decoupling model depth from user history length, reducing compute and memory requirements.