What changed in AI — Page 73

Policy & SafetyOfficialarXiv Cryptography and Security

AI Coding Agents Vulnerable to Supply-Chain Attacks via Setup Documentation

A new preprint demonstrates that AI coding agents can be compromised by malicious instructions embedded in project setup documentation, such as README files. The study systematically evaluates supply-chain attacks delivered through documentation across multiple ecosystems (npm, Cargo) and agent harness-model combinations, showing that agents often install untrusted or vulnerable dependencies. Security-oriented prompts only partially mitigate these risks, and the effectiveness of defenses varies by harness and model.

Why it matters: This exposes a critical security vulnerability in AI coding agents, where project documentation can be weaponized as an attack vector, threatening software supply chain integrity.

ResearchOfficialarXiv Cryptography and Security

PA-HDP: Dynamic Privacy Protection for RAG Under Query-Driven Risks

A new framework, PA-HDP, is proposed to address privacy risks in retrieval-augmented generation (RAG) systems by recognizing that privacy leakage is dynamic and depends on the user's query. PA-HDP uses a prompt-aware risk hierarchy and adaptive protection mechanisms to assess and mitigate privacy risks on a per-query basis. Experimental results show that PA-HDP reduces privacy leakage and maintains retrieval quality better than previous static, document-level approaches.

Why it matters: This work introduces a more nuanced and effective approach to privacy in RAG systems by adapting protection to the actual sensitivity of content in response to specific user queries.

ResearchOfficialarXiv Computation and Language

CityLLM enables natural-language queries of semantic 3D city models

CityLLM is a framework that integrates spatial and graph databases with large language models (LLMs) to allow users to query semantic 3D city models using natural language. In tests on a CityJSON dataset of Rotterdam, CityLLM achieved 85.2–100% answer correctness and 100% query success across 54 queries. The system supports iterative query refinement and chaining across multiple databases, aiming to make complex urban data more accessible to non-experts.

Why it matters: This work could make it easier for a wider range of users to access and analyze complex 3D city data, potentially broadening the use of such models in urban planning and research.

ResearchOfficialarXiv Computation and Language

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

A new preprint demonstrates that generating chains of thought conditioned on the correct answer can significantly degrade the quality of reasoning data used for distilling large language models (LLMs). The study finds that this approach can reduce verifiable-reasoning accuracy by up to 27 points on the hardest problems, and that standard correctness filtering fails to detect the degradation. The authors recommend generating answer-blind chains of thought to avoid introducing hidden flaws into reasoning data.

Why it matters: This result challenges a common practice in LLM reasoning distillation and highlights the risk of introducing subtle but significant flaws when conditioning on answers.

ResearchOfficialarXiv Computation and Language

PReM: Learning What to Preserve and When to Refresh for Context Compression

PReM is a context-compression framework for language models that maintains long context as internal layer-wise key-value (KV) memory, learning dynamically what information to preserve and when to refresh it. The approach introduces a dedicated memory layer and a special memory token, trained using Phase-Separated Refresh Training to align memory selection with generation needs. Experiments on 32K-token contexts demonstrate that PReM outperforms strong baselines at 16x and 32x compression, achieving a favorable balance between answer quality and inference efficiency.

Why it matters: This work offers a novel approach to efficient long-context inference by enabling dynamic adaptation of compressed context, potentially improving both quality and efficiency in large language models.

Policy & SafetyOfficialarXiv Cryptography and Security

Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

A new preprint investigates prompt injection vulnerabilities in agentic systems that use persistent memory, focusing on Anthropic Claude Code and OpenAI Codex across four models. The study finds that while it is challenging for attackers to overwrite memory files directly, malicious payloads already present in memory can persist and compromise both current and future sessions. The persistence and effectiveness of these attacks vary depending on the system, model, and adversarial objectives.

Why it matters: This work demonstrates that persistent memory in agentic systems introduces a novel and significant attack vector for prompt injection, underscoring the need for new defenses that secure memory updates without impeding beneficial adaptation.

ResearchOfficialarXiv Computation and Language

Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents

Researchers introduce a multi-agent framework that combines Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Retrieval-Augmented Generation (RAG) to simulate political coalition negotiations using large language models. Applied to the 2019 Flemish election, the system produces stable coalition rankings and uses a novel tracing method to link agreement clauses back to party manifestos. The framework also benchmarks simulated agreements against real-world outcomes, offering a transparent and interpretable approach to studying party compatibility.

Why it matters: This work provides a transparent and scalable method for simulating and auditing political negotiations with LLMs, potentially advancing computational political science research.

ResearchOfficialarXiv Computation and Language

Reasoning Graphs Enhance Robustness of LLM Authorship Attribution

A new method uses graph neural networks to analyze reasoning structures extracted from LLM-generated text for authorship attribution. This approach significantly outperforms traditional baselines, achieving up to 27 percentage points higher accuracy under obfuscation attacks such as paraphrasing and backtranslation, and 19 points higher on texts from previously unseen model versions.

Why it matters: The method offers a more robust solution for detecting LLM-generated text, addressing vulnerabilities of existing techniques to paraphrasing and model updates.

ResearchOfficialarXiv Cryptography and Security

FlowGuard: Evidence-Grounded Security Detection for MCP Interactions

FlowGuard is a new security detection system for the Model Context Protocol (MCP) that combines semantic analysis with runtime evidence to assess risks in LLM agent tool interactions. In evaluations on an executable benchmark of 1,880 MCP cases, FlowGuard achieved F1 scores of 0.879 and 0.942 for Command Injection and File System Access vulnerabilities, respectively. It also reduced end-to-end latency by up to 2.23x compared to existing dynamic scanners and reported 523 findings across 326 real-world servers.

Why it matters: FlowGuard addresses a critical gap in MCP security by reliably detecting both execution-based and semantic risks in LLM agent tool interactions, improving upon the limitations of current scanners.

ResearchOfficialarXiv Cryptography and Security

Fully Automated End-to-End Adversary Emulation from MITRE ATT&CK-Based Cyber Threat Intelligence Using LLMs

Researchers have developed a fully automated framework that uses large language models (LLMs) to generate, execute, and revise adversary emulation playbooks directly from MITRE ATT&CK-aligned cyber threat intelligence (CTI) reports. The system unifies playbook generation, execution, and failure recovery, eliminating the need for manual intervention present in previous approaches. In evaluations on 11 CTI reports using four leading LLMs, the framework achieved its best performance with Claude Sonnet 4.5, reaching 84.22% execution success after revision and a CTI F1 score of 60.50%. The failure recovery mechanism improved execution success rates by 14.59 to 17.23 percentage points across all tested models.

Why it matters: This work represents a significant advance in automated cybersecurity testing, enabling more scalable and responsive adversary emulation by minimizing manual effort.

ResearchOfficialarXiv Computation and Language

Routing Ceilings Are Domain-Independent: Structural Prior Injection in Code Security Vulnerability Detection

A new preprint replicates findings from mathematical reasoning in the domain of code security, showing that injecting structural priors (cheatsheets) into LLM prompts dramatically boosts in-distribution performance but leads to severe out-of-distribution collapse. The study evaluates three large language models across multiple vulnerability types and finds that the trade-off between in-distribution gains and out-of-distribution failures is robust across domains. Iterative recalibration of cheatsheets does not resolve the collapse, supporting the router hypothesis and suggesting that distribution-aware training is needed.

Why it matters: This work reveals a fundamental, cross-domain limitation of prompt-based structural priors for LLMs, emphasizing the need for distribution-aware approaches in real-world applications.

ResearchOfficialarXiv Computation and Language

Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making

A new method called Multi-Head Latent Control introduces a lightweight layer that reads hidden-state trajectories from a frozen large language or vision-language model to generate control signals at deployment. This enables agents to make decisions such as deferring to a stronger model, requesting clarification, invoking tools, or abstaining, all without modifying the backbone model. Experiments show that this approach can reduce reliance on large models by up to 90.7% on the AndroidWorld benchmark while maintaining most of the performance, and also improves tool-use decision quality.

Why it matters: This technique offers a practical way to enhance the efficiency and reliability of LLM agents by enabling nuanced control decisions directly from model internals, reducing costs and the need for external orchestration.

ResearchOfficialarXiv Computation and Language

Gold-Guided Programmatic Distillation Boosts Financial Reasoning in Smaller LLMs

Researchers introduce a method that distills numerical reasoning abilities from a large language model into a smaller one using execution-verified Python programs, rather than natural-language rationales. On the TAT-QA benchmark, their 7B-parameter student model achieves 87.00 EM / 87.18 F1, outperforming both its 72B-parameter teacher (78.46 EM) and strong existing baselines. The approach includes an iterative recovery stage to further improve training by incorporating newly verified programs.

Why it matters: This work demonstrates that programmatic distillation can enable smaller models to outperform much larger ones in complex financial reasoning tasks, potentially reducing computational costs for high-accuracy applications.

ResearchOfficialarXiv Cryptography and Security

DataShield: Uncovering Risky Fine-Tuning Data Across LLMs Through Consensus Subspace Alignment

DataShield is a framework designed to identify risky fine-tuning data for large language models (LLMs) by measuring alignment with consensus safe and unsafe subspaces derived from multiple safety-aligned models. The method enables both sample-level filtering and segment-level masking, reducing attack success rates by 14.6% and 32.3% respectively, while preserving downstream utility.

Why it matters: This work offers a transferable approach to mitigating safety degradation during LLM fine-tuning by filtering risky data without requiring target-model-specific computation.

ResearchOfficialarXiv Computation and Language

MARS: Multi-hop Adaptive Retrieval and SPARQL Generation for Knowledge Graph Question Answering

MARS is a knowledge graph question answering (KGQA) approach that integrates large language models (LLMs) with knowledge graphs without requiring model fine-tuning. It uses a structured retrieval process to iteratively gather relevant information and generate SPARQL queries, adapting the retrieval depth to the question. MARS demonstrates competitive performance on established KGQA benchmarks and is efficient and scalable.

Why it matters: MARS offers a scalable way to improve the reliability of LLMs in knowledge-intensive tasks by grounding answers in explicit, updatable symbolic knowledge without the need for costly fine-tuning.

ResearchOfficialarXiv Computation and Language

D-Cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding

D-Cut is an adaptive pruning method for speculative decoding in large language models that selects draft tokens jointly across a batch, focusing verification resources on tokens most likely to be accepted. By using cross-request pruning and a runtime cost model, D-Cut adapts to different deployment environments. Experiments show that under high concurrency, D-Cut increases average speedup from 1.26× to 1.65× and achieves up to 3.0× speedup on mixture-of-experts (MoE) models compared to autoregressive decoding.

Why it matters: D-Cut offers a significant advance in efficient large language model inference, particularly under high concurrency, by reducing wasted computation and improving throughput without compromising output quality.

ResearchOfficialarXiv Computation and Language

Harness Engineering Enables Smaller LLMs to Outperform Larger Models in Academic Supervision Tasks

A new preprint compares a GPT-5-based chatbot without scaffolding to a multi-module system (ASuS) that wraps the smaller GPT-4o-mini model in a LangGraph harness for academic supervision. The harnessed system achieved a pooled mean score of 4.08 versus 1.23 for the larger, unscaffolded model across ten independent raters, excelling in reliability, consistency, and other harness-related dimensions. The study's ablation analysis further suggests that the benefits of harness engineering are largely independent of the underlying model size.

Why it matters: This work provides strong evidence that deliberate harness engineering can yield more reliable and consistent AI systems than simply scaling up model size, especially in high-stakes, structured domains.

ResearchOfficialarXiv Cryptography and Security

Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents

A new preprint proposes that evaluations of security agents should incorporate economic efficiency, not just task success, by comparing models at fixed cost levels. The study finds that offensive CTF agent performance improves with increased compute resources, while defensive SOC agent success relies more on disciplined tool use and telemetry navigation than on raw reasoning budget. The authors introduce a cost-aware evaluation framework and provide an interactive website with their results.

Why it matters: This work introduces a cost-aware evaluation framework that could reshape how security agents are benchmarked for practical operational use.

Policy & SafetyOfficialarXiv Cryptography and Security

Disclosure Divergence: Measuring Privacy Policy and Data Safety Misalignment at Scale

A large-scale study of 6,051 Android apps reveals significant inconsistencies between privacy policies and Google Play Data Safety labels, particularly for sensitive data types such as personal information and device identifiers. The research finds that sharing disclosures are less consistent than collection disclosures, and that apps involving persistent monitoring and communication present higher privacy risks. The study introduces a sensitivity-weighted risk score to better assess these misalignments.

Why it matters: This work exposes structural weaknesses in current privacy disclosure mechanisms, highlighting the need for improved verification and transparency in app data practices.

Policy & SafetyOfficialarXiv Cryptography and Security

MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents

A new preprint introduces MemPoison, a benchmark and analysis framework for evaluating persistent memory attacks on LLM agents. The study covers 1,227 hand-validated attack cases across four attack types and three injection channels, revealing that while write-time defenses can suppress direct attacks, they are ineffective against more complex compositional and context-triggered attacks. The authors propose a taxonomy of attack types and demonstrate structural blind spots in current defense mechanisms, advocating for adaptive, context-sensitive memory defense strategies.

Why it matters: This work exposes critical and previously underappreciated security vulnerabilities in LLM agents with persistent memory, indicating that current defenses are inadequate against sophisticated multi-step attacks.