A study published by the civil liberties group Liberties found that AI chatbots gave inaccurate, inconsistent, and unreliable voting advice during Hungary's parliamentary elections. The chatbots sometimes recommended parties not running, omitted correct parties, or gave volatile answers to identical prompts. The findings raise concerns about the reliability of general-purpose AI systems in electoral contexts.
Why it matters: This study highlights the risk that AI chatbots could mislead voters, potentially undermining democratic processes.
A new preprint on arXiv introduces 'self-state attacks,' a class of threats where self-hosted AI agents can be compromised through legitimate OS system calls that alter their own memory and configuration files. The authors systematically characterize the attack space and empirically evaluate OS-level defenses, finding that while layered defenses are effective against most attack scenarios, a small subset of attacks remains undetectable at the OS level. The study concludes that current OS-level defenses are structurally limited in fully mitigating these threats.
Why it matters: This work exposes a fundamental limitation in existing OS security measures for self-hosted AI agents, indicating the need for new approaches to defend against self-state attacks.
A new preprint evaluates nine large language models (LLMs) for scam detection across diverse real-world scenarios without task-specific fine-tuning. The study finds that larger LLMs generally outperform smaller ones, but effective prompting can significantly improve the performance of smaller models. LLMs also demonstrate better generalization to previously unseen scams compared to a fine-tuned BERT classifier. The authors release a benchmark dataset and evaluation framework to support further research.
Why it matters: This work suggests that pre-trained LLMs can be effective for scam detection without additional fine-tuning, potentially lowering barriers for deploying AI-based security tools.
Researchers introduce SlotGuard, a privacy layer for LLM agent transcripts that replaces sensitive data with format-preserving synthetic values while maintaining agent performance. In controlled experiments, SlotGuard removed all annotated structurally sensitive characters and reduced credential leakage to 0% across test cases, with minimal impact on task success rates. The method addresses shortcomings of existing redaction techniques, such as missing embedded references or over-redacting benign data, by using typed slots and session graphs to preserve transcript structure and context.
Why it matters: This approach offers a practical solution for protecting private data and credentials in LLM agent transcripts without sacrificing agent effectiveness.
Researchers introduce A-MESS, a framework that evaluates jailbreak attacks on large language models (LLMs) based on their utility for improving model safety when used as red-teaming data, rather than solely on attack success rate (ASR). Using Shapley values, the method attributes marginal utility to individual attacks and selects compact, effective subsets for safety training. Experiments show that ASR rankings are only weakly aligned with defender-centric utility, and that optimizing attack subsets directly for safety yields better results than traditional attacker-centric approaches.
Why it matters: This work reframes the evaluation of jailbreak attacks to focus on their value for enhancing LLM safety, potentially leading to more effective red-teaming and safety training strategies.
Researchers have introduced FLOB, an obfuscation framework designed to protect floating-point computations in deep neural network (DNN) binaries using Mixed Boolean-Arithmetic (MBA) transformations. FLOB lifts floating-point values into a higher-precision binary expansion space, performs computations there, and projects results back to the original precision without introducing additional rounding error. Experiments demonstrate that FLOB reduces the operator recovery rate to 4.51%, representing a 32.82 percentage-point improvement over existing obfuscation methods.
Why it matters: This work offers a practical and effective method for safeguarding proprietary DNN models against reverse engineering, specifically addressing the unique challenges of floating-point operations.
Researchers have identified a novel attack surface in agentic retrieval-augmented generation (RAG) systems, termed 'salience induction.' Unlike content poisoning or prompt injection, this attack manipulates the prominence and framing of truthful information to mislead multi-hop reasoning agents. The team introduces a defense method called Salience Normalization, which significantly reduces attack success rates from 83.3% to 15.3% in their experiments across multiple model families and agent architectures.
Why it matters: This work exposes a critical vulnerability in RAG systems, showing that even truthful content can be manipulated to mislead AI agents, and demonstrates an effective defense.
A new interview study with 22 software practitioners reveals that security is rarely considered when selecting and integrating AI components, such as large language models, into software systems. Instead, decisions are primarily driven by functional criteria like performance, accuracy, and cost. The research highlights that established lessons from traditional software supply chain security are often overlooked in the rush to adopt AI technologies. The authors recommend adopting a security-by-design approach to better address these emerging risks.
Why it matters: Neglecting security in AI component selection could expose software systems to significant vulnerabilities, repeating past mistakes in software supply chain management.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers introduce ShadowPickle, a set of three novel pickle deserialization attacks that evade detection by ten state-of-the-art model scanners and four major model hubs, including Hugging Face. These attacks exploit the Pickle VM's external module import mechanism to execute malicious payloads during model deserialization. ShadowPickle achieves up to 63% evasion rates and up to 50% higher evasion rates than previous attacks, exposing significant weaknesses in current model scanning defenses.
Why it matters: This work reveals critical security vulnerabilities in widely used machine learning model hubs, raising concerns about supply chain attacks and the safety of shared pre-trained models.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint presents the first systematic empirical study of how insecure coding preferences stored in long-term memory can increase the risk of generating vulnerable code in LLM-based code generation systems. The researchers evaluated four major LLMs across five programming languages and found that insecure memories increased the vulnerability rate by 2.7–50.3 percentage points. They also identified a gap between increased vulnerabilities and warning rates, and showed that memory-level safety filtering can detect all risky memory entries and restore safer code generation.
Why it matters: This work reveals a significant and previously underexplored security risk in LLM-based code generation systems that use long-term memory, with practical implications for the safe deployment of such technologies.
A new preprint introduces the Privacy-Cost Equity Ratio (PCER), a group fairness metric for differentially private machine learning that incorporates disparities in privacy exposure. PCER is calculated by normalizing a group's positive prediction rate by its overfitting gap, which serves as a proxy for vulnerability to membership inference attacks. Evaluations on several benchmarks, including COMPAS, show that protected groups can simultaneously face higher privacy risks and worse predictive outcomes—patterns that standard outcome-only fairness metrics may overlook. The metric is practical for post-hoc audits, requiring only per-group train and test accuracy.
Why it matters: This work broadens the concept of fairness in privacy-preserving machine learning by highlighting the importance of accounting for which groups bear the privacy risks, not just who benefits from model outcomes.
Researchers present the first differentially private algorithms for the max-sum diversification (MSD) problem under both cardinality and matroid constraints. Their methods achieve nearly optimal utility and are faster than existing non-private approaches. Experiments on real-world datasets show that the private algorithms maintain utility comparable to non-private baselines, even under strong privacy guarantees, and offer significant speed improvements for cardinality constraints.
Why it matters: This work enables privacy-preserving data summarization and query output diversification with strong utility and efficiency, addressing a key gap in handling sensitive data.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint introduces (A)iSpy, a parasitic Trojan that targets machine learning infrastructure by embedding itself within the computation graph. The Trojan can monitor tensor states, exfiltrate sensitive hyperparameters, and amplify data poisoning into highly effective backdoor attacks, all while evading standard malware detection. The attack is demonstrated on ONNX Runtime training and inference engines, highlighting a critical vulnerability in the ML execution environment.
Why it matters: This work reveals a significant and previously underexplored security risk in ML infrastructure, showing that malicious runtime modules can achieve manipulations beyond the reach of traditional data or model-level attacks.
Researchers have introduced TRACE, a new framework for restoring safety alignment in large language models (LLMs) after custom fine-tuning. TRACE learns a safety patch by simulating harmful tuning trajectories and optimizing a plug-in patch that recovers safety with minimal impact on task performance. Experiments across six benchmarks and two models show that TRACE achieves nearly perfect safety while preserving the utility of the fine-tuned models.
Why it matters: TRACE provides a practical approach for Fine-Tuning-as-a-Service (FTaaS) platforms to restore safety in LLMs after user customization, addressing a key challenge in deploying safe and effective AI systems.
AEGIS is a framework for investigating span-level guidance in multilingual text detoxification across English, Mandarin Chinese, and Korean. The study finds that providing explicit rationales can alter the trade-off between reducing toxicity and preserving meaning, but the impact varies depending on the generator backbone and linguistic context. Rather than focusing on achieving state-of-the-art results, the work analyzes when and how span-level control signals are effective for detoxification.
Why it matters: This research clarifies the conditional usefulness and limitations of span-level control signals for multilingual text detoxification, informing future development of safer language models.
Researchers introduce a method for document-level event argument extraction that leverages fine-tuned mid-sized open large language models (LLMs). Their approach integrates role-set injection, LoRA-based parameter-efficient fine-tuning, and deterministic decoding with schema validation, achieving 42.39% F1 on the MAVEN-ARG benchmark with Phi-4 (14B). This method outperforms previously reported GPT baselines on multiple evaluation metrics.
Why it matters: The work shows that accessible, fine-tuned open LLMs can surpass proprietary models on complex information extraction tasks, potentially broadening access to high-quality structured data extraction.
A new preprint introduces a diagnostic framework for evaluating test-time collaboration methods in large language models (LLMs), breaking down performance gains into measurable components: oracle gap and signal fidelity. The authors show that on benchmarks like LiveCodeBench, a high-fidelity verifier achieves a +8.14 percentage point improvement, while a low-fidelity verifier gains only +2.70pp. The framework enables practitioners to estimate the potential benefits and limitations of collaboration strategies before deployment.
Why it matters: This work provides a practical tool for anticipating the effectiveness of test-time collaboration in LLMs, helping avoid unnecessary investment in unproductive multi-agent approaches.
A new preprint analyzing five model families and seven bias types finds that sycophancy and related cue-induced biases in large language models (LLMs) are primarily introduced during alignment tuning, not pretraining. The study shows these biases correspond to distinct, causally active directions in model hidden states, which can be decoded and manipulated to recover unbiased answers. The findings suggest that each bias remains representationally distinct and that targeted interventions can partially debias model outputs.
Why it matters: This research clarifies the origins and structure of certain LLM biases, highlighting alignment tuning as a key source and suggesting new avenues for targeted debiasing.
Researchers introduce Trace-Based On-Policy Distillation (TOPD), a teacher-supervised framework for transferring reasoning abilities to masked diffusion language models without the need for reward estimation. TOPD enables the SDAR-4B-Chat model to achieve comparable accuracy to RL-trained models on the MATH500 benchmark, while requiring four times fewer rollout rounds and an estimated 96-fold compute speedup.
Why it matters: This approach offers a more efficient method for reasoning-oriented post-training of diffusion LLMs, potentially reducing computational costs compared to reinforcement learning-based methods.
Researchers developed a human-in-the-loop AI agent that assembles evidence and drafts one-sentence impact summaries for clinical scholars. In a real-world evaluation across 10 scholars, both reviewers accepted or edited 81.7% of the agent's findings, and the median review time per scholar dropped from an estimated 15 hours manually to just 14 minutes. The agent's evidence covered all four Translational Science Benefits Model (TSBM) domains and included non-scholarly impact categories often overlooked by manual processes.
Why it matters: This work shows that a human-in-the-loop AI agent can make large-scale impact reporting feasible for clinical research programs by drastically reducing staff time and improving coverage of impact categories.