A study published by the civil liberties group Liberties found that AI chatbots gave inaccurate, inconsistent, and unreliable voting advice during Hungary's parliamentary elections. The chatbots sometimes recommended parties not running, omitted correct parties, or gave volatile answers to identical prompts. The findings raise concerns about the reliability of general-purpose AI systems in electoral contexts.
Why it matters: This study highlights the risk that AI chatbots could mislead voters, potentially undermining democratic processes.
A new preprint on arXiv introduces 'self-state attacks,' a class of threats where self-hosted AI agents can be compromised through legitimate OS system calls that alter their own memory and configuration files. The authors systematically characterize the attack space and empirically evaluate OS-level defenses, finding that while layered defenses are effective against most attack scenarios, a small subset of attacks remains undetectable at the OS level. The study concludes that current OS-level defenses are structurally limited in fully mitigating these threats.
Why it matters: This work exposes a fundamental limitation in existing OS security measures for self-hosted AI agents, indicating the need for new approaches to defend against self-state attacks.
A new preprint evaluates nine large language models (LLMs) for scam detection across diverse real-world scenarios without task-specific fine-tuning. The study finds that larger LLMs generally outperform smaller ones, but effective prompting can significantly improve the performance of smaller models. LLMs also demonstrate better generalization to previously unseen scams compared to a fine-tuned BERT classifier. The authors release a benchmark dataset and evaluation framework to support further research.
Why it matters: This work suggests that pre-trained LLMs can be effective for scam detection without additional fine-tuning, potentially lowering barriers for deploying AI-based security tools.
Researchers introduce SlotGuard, a privacy layer for LLM agent transcripts that replaces sensitive data with format-preserving synthetic values while maintaining agent performance. In controlled experiments, SlotGuard removed all annotated structurally sensitive characters and reduced credential leakage to 0% across test cases, with minimal impact on task success rates. The method addresses shortcomings of existing redaction techniques, such as missing embedded references or over-redacting benign data, by using typed slots and session graphs to preserve transcript structure and context.
Why it matters: This approach offers a practical solution for protecting private data and credentials in LLM agent transcripts without sacrificing agent effectiveness.
A new preprint introduces RRAM-DP, a hardware-algorithm co-design that utilizes the inherent stochastic write behavior of resistive-switching random-access memory (RRAM) devices to inject calibrated noise for differential privacy in edge AIoT systems. The approach achieves at most a 3.8% accuracy drop at (ε=2, δ=O(1/n))-DP on benchmarks such as CIFAR-10/100, and demonstrates up to 57x energy savings and 2.7x speedups compared to GPU baselines. This method represents a novel use of device-level randomness for privacy-preserving, efficient in-memory training.
Why it matters: This work offers a significant advance in privacy-preserving machine learning for edge devices by leveraging physical properties of emerging memory technology, potentially enabling more secure and efficient AIoT applications.
Policy & Safety→Official→arXiv Cryptography and Security
A systematic study of cross-modal unlearning in vision-language models (LLaVA-1.5, InstructBLIP, IDEFICS) finds that unlearning knowledge in one modality (text or vision) transfers asymmetrically and incompletely to the other. The research shows that typographic attacks—manipulating the visual presentation of text—can recover previously unlearned knowledge, revealing that current unlearning methods are shallow. The proposed CrossInf mitigation strategy reduces the transfer gap by more than half and lowers attack success rates to near zero, while preserving model utility.
Why it matters: This work reveals a critical vulnerability in current unlearning methods for multimodal AI, showing that knowledge can be recovered via cross-modal attacks, and introduces a practical mitigation to improve the safety and reliability of vision-language models.
Researchers introduce A-MESS, a framework that evaluates jailbreak attacks on large language models (LLMs) based on their utility for improving model safety when used as red-teaming data, rather than solely on attack success rate (ASR). Using Shapley values, the method attributes marginal utility to individual attacks and selects compact, effective subsets for safety training. Experiments show that ASR rankings are only weakly aligned with defender-centric utility, and that optimizing attack subsets directly for safety yields better results than traditional attacker-centric approaches.
Why it matters: This work reframes the evaluation of jailbreak attacks to focus on their value for enhancing LLM safety, potentially leading to more effective red-teaming and safety training strategies.
Researchers have introduced FLOB, an obfuscation framework designed to protect floating-point computations in deep neural network (DNN) binaries using Mixed Boolean-Arithmetic (MBA) transformations. FLOB lifts floating-point values into a higher-precision binary expansion space, performs computations there, and projects results back to the original precision without introducing additional rounding error. Experiments demonstrate that FLOB reduces the operator recovery rate to 4.51%, representing a 32.82 percentage-point improvement over existing obfuscation methods.
Why it matters: This work offers a practical and effective method for safeguarding proprietary DNN models against reverse engineering, specifically addressing the unique challenges of floating-point operations.
Researchers have identified a novel attack surface in agentic retrieval-augmented generation (RAG) systems, termed 'salience induction.' Unlike content poisoning or prompt injection, this attack manipulates the prominence and framing of truthful information to mislead multi-hop reasoning agents. The team introduces a defense method called Salience Normalization, which significantly reduces attack success rates from 83.3% to 15.3% in their experiments across multiple model families and agent architectures.
Why it matters: This work exposes a critical vulnerability in RAG systems, showing that even truthful content can be manipulated to mislead AI agents, and demonstrates an effective defense.
A new interview study with 22 software practitioners reveals that security is rarely considered when selecting and integrating AI components, such as large language models, into software systems. Instead, decisions are primarily driven by functional criteria like performance, accuracy, and cost. The research highlights that established lessons from traditional software supply chain security are often overlooked in the rush to adopt AI technologies. The authors recommend adopting a security-by-design approach to better address these emerging risks.
Why it matters: Neglecting security in AI component selection could expose software systems to significant vulnerabilities, repeating past mistakes in software supply chain management.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers introduce ShadowPickle, a set of three novel pickle deserialization attacks that evade detection by ten state-of-the-art model scanners and four major model hubs, including Hugging Face. These attacks exploit the Pickle VM's external module import mechanism to execute malicious payloads during model deserialization. ShadowPickle achieves up to 63% evasion rates and up to 50% higher evasion rates than previous attacks, exposing significant weaknesses in current model scanning defenses.
Why it matters: This work reveals critical security vulnerabilities in widely used machine learning model hubs, raising concerns about supply chain attacks and the safety of shared pre-trained models.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint presents the first systematic empirical study of how insecure coding preferences stored in long-term memory can increase the risk of generating vulnerable code in LLM-based code generation systems. The researchers evaluated four major LLMs across five programming languages and found that insecure memories increased the vulnerability rate by 2.7–50.3 percentage points. They also identified a gap between increased vulnerabilities and warning rates, and showed that memory-level safety filtering can detect all risky memory entries and restore safer code generation.
Why it matters: This work reveals a significant and previously underexplored security risk in LLM-based code generation systems that use long-term memory, with practical implications for the safe deployment of such technologies.
A new preprint introduces the Privacy-Cost Equity Ratio (PCER), a group fairness metric for differentially private machine learning that incorporates disparities in privacy exposure. PCER is calculated by normalizing a group's positive prediction rate by its overfitting gap, which serves as a proxy for vulnerability to membership inference attacks. Evaluations on several benchmarks, including COMPAS, show that protected groups can simultaneously face higher privacy risks and worse predictive outcomes—patterns that standard outcome-only fairness metrics may overlook. The metric is practical for post-hoc audits, requiring only per-group train and test accuracy.
Why it matters: This work broadens the concept of fairness in privacy-preserving machine learning by highlighting the importance of accounting for which groups bear the privacy risks, not just who benefits from model outcomes.
Researchers present the first differentially private algorithms for the max-sum diversification (MSD) problem under both cardinality and matroid constraints. Their methods achieve nearly optimal utility and are faster than existing non-private approaches. Experiments on real-world datasets show that the private algorithms maintain utility comparable to non-private baselines, even under strong privacy guarantees, and offer significant speed improvements for cardinality constraints.
Why it matters: This work enables privacy-preserving data summarization and query output diversification with strong utility and efficiency, addressing a key gap in handling sensitive data.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint introduces (A)iSpy, a parasitic Trojan that targets machine learning infrastructure by embedding itself within the computation graph. The Trojan can monitor tensor states, exfiltrate sensitive hyperparameters, and amplify data poisoning into highly effective backdoor attacks, all while evading standard malware detection. The attack is demonstrated on ONNX Runtime training and inference engines, highlighting a critical vulnerability in the ML execution environment.
Why it matters: This work reveals a significant and previously underexplored security risk in ML infrastructure, showing that malicious runtime modules can achieve manipulations beyond the reach of traditional data or model-level attacks.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers have introduced BSB, a structured jailbreak framework targeting text-to-video (T2V) models by exploiting temporal consistency—encoding harmful intent as transitions between otherwise harmless boundary states. Using Monte Carlo Tree Search in a textual proxy space, BSB achieves an average 18.6% relative gain in attack success rate over existing methods on models such as Veo 3.1, Sora 2, Seedance, and Kling v1. The study demonstrates that temporal consistency is a critical and previously underexplored vulnerability in T2V systems.
Why it matters: This work reveals a significant new attack surface in T2V models, raising important safety concerns as video generation technologies are increasingly adopted.
A new preprint introduces RT-SHCUA, an architecture designed to enable real-time, secure use of self-hosted computer-use agents (SHCUAs) for controlling unmanned aerial vehicles (UAVs). The system addresses the mismatch between language agent reasoning and the strict timing, safety, and security requirements of UAV operation by separating semantic reasoning from onboard execution. It enforces contract-bound skill invocations with explicit checks for timing, state, and authority, and supports degraded handling and auditable evidence preservation. Prototype evaluation demonstrates bounded responsiveness and trusted enforcement without embedding the full agent in critical control loops.
Why it matters: This work proposes a novel architecture to safely and securely bridge language agents with real-time UAV control, addressing key challenges in timing, authorization, and accountability.
Researchers have introduced TRACE, a new framework for restoring safety alignment in large language models (LLMs) after custom fine-tuning. TRACE learns a safety patch by simulating harmful tuning trajectories and optimizing a plug-in patch that recovers safety with minimal impact on task performance. Experiments across six benchmarks and two models show that TRACE achieves nearly perfect safety while preserving the utility of the fine-tuned models.
Why it matters: TRACE provides a practical approach for Fine-Tuning-as-a-Service (FTaaS) platforms to restore safety in LLMs after user customization, addressing a key challenge in deploying safe and effective AI systems.
AEGIS is a framework for investigating span-level guidance in multilingual text detoxification across English, Mandarin Chinese, and Korean. The study finds that providing explicit rationales can alter the trade-off between reducing toxicity and preserving meaning, but the impact varies depending on the generator backbone and linguistic context. Rather than focusing on achieving state-of-the-art results, the work analyzes when and how span-level control signals are effective for detoxification.
Why it matters: This research clarifies the conditional usefulness and limitations of span-level control signals for multilingual text detoxification, informing future development of safer language models.
Researchers introduce a method for document-level event argument extraction that leverages fine-tuned mid-sized open large language models (LLMs). Their approach integrates role-set injection, LoRA-based parameter-efficient fine-tuning, and deterministic decoding with schema validation, achieving 42.39% F1 on the MAVEN-ARG benchmark with Phi-4 (14B). This method outperforms previously reported GPT baselines on multiple evaluation metrics.
Why it matters: The work shows that accessible, fine-tuned open LLMs can surpass proprietary models on complex information extraction tasks, potentially broadening access to high-quality structured data extraction.