A new interview study with 22 software practitioners reveals that security is rarely considered when selecting and integrating AI components, such as large language models, into software systems. Instead, decisions are primarily driven by functional criteria like performance, accuracy, and cost. The research highlights that established lessons from traditional software supply chain security are often overlooked in the rush to adopt AI technologies. The authors recommend adopting a security-by-design approach to better address these emerging risks.
Why it matters: Neglecting security in AI component selection could expose software systems to significant vulnerabilities, repeating past mistakes in software supply chain management.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers introduce ShadowPickle, a set of three novel pickle deserialization attacks that evade detection by ten state-of-the-art model scanners and four major model hubs, including Hugging Face. These attacks exploit the Pickle VM's external module import mechanism to execute malicious payloads during model deserialization. ShadowPickle achieves up to 63% evasion rates and up to 50% higher evasion rates than previous attacks, exposing significant weaknesses in current model scanning defenses.
Why it matters: This work reveals critical security vulnerabilities in widely used machine learning model hubs, raising concerns about supply chain attacks and the safety of shared pre-trained models.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint presents the first systematic empirical study of how insecure coding preferences stored in long-term memory can increase the risk of generating vulnerable code in LLM-based code generation systems. The researchers evaluated four major LLMs across five programming languages and found that insecure memories increased the vulnerability rate by 2.7–50.3 percentage points. They also identified a gap between increased vulnerabilities and warning rates, and showed that memory-level safety filtering can detect all risky memory entries and restore safer code generation.
Why it matters: This work reveals a significant and previously underexplored security risk in LLM-based code generation systems that use long-term memory, with practical implications for the safe deployment of such technologies.
A new preprint introduces the Privacy-Cost Equity Ratio (PCER), a group fairness metric for differentially private machine learning that incorporates disparities in privacy exposure. PCER is calculated by normalizing a group's positive prediction rate by its overfitting gap, which serves as a proxy for vulnerability to membership inference attacks. Evaluations on several benchmarks, including COMPAS, show that protected groups can simultaneously face higher privacy risks and worse predictive outcomes—patterns that standard outcome-only fairness metrics may overlook. The metric is practical for post-hoc audits, requiring only per-group train and test accuracy.
Why it matters: This work broadens the concept of fairness in privacy-preserving machine learning by highlighting the importance of accounting for which groups bear the privacy risks, not just who benefits from model outcomes.
Researchers present the first differentially private algorithms for the max-sum diversification (MSD) problem under both cardinality and matroid constraints. Their methods achieve nearly optimal utility and are faster than existing non-private approaches. Experiments on real-world datasets show that the private algorithms maintain utility comparable to non-private baselines, even under strong privacy guarantees, and offer significant speed improvements for cardinality constraints.
Why it matters: This work enables privacy-preserving data summarization and query output diversification with strong utility and efficiency, addressing a key gap in handling sensitive data.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint introduces (A)iSpy, a parasitic Trojan that targets machine learning infrastructure by embedding itself within the computation graph. The Trojan can monitor tensor states, exfiltrate sensitive hyperparameters, and amplify data poisoning into highly effective backdoor attacks, all while evading standard malware detection. The attack is demonstrated on ONNX Runtime training and inference engines, highlighting a critical vulnerability in the ML execution environment.
Why it matters: This work reveals a significant and previously underexplored security risk in ML infrastructure, showing that malicious runtime modules can achieve manipulations beyond the reach of traditional data or model-level attacks.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers have introduced BSB, a structured jailbreak framework targeting text-to-video (T2V) models by exploiting temporal consistency—encoding harmful intent as transitions between otherwise harmless boundary states. Using Monte Carlo Tree Search in a textual proxy space, BSB achieves an average 18.6% relative gain in attack success rate over existing methods on models such as Veo 3.1, Sora 2, Seedance, and Kling v1. The study demonstrates that temporal consistency is a critical and previously underexplored vulnerability in T2V systems.
Why it matters: This work reveals a significant new attack surface in T2V models, raising important safety concerns as video generation technologies are increasingly adopted.
A new preprint introduces RT-SHCUA, an architecture designed to enable real-time, secure use of self-hosted computer-use agents (SHCUAs) for controlling unmanned aerial vehicles (UAVs). The system addresses the mismatch between language agent reasoning and the strict timing, safety, and security requirements of UAV operation by separating semantic reasoning from onboard execution. It enforces contract-bound skill invocations with explicit checks for timing, state, and authority, and supports degraded handling and auditable evidence preservation. Prototype evaluation demonstrates bounded responsiveness and trusted enforcement without embedding the full agent in critical control loops.
Why it matters: This work proposes a novel architecture to safely and securely bridge language agents with real-time UAV control, addressing key challenges in timing, authorization, and accountability.
Researchers have introduced TRACE, a new framework for restoring safety alignment in large language models (LLMs) after custom fine-tuning. TRACE learns a safety patch by simulating harmful tuning trajectories and optimizing a plug-in patch that recovers safety with minimal impact on task performance. Experiments across six benchmarks and two models show that TRACE achieves nearly perfect safety while preserving the utility of the fine-tuned models.
Why it matters: TRACE provides a practical approach for Fine-Tuning-as-a-Service (FTaaS) platforms to restore safety in LLMs after user customization, addressing a key challenge in deploying safe and effective AI systems.
A systematic evaluation finds that off-the-shelf large language models (LLMs), specifically GPT-5.2 and Gemini-2.5, can match or exceed the performance of six specialized privacy policy analysis tools across tasks such as contradiction detection, regulatory compliance analysis, and summarization. In manual labeling tasks, LLMs achieved precision up to 91.4% and recall up to 70.9%, demonstrating their ability to perform tool-specific functionalities without additional domain-specific training.
Why it matters: This study suggests that general-purpose LLMs could serve as effective alternatives to specialized privacy analysis tools, potentially reducing costs and simplifying compliance workflows.
A new preprint introduces the Signal-based Model Access Risk Taxonomy (SMART), a framework for classifying attacker access to deployed AI systems based on the specific information signals (such as decisions, confidence scores, or intermediate representations) they can obtain. Unlike traditional white-box/gray-box/black-box distinctions, SMART focuses on the practical deployment context and how different output signals enable distinct evasion attack strategies. The taxonomy aims to help organizations better assess and mitigate risks when deploying or procuring AI systems.
Why it matters: This framework offers a more nuanced and practical approach to evaluating and managing security risks in real-world AI deployments, which is increasingly important as AI systems are integrated into critical sectors.
Microsoft researchers have introduced Adaptive Incident Prioritization (AIP), the ranking algorithm powering Defender Queue Assistant, which continuously prioritizes security incidents for analyst review. AIP, now deployed across tens of thousands of customers, achieves 92.8% Precision@10 in expert-reviewed evaluations and measurably increases analyst engagement metrics. The work also extends the Microsoft GUIDE dataset with what is claimed to be the first public label source for security operations center (SOC) queue prioritization.
Why it matters: This research offers a scalable, adaptive solution to alert fatigue in security operations centers, with demonstrated improvements in incident triage and analyst efficiency.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers have developed Jailbreak Foundry (JBF), a system that automates the translation of jailbreak research papers into executable attack modules for standardized evaluation. JBF uses a multi-agent workflow and shared infrastructure to reproduce 30 jailbreak attacks with high fidelity, achieving a mean attack success rate deviation of just +0.26 percentage points compared to original reports. The system also reduces attack-specific implementation code by more than half and enables consistent benchmarking across multiple language models using a unified evaluation protocol.
Why it matters: JBF provides a scalable and automated solution for reproducible and comparable benchmarking in LLM jailbreak research, addressing a key challenge in evaluating and tracking evolving security threats.
Researchers have introduced a triple-hoisted baby-step giant-step algorithm that further decomposes the baby step to significantly reduce the number of ciphertext rotations required for linear transformations in CKKS homomorphic encryption. They also propose a memory-optimized data path and an FPGA-based hardware accelerator, which together reduce off-chip memory access by 2.9x and computational latency by 5.8x compared to previous designs.
Why it matters: This work represents a notable advance in making privacy-preserving neural network inference more efficient by addressing key computational and memory bottlenecks in homomorphic encryption.
Researchers introduce three client-side detection techniques—label flipping, backdoor trigger injection, and model fingerprinting—to identify targeted overfitting attacks orchestrated by a dishonest aggregator in federated learning. Their methods can detect such attacks within 1-2 training rounds, achieving F1 scores up to 0.7, and allow clients to disengage early. Experiments indicate that detection effectiveness varies with cohort composition and method parameters.
Why it matters: This work offers practical early-warning tools for clients in federated learning, potentially improving the security and trustworthiness of these systems against orchestrator-driven attacks.
VeriX-Anon is a multi-layered verification framework for outsourced target-driven k-anonymization, combining deterministic, probabilistic, and explainable AI-based mechanisms. In evaluations across seven datasets and four cloud profiles, it detected 25 to 27 out of 28 adversarial deviations with no false alarms, and client-side verification completed in under one second for datasets with one million rows. The framework addresses a gap in verifying that privacy-sensitive data transformations are faithfully executed by cloud providers.
Why it matters: VeriX-Anon offers a practical, mathematically grounded method for data owners to verify the integrity of outsourced privacy-preserving data transformations, addressing a key trust issue in cloud-based data processing.
Researchers introduce memorywire, a JSON-Schema 2020-12 wire format supporting five core memory operations (remember, recall, forget, merge, expire) across four memory types (semantic, episodic, procedural, emotional) for agent-memory frameworks. The open-source reference implementation includes five backend adapters and demonstrates strong performance on a microbenchmark, achieving recall@5=1.000 and low median latencies for ingest and recall. The protocol also features an optional human-in-the-loop governance channel and is designed for compatibility with the Model Context Protocol.
Why it matters: memorywire provides a standardized, vendor-neutral protocol for agent memory operations, facilitating interoperability, migration, and governance across diverse memory frameworks.
A new preprint introduces ADS-C, an antidistillation defense for classification models that provably preserves top-1 accuracy while significantly degrading the performance of student models trained via knowledge distillation. ADS-C adapts gradient-directed perturbation techniques from large language models to classifiers, using a closed-form, per-input margin budget to ensure the teacher model's predictions remain unchanged. The method achieves up to 29.6 percentage point degradation in student accuracy on standard benchmarks, with no loss in teacher accuracy.
Why it matters: ADS-C provides a practical and provable defense against model stealing attacks via knowledge distillation, enabling secure deployment of classifiers without sacrificing their accuracy.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers formalize the concept of payload recoverability in LLM steganography and introduce new low-recoverability schemes using embedding-space mappings. Their experiments show that while these schemes reduce the ease of secret recovery, mechanistic interpretability methods—specifically linear probes on later-layer activations—can still detect steganographic content with up to 33% higher accuracy in fine-tuned models compared to base models.
Why it matters: This work highlights both the evolving threat of covert communication via LLMs and the potential for interpretability-based defenses, signaling a new dynamic in model security.
Researchers have demonstrated that intentional electromagnetic interference (IEMI) using commonly available radio frequency (RF) equipment can disrupt the operation of facial recognition systems. The study evaluates the robustness of state-of-the-art face recognition methods against such hardware-level attacks and introduces a benchmark dataset of facial images captured with and without RF interference.
Why it matters: This work reveals a previously underexplored vulnerability in biometric systems, showing that facial recognition can be compromised by hardware-based attacks, which has significant implications for the security of systems relying on these technologies.