A new arXiv preprint introduces the PhantomFill benchmark, showing that large language models (LLMs) frequently fabricate information when required to fill structured fields, such as in JSON forms, even when they would otherwise admit to having no data in free text. Across 13 models, the study found that required fields led to coerced fabrication nearly every time, while free-text responses were mostly honest. The benchmark quantifies this phenomenon and highlights that current evaluation practices may overlook this critical failure mode.
Why it matters: This finding exposes a significant reliability risk for LLMs in real-world applications that depend on structured outputs, suggesting that required fields can silently induce systematic hallucination.
A new arXiv preprint reports that quantizing large language models—a common step to make them more efficient—can introduce measurable bias in open-ended text generation, even when standard safety checks show no change. The study finds that quantized models are more likely to produce stereotyped responses across eight languages, a phenomenon not detected by typical refusal rate or multiple-choice fairness metrics. The authors introduce QuantiBias, a benchmark designed to reveal this hidden bias.
Why it matters: This work highlights a previously overlooked risk in LLM deployment, showing that quantization can silently increase bias without triggering standard safety evaluations.
A new arXiv preprint demonstrates that large language models (LLMs), when paired with prediction market data, can quantify how information sources bias strategic predictions. Analyzing 111 Ukraine-related prediction markets, the study finds that English news context systematically biases territorial forecasts, with predictions favoring territorial capture being incorrect 64–72% of the time. The analysis suggests that this bias originates mainly from the text sources rather than the LLMs themselves, and persists across multiple model architectures.
Why it matters: The findings highlight that AI systems relying on real-world text sources can inherit and propagate significant biases, which could impact strategic decision-making in high-stakes domains.
A new arXiv preprint examines political bias in large language models (LLMs) using both abstract policy questionnaires and real-world Swiss referenda. The study finds that while LLMs appear left-leaning on surveys, their responses to actual referenda are more centrist and sometimes show a general aversion to change rather than a clear left-right bias. The language in which questions are posed can also significantly affect model responses.
Why it matters: This suggests that previous claims of consistent leftward bias in LLMs may not generalize to real-world decision contexts, raising questions about how AI political alignment should be measured and interpreted.
A new arXiv preprint introduces the Frontier Financial Judgement benchmark, developed with professional equity analysts, to test AI agents' ability to identify news that could impact stock valuations. The best-performing AI agent matched expert human labels just 52.4% of the time, and false-positive rates varied widely across models. The benchmark consists of 656 items, including both synthetic and real news articles, designed to reflect real-world financial information challenges.
Why it matters: The results highlight a significant gap in current AI capabilities for reliably filtering valuation-relevant financial news, which is critical for safe and effective deployment in equity analysis.
A new arXiv preprint presents a controlled evaluation of how different evidence interfaces affect the validity of large language model (LLM)-generated policy audit reports. Using 60 AGORA policy cases, the study finds that reports can appear substantively plausible even when citing irrelevant internal evidence, especially in a shuffled control condition. The hybrid evidence interface was rated most useful by human reviewers, and the findings suggest that simply providing internal model access does not guarantee meaningful transparency.
Why it matters: The work highlights a significant governance risk: AI-generated audit reports may sound convincing while relying on irrelevant or misleading evidence, emphasizing the need for careful evidence design in AI oversight.
Policy & Safety→Official→arXiv Cryptography and Security
A new arXiv preprint introduces PISmith, a reinforcement learning-based framework designed to systematically test prompt injection defenses in large language models (LLMs). By training an attack model to optimize injected prompts in a black-box setting, the method reveals that state-of-the-art defenses remain vulnerable to adaptive attacks, achieving high success rates across 13 benchmarks and in agentic scenarios against both open-source and closed-source models. The study highlights the ongoing challenge of securing LLMs against evolving prompt injection strategies.
Why it matters: The work underscores a significant security gap in current LLM defenses, suggesting that widely used protections may not be sufficient against adaptive adversaries.
A systematic human evaluation of ten leading large language models (LLMs) across 62 multidomain questions found that all models exhibit response drift—deviations from expert-validated references. Most models showed high rates of drift (78-81%), while two had notably lower rates (47-49%). The extent and pattern of drift varied by domain and question, and automated metrics accounted for less than 2% of the variance in human judgments.
Why it matters: The findings highlight a widespread and domain-dependent limitation in current LLMs that cannot be reliably detected by automated metrics alone, underscoring the need for human evaluation in assessing model reliability.
A new arXiv preprint finds that large language models (LLMs) often change their answers' correctness when mathematically equivalent problems are presented in different surface forms, such as story, symbolic, or equation formats. Introducing code-based reasoning, where models generate and execute Python code, improves performance for some LLMs but does not fully resolve this sensitivity; errors instead shift to issues like protocol violations or code execution failures. The study highlights that even subtle changes in problem wording can significantly affect LLM reliability.
Why it matters: This work suggests that the way problems are presented remains a critical factor in LLM reliability for mathematical tasks, with implications for evaluation and deployment in real-world applications.
A new arXiv preprint introduces Rushes, a dataset of over 44,000 decision events from interactive narrative games, capturing how thousands of users make sequential choices. The study finds that leading large language models, including GPT-5, do not outperform simple baselines in predicting individual user choices, highlighting a persistent 'Engagement Gap.' This suggests that current alignment methods, which optimize for population-level preferences, may be inadequate for capturing diverse, context-dependent user behaviors.
Why it matters: The findings raise questions about the ability of current AI alignment techniques to personalize responses and adapt to individual users, a key challenge for future AI systems.
A new framework for generating synthetic lung CT slices combines Optimal Transport Conditional Flow Matching with geometric latent space filtering to enhance privacy. While the method reduces direct image memorization and some privacy risks, official challenge results show that deeper anatomical identity can still be inferred from generated images. This suggests that current filtering techniques are insufficient to fully protect patient identity in medical image synthesis.
Why it matters: The findings underscore a significant limitation in privacy-preserving medical image generation, with implications for data sharing and regulatory compliance.
A new arXiv preprint finds that, in production-like settings, adding LLM alignment to a regex filter does not increase the block rate for natural-language harmful requests—coverage remains at zero across tested categories. However, when adversarially-framed probes are used, an LLM-based judge can detect refusals that simple substring matching misses, indicating that alignment's effectiveness depends on the evaluation metric. The study uses a locked corpus and multiple probe types to isolate these effects.
Why it matters: The findings question the added value of LLM alignment for safety in scenarios where regex filters are already deployed, potentially impacting how LLM safety pipelines are designed.
A new arXiv preprint audits five commonly used diversity metrics for selecting large language models (LLMs) in ensemble majority voting. The study finds that these metrics are largely entangled with model capability, rather than measuring true diversity. After controlling for capability, only a modest residual link remains between shared errors and ensemble voting gains, challenging the assumption that diversity metrics reliably predict ensemble improvement.
Why it matters: This result questions the reliability of widely used diversity metrics in LLM ensemble construction, potentially impacting how practitioners combine models for better performance.
Policy & Safety→Official→arXiv Cryptography and Security
A new arXiv preprint introduces IssueTrojanBench, a benchmark designed to test AI coding agents against malicious issue requests. The study finds that 66.5% of these adversarial issues bypass all current guardrails in leading coding agents, with most rejections coming from the underlying language models rather than the agent frameworks. The results suggest that existing agent-level defenses provide little additional protection beyond what the LLMs themselves offer.
Why it matters: This work exposes significant, broadly relevant security vulnerabilities in widely-used AI coding agents, underscoring the need for improved safety mechanisms as these tools are increasingly adopted in software development.
A new arXiv preprint introduces Merge-Adversarial Training, a method designed to make watermarks in open-source large language models (LLMs) more robust to removal by model merging—a common post-training modification. The approach reportedly increases watermark detection rates by up to 51 percentage points at a 1% false positive rate, without degrading model performance. The study also evaluates watermark durability across several realistic model merging scenarios.
Why it matters: This work addresses a key challenge in tracing the origins of text generated by open-source LLMs, even after models are merged or modified post-release.
Policy & Safety→Official→arXiv Cryptography and Security
A new arXiv preprint examines eight X.509 certificate validation stacks and finds that, under hybrid-required policies, most accept certificates based solely on classical cryptography, without requiring post-quantum evidence to influence the authentication outcome. The study introduces a verifier model and reference contract to clarify this gap and analyzes why current standards do not mandate binding post-quantum credentials to authentication success. The findings highlight a structural issue in the transition to post-quantum security for widely used certificate infrastructures.
Why it matters: This work exposes a significant security risk in post-quantum migration, where systems may appear to offer hybrid authentication but actually rely only on classical cryptography, potentially undermining future cryptographic resilience.
A new arXiv preprint reports that while human-written natural-language policies can significantly improve the performance of frozen large language model (LLM) agents in the TextWorldExpress environment, the TextGrad method—which attempts to learn such policies from agent experience using natural-language feedback—does not yield better results than fixed prompting. The study finds that the main obstacle is not in executing policy updates, but in reliably generating and selecting effective policies from agent trajectories, even when using richer feedback or advanced search techniques.
Why it matters: This highlights a key limitation in current approaches to learning agent policies from experience using natural-language feedback, suggesting that simply leveraging agent trajectories is insufficient for policy optimization in LLM agents.
A new arXiv preprint finds that large language models (LLMs) are unreliable at distinguishing their own generated content from human-written responses in short-answer educational tasks. While LLMs perform well at detecting AI-generated code and longer reflective writing, they often misclassify their own short answers as more human-like than authentic student work. The study also shows that prompt variations can significantly affect detection accuracy in reflective writing, but have less impact on programming tasks.
Why it matters: This highlights a critical limitation for using LLMs as automated detectors of AI-generated student work, especially for short-answer formats that are common in education.
Policy & Safety→Official→arXiv Cryptography and Security
A new arXiv preprint reports that code generated by ChatGPT, Microsoft Copilot, and Google Gemini for routine automation tasks consistently contained exploitable security vulnerabilities. The study found that 9 out of 17 vulnerability classes appeared in code from all three models, and overall risk scores were similar across platforms, suggesting the vulnerabilities are linked to the nature of the tasks rather than any specific model.
Why it matters: This highlights a broad security risk in deploying LLM-generated automation code without human review, regardless of the AI tool used.
A new arXiv preprint introduces Instruct-FD, a benchmark designed to test whether full-duplex spoken dialogue systems can reliably follow explicit turn-taking instructions. When evaluated on six leading systems, the highest adherence rate was only 64.4%, with proactive conversational behaviors such as backchanneling and interruption posing particular difficulties. The results suggest that current systems have significant limitations in controllable turn management.
Why it matters: This highlights a key challenge for deploying adaptable full-duplex speech systems in real-world applications where flexible turn-taking is required.