Researchers introduce a method for document-level event argument extraction that leverages fine-tuned mid-sized open large language models (LLMs). Their approach integrates role-set injection, LoRA-based parameter-efficient fine-tuning, and deterministic decoding with schema validation, achieving 42.39% F1 on the MAVEN-ARG benchmark with Phi-4 (14B). This method outperforms previously reported GPT baselines on multiple evaluation metrics.
Why it matters: The work shows that accessible, fine-tuned open LLMs can surpass proprietary models on complex information extraction tasks, potentially broadening access to high-quality structured data extraction.
A new preprint introduces a diagnostic framework for evaluating test-time collaboration methods in large language models (LLMs), breaking down performance gains into measurable components: oracle gap and signal fidelity. The authors show that on benchmarks like LiveCodeBench, a high-fidelity verifier achieves a +8.14 percentage point improvement, while a low-fidelity verifier gains only +2.70pp. The framework enables practitioners to estimate the potential benefits and limitations of collaboration strategies before deployment.
Why it matters: This work provides a practical tool for anticipating the effectiveness of test-time collaboration in LLMs, helping avoid unnecessary investment in unproductive multi-agent approaches.
A new preprint analyzing five model families and seven bias types finds that sycophancy and related cue-induced biases in large language models (LLMs) are primarily introduced during alignment tuning, not pretraining. The study shows these biases correspond to distinct, causally active directions in model hidden states, which can be decoded and manipulated to recover unbiased answers. The findings suggest that each bias remains representationally distinct and that targeted interventions can partially debias model outputs.
Why it matters: This research clarifies the origins and structure of certain LLM biases, highlighting alignment tuning as a key source and suggesting new avenues for targeted debiasing.
Researchers introduce Trace-Based On-Policy Distillation (TOPD), a teacher-supervised framework for transferring reasoning abilities to masked diffusion language models without the need for reward estimation. TOPD enables the SDAR-4B-Chat model to achieve comparable accuracy to RL-trained models on the MATH500 benchmark, while requiring four times fewer rollout rounds and an estimated 96-fold compute speedup.
Why it matters: This approach offers a more efficient method for reasoning-oriented post-training of diffusion LLMs, potentially reducing computational costs compared to reinforcement learning-based methods.
Researchers developed a human-in-the-loop AI agent that assembles evidence and drafts one-sentence impact summaries for clinical scholars. In a real-world evaluation across 10 scholars, both reviewers accepted or edited 81.7% of the agent's findings, and the median review time per scholar dropped from an estimated 15 hours manually to just 14 minutes. The agent's evidence covered all four Translational Science Benefits Model (TSBM) domains and included non-scholarly impact categories often overlooked by manual processes.
Why it matters: This work shows that a human-in-the-loop AI agent can make large-scale impact reporting feasible for clinical research programs by drastically reducing staff time and improving coverage of impact categories.
TalTech's systems achieved first place in both tracks of the Beyond Transcription Challenge (BeTraC), generating SOAP notes directly from long doctor-patient conversation recordings without intermediate transcription. They adapted Voxtral Mini and Voxtral Small models using LoRA fine-tuning and DAPO reinforcement learning with the Open Medical Concept F1 metric as a reward. Independent evaluation found their submissions had the lowest hallucination rate among all entries.
Why it matters: This result shows that reinforcement learning with a concept-matching metric can improve factual reliability in medical summarization from speech, and that text-based fine-tuning can transfer effectively to speech input.
JOR-Bench is a suite of five Japanese-language benchmarks designed to evaluate large language models (LLMs) on operations research (OR) problems, comprising 1,319 problems across linear, mixed-integer, non-linear, and combinatorial optimization. The benchmarks are Japanese translations of established English datasets, enabling direct cross-lingual comparison. Evaluation of seven LLMs shows that strong multilingual models exhibit nearly identical OR formulation accuracy in Japanese and English, with only a -0.3 percentage point difference, though error analysis uncovers subtle language-specific failure modes.
Why it matters: JOR-Bench provides a rigorous tool for assessing LLMs' mathematical reasoning in Japanese, highlighting both the strengths and nuanced limitations of multilingual models across languages.
Researchers introduce KyrgyzLLM-Bench, a benchmark suite for evaluating large language models (LLMs) in Kyrgyz. The suite includes two natively authored datasets (KyrgyzMMLU and KyrgyzRC) and translated, manually post-edited versions of WinoGrande, HellaSwag, BoolQ, and TruthfulQA. Evaluation of 26 models shows that cross-lingual transfer varies by task, with translation artifacts notably affecting HellaSwag performance. Few-shot prompting improves some open-source models on reading comprehension, but results are inconsistent for proprietary models on translated tasks.
Why it matters: This work provides the first large-scale, natively authored evaluation benchmark for Kyrgyz, enabling more accurate assessment of LLM capabilities and cross-lingual transfer in a low-resource language.
A new preprint argues that achieving zero hallucination in large language models (LLMs) cannot be accomplished by the model alone, but must be enforced at the system level. The authors introduce HALO, a six-layer assurance architecture that includes grounded generation, constrained execution, multi-signal verification, calibrated abstention, total traceability, and continuous oversight. The framework is demonstrated on a regulated claims-extraction task.
Why it matters: This work reframes hallucination as a system-level failure mode and proposes a practical, multi-layered architecture for deploying trustworthy AI in regulated enterprise environments.
Policy & Safety→Official→arXiv Computation and Language
A preprint study evaluated the clinical safety of large language models (LLMs) in both English and Hausa, focusing on locally deployable models (4-9B parameters) and a frontier model. The results show that deployable models experienced a sharp drop in clinical correctness when answering in Hausa (mean score fell from 1.57 to -0.03), while the frontier model maintained high performance (2.00 to 1.75). The study attributes this deficit to the deployable model tier, rather than the language or clinical task.
Why it matters: This highlights a critical safety gap for low-resource healthcare settings that rely on small, locally-run language models, as clinical correctness may not transfer across languages.
Researchers introduce ESCUCHA, the first Spanish speech understanding benchmark designed to evaluate large audio language models across diverse acoustic conditions and reasoning abilities. The benchmark features 1,000 human-curated questions paired with 162.9 hours of real-world audio, encompassing multiple Spanish accents and non-normative speech. Initial evaluations show substantial performance gaps between state-of-the-art models and trained humans.
Why it matters: This benchmark fills a critical gap in evaluating audio language models for Spanish under realistic and challenging conditions, highlighting the need for more robust and capable multilingual speech AI systems.
A new benchmark dataset of 1,516 expert-verified Bangla sentences reveals that large language models (LLMs) exhibit a systematic bias toward the dominant meaning when disambiguating Bangla homographs that serve as both personal names and common nouns. The study demonstrates that contrastive chain-of-thought prompting and distilled cultural explanations can reduce this bias from as high as 100% to under 5% in small models, even turning a previously unsuccessful Bangla-specific model into the strongest performer.
Why it matters: This work demonstrates that language-specific pretraining is insufficient for cultural grounding and offers practical methods to mitigate cultural biases in low-resource language models.
C²KV is a unified framework for non-prefix key-value (KV) cache reuse in large language models that jointly optimizes KV extraction and inference-time concatenation. The method learns a composable and compressed KV cache manifold that is position-agnostic, enabling modular KV representations to be reused without modifying the base model. Experiments show that C²KV can achieve up to 17× inference speedup under long-context scenarios while preserving generation quality.
Why it matters: This work addresses the bottleneck of storing and accessing large KV caches in long-context LLM serving, offering significant speedups without loss of accuracy.
Researchers introduce Group Entropy-Controlled Policy Optimization (GEPO), a lightweight extension to GRPO that leverages group-level entropy to adapt advantage signals for reinforcement learning alignment of large language models (LLMs) on heterogeneous tasks. GEPO addresses the challenge of varying exploration needs across diverse task groups and demonstrates consistent outperformance over GRPO and other entropy-controlled methods on 13 benchmarks, including math, physics, code, and instruction following.
Why it matters: GEPO offers a practical advance for aligning LLMs on mixed-task datasets, enabling more balanced and effective training across diverse capabilities.
Researchers have found that hidden-state probes intended to measure output correctness in language models often capture the model's own self-judgement rather than objective correctness. By creating cases where the model's self-judgement and objective correctness disagree, they show that the self-judgement signal transfers reliably across tasks, while the objective-correctness signal does not. This indicates that transferability of probe signals does not guarantee they reflect objective truth.
Why it matters: This challenges the reliability of correctness probes for interpretability and safety, as they may conflate a model's self-assessment with actual correctness.
EvolvingWorld is a framework and benchmark designed for co-evolving characters and worlds in interactive literary simulations. It features a Character Agent and an LLM-based World Model that enable persistent, long-horizon state updates across diverse literary settings. The framework is supported by a dataset derived from 57 books, comprising 138,596 training samples and 222 test snapshots, and introduces a trajectory-level evaluation protocol. Experiments indicate that EvolvingWorld can improve the coherence and persistence of character and world development over extended simulations.
Why it matters: This work represents a notable advance in interactive literary simulation by enabling dynamic, persistent co-evolution of characters and worlds, addressing limitations of prior static or isolated approaches.
A controlled study evaluated the impact of biographical personas in system prompts on code generation by large language models (LLMs). The research found that persona effects varied by model: on Claude Opus, a librarian persona led to frequent in-character disclaimers and a drop in code correctness from 0.92 to 0.67, while GPT-5.5 did not exhibit these behaviors. The findings indicate that personas introduce model-dependent behavioral biases rather than consistent quality improvements.
Why it matters: This work highlights that persona-based prompting can unpredictably affect LLM performance, emphasizing the importance of model-specific prompt engineering.
DeLIVeR is a new framework that frames evidence retrieval for automated fact-checking as a reinforced strategic exploration task. It uses a Planner LLM to decompose complex claims into targeted questions, guiding traversal of structured knowledge graphs for high-precision evidence. On the LIAR, FEVER, and PolitiFact datasets, DeLIVeR achieves peak F1-scores of 83.73, 84.57, and 79.70, representing a 10-15% improvement over the HippoRAG2 baseline.
Why it matters: This work advances automated fact-checking by combining reinforcement learning with knowledge graph traversal, enabling more transparent and auditable multi-hop reasoning for misinformation detection.
A new preprint identifies that model collapse during iterative instruction tuning with synthetic data often appears as a polarization of competence, where strong skills are reinforced but weak skills degrade. The authors introduce KITE, a two-stage framework that uses failure-guided data generation and boundary-aware uncertainty curation to address this issue. Experiments across several datasets and open-source LLMs show that KITE achieves more stable improvements compared to strong synthetic-data baselines.
Why it matters: This work offers a practical approach to mitigating performance degradation in LLMs trained with synthetic data, addressing a growing challenge as model-generated data becomes more common.
A systematic evaluation finds that off-the-shelf large language models (LLMs), specifically GPT-5.2 and Gemini-2.5, can match or exceed the performance of six specialized privacy policy analysis tools across tasks such as contradiction detection, regulatory compliance analysis, and summarization. In manual labeling tasks, LLMs achieved precision up to 91.4% and recall up to 70.9%, demonstrating their ability to perform tool-specific functionalities without additional domain-specific training.
Why it matters: This study suggests that general-purpose LLMs could serve as effective alternatives to specialized privacy analysis tools, potentially reducing costs and simplifying compliance workflows.