Researchers introduce SIFT (Self-Improving, Frozen-gate Training), a dynamic document classification system that combines a cost-effective CPU-based pipeline (SPLADE + LightGBM) with selective escalation of uncertain cases to a large language model (LLM) judge. The LLM's decisions are incorporated into the training data, enabling the system to improve accuracy and reduce the need for manual labeling over time. SIFT employs a frozen-gate mechanism, including F1 regression checks and a golden test set, to prevent silent performance regressions during autonomous retraining.
Why it matters: SIFT offers a novel approach to reducing the high costs and risks associated with manual labeling and autonomous retraining in enterprise document classification.
Researchers present Search-on-Graph-R1, a method for training an 8B parameter language model to answer knowledge graph questions by internalizing graph navigation skills. The approach combines supervised fine-tuning and reinforcement learning, using a frontier teacher guided by gold SPARQL queries to generate grounded trajectories. Search-on-Graph-R1 achieves state-of-the-art results on the CWQ benchmark and outperforms larger frozen models on several datasets, without relying on auxiliary modules or LLM judges during training or inference.
Why it matters: This work shows that smaller, efficiently trained models can match or surpass much larger systems in knowledge graph question answering, potentially lowering deployment costs and complexity.
A new preprint demonstrates that adding lightweight depthwise convolution to Qwen3 Transformer blocks can improve average accuracy across seven benchmarks, while increasing model parameters by less than 0.01%. The most effective configuration applies convolution to the projected queries, keys, and values before attention, using a kernel size of 3 and no extra normalization. The method enhances the model's sensitivity to local context, particularly for repeated token IDs.
Why it matters: This approach provides a simple and highly parameter-efficient way to introduce local inductive bias into large language models, potentially improving their performance with negligible computational overhead.
Researchers introduce Relay-Bench, a text-only benchmark designed to evaluate large language models (LLMs) on their ability to solve multi-domain reasoning chains. The benchmark features composite problems that combine subproblems from distinct domains such as visual reasoning, coding, math, information extraction, and data analysis. The top-performing model, GPT-5.5 (xHigh), achieves a score of 43.3%, indicating substantial room for improvement. Models are allowed to use external tools like code execution and web search during evaluation.
Why it matters: Relay-Bench offers a more comprehensive and challenging assessment of LLMs' capacity to integrate and reason across multiple domains, revealing current limitations in multi-step reasoning.
AEGIS is a framework for investigating span-level guidance in multilingual text detoxification across English, Mandarin Chinese, and Korean. The study finds that providing explicit rationales can alter the trade-off between reducing toxicity and preserving meaning, but the impact varies depending on the generator backbone and linguistic context. Rather than focusing on achieving state-of-the-art results, the work analyzes when and how span-level control signals are effective for detoxification.
Why it matters: This research clarifies the conditional usefulness and limitations of span-level control signals for multilingual text detoxification, informing future development of safer language models.
Researchers introduce a method for document-level event argument extraction that leverages fine-tuned mid-sized open large language models (LLMs). Their approach integrates role-set injection, LoRA-based parameter-efficient fine-tuning, and deterministic decoding with schema validation, achieving 42.39% F1 on the MAVEN-ARG benchmark with Phi-4 (14B). This method outperforms previously reported GPT baselines on multiple evaluation metrics.
Why it matters: The work shows that accessible, fine-tuned open LLMs can surpass proprietary models on complex information extraction tasks, potentially broadening access to high-quality structured data extraction.
A new preprint introduces a diagnostic framework for evaluating test-time collaboration methods in large language models (LLMs), breaking down performance gains into measurable components: oracle gap and signal fidelity. The authors show that on benchmarks like LiveCodeBench, a high-fidelity verifier achieves a +8.14 percentage point improvement, while a low-fidelity verifier gains only +2.70pp. The framework enables practitioners to estimate the potential benefits and limitations of collaboration strategies before deployment.
Why it matters: This work provides a practical tool for anticipating the effectiveness of test-time collaboration in LLMs, helping avoid unnecessary investment in unproductive multi-agent approaches.
A new preprint analyzing five model families and seven bias types finds that sycophancy and related cue-induced biases in large language models (LLMs) are primarily introduced during alignment tuning, not pretraining. The study shows these biases correspond to distinct, causally active directions in model hidden states, which can be decoded and manipulated to recover unbiased answers. The findings suggest that each bias remains representationally distinct and that targeted interventions can partially debias model outputs.
Why it matters: This research clarifies the origins and structure of certain LLM biases, highlighting alignment tuning as a key source and suggesting new avenues for targeted debiasing.
Researchers introduce Trace-Based On-Policy Distillation (TOPD), a teacher-supervised framework for transferring reasoning abilities to masked diffusion language models without the need for reward estimation. TOPD enables the SDAR-4B-Chat model to achieve comparable accuracy to RL-trained models on the MATH500 benchmark, while requiring four times fewer rollout rounds and an estimated 96-fold compute speedup.
Why it matters: This approach offers a more efficient method for reasoning-oriented post-training of diffusion LLMs, potentially reducing computational costs compared to reinforcement learning-based methods.
Researchers developed a human-in-the-loop AI agent that assembles evidence and drafts one-sentence impact summaries for clinical scholars. In a real-world evaluation across 10 scholars, both reviewers accepted or edited 81.7% of the agent's findings, and the median review time per scholar dropped from an estimated 15 hours manually to just 14 minutes. The agent's evidence covered all four Translational Science Benefits Model (TSBM) domains and included non-scholarly impact categories often overlooked by manual processes.
Why it matters: This work shows that a human-in-the-loop AI agent can make large-scale impact reporting feasible for clinical research programs by drastically reducing staff time and improving coverage of impact categories.
TalTech's systems achieved first place in both tracks of the Beyond Transcription Challenge (BeTraC), generating SOAP notes directly from long doctor-patient conversation recordings without intermediate transcription. They adapted Voxtral Mini and Voxtral Small models using LoRA fine-tuning and DAPO reinforcement learning with the Open Medical Concept F1 metric as a reward. Independent evaluation found their submissions had the lowest hallucination rate among all entries.
Why it matters: This result shows that reinforcement learning with a concept-matching metric can improve factual reliability in medical summarization from speech, and that text-based fine-tuning can transfer effectively to speech input.
JOR-Bench is a suite of five Japanese-language benchmarks designed to evaluate large language models (LLMs) on operations research (OR) problems, comprising 1,319 problems across linear, mixed-integer, non-linear, and combinatorial optimization. The benchmarks are Japanese translations of established English datasets, enabling direct cross-lingual comparison. Evaluation of seven LLMs shows that strong multilingual models exhibit nearly identical OR formulation accuracy in Japanese and English, with only a -0.3 percentage point difference, though error analysis uncovers subtle language-specific failure modes.
Why it matters: JOR-Bench provides a rigorous tool for assessing LLMs' mathematical reasoning in Japanese, highlighting both the strengths and nuanced limitations of multilingual models across languages.
Researchers introduce KyrgyzLLM-Bench, a benchmark suite for evaluating large language models (LLMs) in Kyrgyz. The suite includes two natively authored datasets (KyrgyzMMLU and KyrgyzRC) and translated, manually post-edited versions of WinoGrande, HellaSwag, BoolQ, and TruthfulQA. Evaluation of 26 models shows that cross-lingual transfer varies by task, with translation artifacts notably affecting HellaSwag performance. Few-shot prompting improves some open-source models on reading comprehension, but results are inconsistent for proprietary models on translated tasks.
Why it matters: This work provides the first large-scale, natively authored evaluation benchmark for Kyrgyz, enabling more accurate assessment of LLM capabilities and cross-lingual transfer in a low-resource language.
A new preprint argues that achieving zero hallucination in large language models (LLMs) cannot be accomplished by the model alone, but must be enforced at the system level. The authors introduce HALO, a six-layer assurance architecture that includes grounded generation, constrained execution, multi-signal verification, calibrated abstention, total traceability, and continuous oversight. The framework is demonstrated on a regulated claims-extraction task.
Why it matters: This work reframes hallucination as a system-level failure mode and proposes a practical, multi-layered architecture for deploying trustworthy AI in regulated enterprise environments.
Policy & Safety→Official→arXiv Computation and Language
A preprint study evaluated the clinical safety of large language models (LLMs) in both English and Hausa, focusing on locally deployable models (4-9B parameters) and a frontier model. The results show that deployable models experienced a sharp drop in clinical correctness when answering in Hausa (mean score fell from 1.57 to -0.03), while the frontier model maintained high performance (2.00 to 1.75). The study attributes this deficit to the deployable model tier, rather than the language or clinical task.
Why it matters: This highlights a critical safety gap for low-resource healthcare settings that rely on small, locally-run language models, as clinical correctness may not transfer across languages.
Researchers introduce ESCUCHA, the first Spanish speech understanding benchmark designed to evaluate large audio language models across diverse acoustic conditions and reasoning abilities. The benchmark features 1,000 human-curated questions paired with 162.9 hours of real-world audio, encompassing multiple Spanish accents and non-normative speech. Initial evaluations show substantial performance gaps between state-of-the-art models and trained humans.
Why it matters: This benchmark fills a critical gap in evaluating audio language models for Spanish under realistic and challenging conditions, highlighting the need for more robust and capable multilingual speech AI systems.
A new benchmark dataset of 1,516 expert-verified Bangla sentences reveals that large language models (LLMs) exhibit a systematic bias toward the dominant meaning when disambiguating Bangla homographs that serve as both personal names and common nouns. The study demonstrates that contrastive chain-of-thought prompting and distilled cultural explanations can reduce this bias from as high as 100% to under 5% in small models, even turning a previously unsuccessful Bangla-specific model into the strongest performer.
Why it matters: This work demonstrates that language-specific pretraining is insufficient for cultural grounding and offers practical methods to mitigate cultural biases in low-resource language models.
C²KV is a unified framework for non-prefix key-value (KV) cache reuse in large language models that jointly optimizes KV extraction and inference-time concatenation. The method learns a composable and compressed KV cache manifold that is position-agnostic, enabling modular KV representations to be reused without modifying the base model. Experiments show that C²KV can achieve up to 17× inference speedup under long-context scenarios while preserving generation quality.
Why it matters: This work addresses the bottleneck of storing and accessing large KV caches in long-context LLM serving, offering significant speedups without loss of accuracy.
Researchers introduce Group Entropy-Controlled Policy Optimization (GEPO), a lightweight extension to GRPO that leverages group-level entropy to adapt advantage signals for reinforcement learning alignment of large language models (LLMs) on heterogeneous tasks. GEPO addresses the challenge of varying exploration needs across diverse task groups and demonstrates consistent outperformance over GRPO and other entropy-controlled methods on 13 benchmarks, including math, physics, code, and instruction following.
Why it matters: GEPO offers a practical advance for aligning LLMs on mixed-task datasets, enabling more balanced and effective training across diverse capabilities.
Researchers have found that hidden-state probes intended to measure output correctness in language models often capture the model's own self-judgement rather than objective correctness. By creating cases where the model's self-judgement and objective correctness disagree, they show that the self-judgement signal transfers reliably across tasks, while the objective-correctness signal does not. This indicates that transferability of probe signals does not guarantee they reflect objective truth.
Why it matters: This challenges the reliability of correctness probes for interpretability and safety, as they may conflate a model's self-assessment with actual correctness.