A new method called Dual Attention Residuals (DAR) introduces reciprocal cross-stream interaction for historical retrieval in Transformer models. DAR computes depth weights from one stream to select information from another stream's history, improving validation loss across dense models (0.1B–1B parameters) and a 7B sparse-MoE model. The approach preserves depth-wise diversity and avoids redundancy seen in other two-stream architectures.
Why it matters: DAR provides a simple architectural change that consistently improves Transformer performance without increasing parameter count, potentially benefiting a broad range of language models.
A new preprint demonstrates that the narrative context of a task has a much greater influence on large language model (LLM) agent behavior than the assigned persona. Analyzing 1,890 sessions across three models and ten personas, the authors find that narrative priors account for 5-31 times more behavioral variance than persona, with this effect consistent across models and often linked to lower task success. The study also shows that removing anchor words from persona descriptions reduces cross-narrative behavioral consistency by 95%.
Why it matters: Understanding that narrative framing, not just persona, shapes LLM behavior is crucial for designing more robust and predictable AI agents.
Researchers introduce HPD-Parsing, a hierarchical parallel decoding paradigm for vision-language model (VLM)-based document parsing. By replacing full-page autoregressive generation with a main layout branch and concurrent block-level decoders, HPD-Parsing achieves 4,752 tokens per second—2.62 times the throughput of the fastest existing document parsing model—while maintaining competitive accuracy.
Why it matters: This work demonstrates a significant advance in document parsing efficiency, potentially enabling much faster processing of complex documents.
Researchers introduce CASE, a framework that enhances the faithfulness of chain-of-thought (CoT) reasoning in large language models by combining training-time causal alignment with inference-time structural enforcement. CASE employs counterfactual datasets and selective-loss fine-tuning to reinforce the dependence of answers on reasoning chains while suppressing shortcuts from instructions directly to answers. Experimental results across multiple models and benchmarks demonstrate a 37% average relative improvement in CoT faithfulness compared to strong baselines, without sacrificing accuracy.
Why it matters: Improving the faithfulness of CoT reasoning could make language model outputs more reliable and interpretable, addressing a key limitation in current LLM reasoning methods.
A new preprint introduces MaLoRA and MaRA, two adapter families that add selective state-space recurrence at token and context levels to improve reasoning in frozen large language models (LLMs). Evaluated on three LLM backbones and two multi-hop reasoning benchmarks, these methods achieve up to +9.3 F1 improvement over standard LoRA adapters, with consistent gains across all tested settings. The results suggest that dynamic, stateful adaptation can outperform static low-rank updates for complex reasoning tasks.
Why it matters: This work demonstrates a practical and effective way to boost LLM reasoning accuracy without modifying the base model, potentially enabling more efficient and adaptable AI systems.
Researchers introduce Dependency-Aware Intermediate QA Supervision (DAIS), a training framework that transforms teacher rationales into stage-level QA records, each conditioned on relevant previous states. DAIS improves final-answer accuracy across multiple datasets and model backbones, achieving up to a 5.6% accuracy gain over the strongest baseline on policy-compliance benchmarks. Controlled ablations indicate that dependency-conditioned supervision provides benefits beyond simply adding intermediate text.
Why it matters: DAIS demonstrates a lightweight method for enhancing complex reasoning in language models without altering inference procedures.
Researchers introduce MedDDC-Eval, a benchmark that decouples the evaluation of information-gathering policies from diagnosis generation in multi-turn medical consultation agents. By using a shared frozen diagnostic reader, the framework enables fairer comparison of evidence acquisition strategies. The study demonstrates that changing the diagnostic model alone can shift F1 scores by up to 19 points and reverse policy rankings, highlighting the importance of decoupled evaluation. Additionally, training policies with Group Relative Policy Optimization (GRPO) on this benchmark led to notable improvements in total scores on two test splits.
Why it matters: This work provides a more rigorous and interpretable framework for evaluating and improving medical AI agents by isolating evidence collection from diagnosis, enabling clearer attribution of performance.
Researchers introduce FiT, a diagnostic framework that evaluates small language models (LLMs) on vocabulary, knowledge, and contextualization for cybersecurity question answering. Their empirical study finds that fine-tuning often degrades vocabulary and parametric knowledge in small LLMs, with instruction-focused tuning causing significant knowledge collapse. Pre-fine-tuning FiT scores can predict the direction of post-tuning changes, enabling screening of unsuitable models before adaptation.
Why it matters: FiT could help reduce unnecessary fine-tuning and support safer deployment of small LLMs in critical domains like cybersecurity by identifying unsuitable models in advance.
A new preprint introduces Stochastic Meta-Unlearning (SMU), a bilevel framework designed to improve machine unlearning in vision-language models (VLMs). SMU leverages VLM-level feedback to optimize the language backbone for unlearning, addressing the challenge that text-only unlearning is insufficient when image information is present. Experimental results on two VLMs and multiple datasets show that SMU achieves superior forget-retain trade-offs compared to existing baselines and demonstrates transferability to new unlearning targets and methods.
Why it matters: This work advances the field of machine unlearning by demonstrating that multimodal feedback is essential for effective and transferable unlearning in vision-language models.
A new preprint presents controlled experiments examining how prompt format, instruction count, and context length affect large language model (LLM) instruction adherence and hallucination. The study finds that perfect response rates drop to zero by 80 instructions across all tested models and formats, and recall accuracy remains high up to 64-128k tokens before degrading sharply, with significant format-dependent differences. Notably, the experiments observe no fabrication (hallucinated facts) but a sharp rise in outright refusal to answer as models approach their context limits.
Why it matters: This work provides rare, systematic evidence on prompt design tradeoffs, clarifying practical limits on instruction count and context length for LLM reliability.
Researchers present RF-Agent, a framework that generates a domain-specific reasoning dataset for radio-frequency integrated circuit (RFIC) design using knowledge distilled from seven canonical RF textbooks. The resulting dataset contains over 11,000 samples and includes a dedicated multiple-choice benchmark. Experiments demonstrate that supervised fine-tuning notably enhances RF reasoning performance, particularly for small and medium-sized language models, and that semantic retrieval is the most effective retrieval-augmented generation (RAG) strategy tested.
Why it matters: This work introduces the first large-scale, reusable dataset and benchmark for RF circuit design, addressing a key bottleneck in applying language models to this specialized engineering domain.
Researchers introduce RLAES, a unified large language model framework that jointly optimizes automated essay scoring and feedback generation using reinforcement learning. The system incorporates Rubric-based Feedback Evaluation (RFE), which uses 166 fine-grained binary rubric items and an LLM-as-judge to make feedback quality measurable. On the ASAP benchmark, RLAES achieves the highest scoring performance among LLM-based methods (QWK = 0.803) while maintaining feedback quality comparable to GPT-5.5.
Why it matters: This work demonstrates a significant advance in automated essay scoring and feedback generation by leveraging reinforcement learning and rubric-based evaluation to achieve state-of-the-art performance without sacrificing feedback quality.
A new method enables frozen language-model agents to self-evolve their dialogue skills by predicting whether a given response will lead to positive or negative user feedback, using only fixed logged data. This 'future-feedback prediction' approach achieves over 75% accuracy on a proprietary sales-assistant dataset and allows for reproducible, offline optimization of conversational skills without requiring live user interactions.
Why it matters: This work introduces a verifiable offline optimization stage for conversational AI, addressing the challenge of evaluating counterfactual responses and potentially enabling more scalable and reliable self-improvement.
Researchers have identified a critical failure mode in long-context large language models (LLMs) called repetitive copying, where models copy input text into their reasoning traces instead of engaging in productive problem-solving. They introduce GEAR, a reward shaping method that encourages grounding in key evidence and penalizes copying from irrelevant context, leading to consistent improvements of up to +4.6 average points over standard reinforcement learning approaches across multiple benchmarks and model scales.
Why it matters: This work highlights and addresses a pervasive limitation in long-context LLMs, demonstrating that improved evidence grounding can significantly enhance reasoning performance and reduce unproductive copying.
A new preprint investigates cross-lingual factual inconsistency in large language models (LLMs), where answers shift depending on the prompt language. The researchers evaluate four inference-time interventions on Gemma 3 12B Instruct, including persona prompting, internal representation manipulation, and weight modification. They find that persona prompting—a simple contextual intervention—most effectively balances efficacy, safety, and generalization across languages, outperforming more invasive methods.
Why it matters: This work highlights a practical approach to improving factual consistency in multilingual LLMs, suggesting that simple prompt-based methods can be more robust and transferable than complex interventions.
Researchers present Fusion Embedding, a family of models that extend a frozen vision-language embedding base to include audio without updating its parameters. The first generation uses a 16.4M-parameter connector for audio, while the second adds modality-gated deep adapters (44.2M parameters) that activate only for audio inputs. The models enable zero-shot audio-image retrieval by aligning audio to text, and both generations can be trained in hours on a single GPU.
Why it matters: This work demonstrates a practical method for unifying text, image, video, and audio embeddings in a single space, enabling cross-modal retrieval without requiring paired audio-visual training data.
Researchers model Chain-of-Thought reasoning in language models as a switching dynamical system, revealing that reasoning fine-tuning globally reorganizes latent dynamics rather than merely improving token-level competence. The study finds that the recovered latent policy states show functional specialization aligned with reasoning stages, and causal interventions demonstrate their functional significance. Additionally, SDS-guided pruning of failure-prone reasoning prefixes outperforms self-consistency in 11 of 12 settings, with gains up to 12.5 percentage points.
Why it matters: This work introduces a novel mechanistic framework for analyzing and controlling reasoning in language models, with practical implications for enhancing reasoning performance through process-level interventions.
A collaboration between the European Commission's Directorate-General for Translation and the European Master's in Translation network has localized the MMLU dataset into 11 European languages. The project not only aims to create a more inclusive benchmark for evaluating large language models (LLMs), but also provides master's students with authentic, project-based professional training in translation and related skills.
Why it matters: This initiative addresses the lack of multilingual evaluation benchmarks for LLMs in European languages and demonstrates a novel approach to dataset creation through educational partnerships.
Researchers introduce SAGE (ScAffolded Generative models for Explanation), a neuro-symbolic framework that integrates language models with cognitive models to address pragmatic reasoning. SAGE decomposes pragmatic tasks into proposers (generating alternatives), evaluators (assessing alternatives), and selectors (implementing rule-based decisions), and is tested on referential expression generation, M-implicatures, and Gricean implicatures. The framework achieved high accuracy and often outperformed baselines, but analysis revealed that language model proposers excel at generating alternatives, while evaluators are better at intuitive rather than formal judgments.
Why it matters: This work represents a notable advance in computational pragmatics by providing a flexible, interpretable approach that leverages both cognitive modeling and generative language models to improve context-sensitive language understanding and production in AI.
A new preprint reports that when 44 language models are prompted to reply in structured formats like JSON, their answer diversity drops significantly compared to plain chat. The modal answer rate increases from 41% to 64%, and the number of distinct answers falls from 52 to 36. This effect is specific to formats such as JSON and XML, and is caused by the request for structured output itself, not by decoder constraints.
Why it matters: The findings suggest that the structured output formats widely used for software integration lead to less diverse model responses than the chat interfaces typically used for evaluation, potentially impacting downstream applications and model comparisons.