A new preprint reports that when 44 language models are prompted to reply in structured formats like JSON, their answer diversity drops significantly compared to plain chat. The modal answer rate increases from 41% to 64%, and the number of distinct answers falls from 52 to 36. This effect is specific to formats such as JSON and XML, and is caused by the request for structured output itself, not by decoder constraints.
Why it matters: The findings suggest that the structured output formats widely used for software integration lead to less diverse model responses than the chat interfaces typically used for evaluation, potentially impacting downstream applications and model comparisons.
Researchers introduce SIFT (Self-Improving, Frozen-gate Training), a dynamic document classification system that combines a cost-effective CPU-based pipeline (SPLADE + LightGBM) with selective escalation of uncertain cases to a large language model (LLM) judge. The LLM's decisions are incorporated into the training data, enabling the system to improve accuracy and reduce the need for manual labeling over time. SIFT employs a frozen-gate mechanism, including F1 regression checks and a golden test set, to prevent silent performance regressions during autonomous retraining.
Why it matters: SIFT offers a novel approach to reducing the high costs and risks associated with manual labeling and autonomous retraining in enterprise document classification.
Researchers present OntoBook, a method that transforms medical ontology structures into synthetic textbook-style prose using large language models, and uses this data to pretrain a French medical encoder (ModernCamemBERT). The approach leads to significant improvements on three French medical coding benchmarks, including up to +8.0 micro-F1 on Distemist-FR. The team also releases 1.3 million generated textbooks and pretrained model checkpoints.
Why it matters: This work offers a scalable way to inject structured medical knowledge into language models, potentially advancing clinical NLP in low-resource languages.
PhoenixRepair is a multi-agent framework designed to enhance automated software repair by systematically exploring multiple candidate edit locations and employing iterative reflection and refinement in patch generation. On the SWE-bench-Verified benchmark, PhoenixRepair achieves a 76.0% Pass@1 resolution rate under MiniMax-M2.5 and demonstrates a 7.8% relative improvement over SWE-agent under DeepSeek-V3.1. The framework also reports higher fault localization accuracy compared to existing methods.
Why it matters: PhoenixRepair expands the search space for automated software repair, leading to notable improvements in resolution rates and fault localization accuracy over prior approaches.
Researchers present Search-on-Graph-R1, a method for training an 8B parameter language model to answer knowledge graph questions by internalizing graph navigation skills. The approach combines supervised fine-tuning and reinforcement learning, using a frontier teacher guided by gold SPARQL queries to generate grounded trajectories. Search-on-Graph-R1 achieves state-of-the-art results on the CWQ benchmark and outperforms larger frozen models on several datasets, without relying on auxiliary modules or LLM judges during training or inference.
Why it matters: This work shows that smaller, efficiently trained models can match or surpass much larger systems in knowledge graph question answering, potentially lowering deployment costs and complexity.
A new preprint demonstrates that adding lightweight depthwise convolution to Qwen3 Transformer blocks can improve average accuracy across seven benchmarks, while increasing model parameters by less than 0.01%. The most effective configuration applies convolution to the projected queries, keys, and values before attention, using a kernel size of 3 and no extra normalization. The method enhances the model's sensitivity to local context, particularly for repeated token IDs.
Why it matters: This approach provides a simple and highly parameter-efficient way to introduce local inductive bias into large language models, potentially improving their performance with negligible computational overhead.
Researchers introduce Relay-Bench, a text-only benchmark designed to evaluate large language models (LLMs) on their ability to solve multi-domain reasoning chains. The benchmark features composite problems that combine subproblems from distinct domains such as visual reasoning, coding, math, information extraction, and data analysis. The top-performing model, GPT-5.5 (xHigh), achieves a score of 43.3%, indicating substantial room for improvement. Models are allowed to use external tools like code execution and web search during evaluation.
Why it matters: Relay-Bench offers a more comprehensive and challenging assessment of LLMs' capacity to integrate and reason across multiple domains, revealing current limitations in multi-step reasoning.
MAGE is a multimodal, multi-agent framework for macro placement refinement in chip design, combining structured floorplanning rules, visual checks, and iterative refinement. In tests across nine designs, MAGE achieved geometric-mean improvements of 11.1%-19.3% in Worst Negative Slack (WNS) and 70.0%-74.0% in Total Negative Slack (TNS) over commercial macro placers. On three designs with human-expert baselines, MAGE outperformed human experts by 18.3% in WNS and 72.5% in TNS, and also improved human-likeness metrics by 6%-48% over all baselines.
Why it matters: This work demonstrates a significant advance in automated chip floorplanning, showing that a multimodal, agentic approach can outperform both commercial tools and human experts in macro placement.
Researchers introduce Evidence Chain Evaluation (ECE), a selective fact-checking framework for large language models (LLMs) that allows the system to abstain from issuing a verdict when supporting evidence is weak or inconsistent. On the ECE-Bench benchmark, ECE achieves 91.6% standard accuracy, 93.7% coverage, and 97.8% selective accuracy on claims it chooses to answer, deferring 6 out of 95 cases, mostly in low-reliability evidence settings. While ECE does not outperform the strongest retrieval baseline on aggregate calibration metrics, it demonstrates a strong trade-off between accuracy and selective abstention.
Why it matters: This work provides a practical safety mechanism for LLM-based fact-checking by enabling abstention on uncertain claims, addressing a key reliability challenge in automated verification systems.
Researchers fine-tuned Qwen3-0.6B and Qwen3-1.7B language models on a 21,250-example synthetic arithmetic dataset generated by GPT-5-mini. This fine-tuning improved exact-match accuracy on the GSM8K benchmark from 36.5% to 49.1% for Qwen3-0.6B and from 53.5% to 66.5% for Qwen3-1.7B. The fine-tuned Qwen3-1.7B model also showed strong transfer performance on MultiArith (98.9%) and SVAMP (73.0%), compared to much lower scores for the base model. The study demonstrates that structured synthetic reasoning data can substantially enhance arithmetic reasoning in small language models under consumer hardware constraints.
Why it matters: This work demonstrates a practical and effective approach to improving arithmetic reasoning in small language models, making them more capable for local use on consumer devices.
A new preprint introduces OrderBench, a benchmark designed to evaluate restaurant ordering agents powered by large language models (LLMs). The study finds that while LLMs can reliably produce outputs that are syntactically and schema-valid, these outputs often contain significant semantic errors. Even the strongest tested model achieved 100% schema validity but only around 80% semantic success, and weaker models exhibited double-digit rates of unsafe acceptances. The authors caution that structured output formats like JSON Schema do not guarantee transaction safety or correctness.
Why it matters: This work exposes a critical limitation in relying solely on schema-constrained outputs for LLM-driven transactional systems, emphasizing the need for additional domain verification to ensure safety and correctness.
A new diagnostic framework, SAAG, breaks down agent-calling evaluation into three sequential stages: registry conformance, structural completeness, and argument grounding. Experiments on a function-calling benchmark show that providing stage-specific feedback enables more precise argument selection and reduces hallucinated values compared to binary feedback. The framework also supports targeted self-repair of agent calls without exposing ground-truth values.
Why it matters: SAAG offers a more granular and actionable approach to diagnosing and improving agent-calling reliability, addressing limitations of traditional binary evaluation methods.
AlayaWorld is a 15-billion-parameter video diffusion transformer capable of generating interactive, persistent 24-fps video at 540p and 720p resolutions from text, image, or video inputs. It employs autoregressive latent chunk generation with bounded visual context and introduces a novel distillation method that reduces inference steps from about 30 to 4 per chunk. The model demonstrates state-of-the-art performance on long-horizon video generation benchmarks.
Why it matters: AlayaWorld represents a significant advance in interactive world modeling by enabling efficient, persistent, and customizable long-horizon video generation from minimal input.
Researchers introduce S2T-RLHF, a sentence-to-token reward decomposition framework designed to improve training stability in reinforcement learning from human feedback (RLHF). By assigning preference rewards at the sentence level before refining them at the token level, S2T-RLHF addresses instability caused by noisy, overly fine-grained token-level credit assignment. Experiments demonstrate that this approach enhances stability and robustness while maintaining competitive alignment performance.
Why it matters: This work proposes a novel granularity-aware credit assignment method that could make preference-based RLHF training more stable and reliable for large language models.
A new method called MUX enables language models to perform more efficient reasoning by distilling discrete reasoning steps into continuous multiplexed tokens in a latent space. This approach uses lossless superposition of subword spans, allowing for parallel exploration in search problems and more compact reasoning. In evaluations across 32 settings and four language models, MUX outperformed strong latent reasoning baselines. Analyses indicate that the latent tokens encode faithful and interpretable reasoning.
Why it matters: MUX demonstrates a novel and practical advance in language model reasoning efficiency, potentially reducing computational bottlenecks by enabling higher-bandwidth latent representations.
A new method uses inference-time schema-derived constraints to generate Multi-Level Intermediate Representation (MLIR) code across different dialects without retraining language models. The researchers introduce four new NL-to-MLIR benchmarks and demonstrate that a 1.7B parameter model, using their constraint stack, can match or outperform much larger 15B-34B code language models on structurally constrained dialects, achieving 8-25x faster generation. The approach requires no new constraint-layer code when porting to new dialects dominated by structural constraints, and all benchmarks, code, and reproducibility tools are released.
Why it matters: This work shows that schema-derived priors can replace retraining for MLIR code generation across dialects, potentially reducing the need for per-dialect fine-tuning in ML compiler infrastructure.
A new preprint introduces a latency-aware query router for large language models (LLMs) that jointly optimizes for latency, accuracy, and cost. The proposed system uses a lightweight estimator to predict time-to-first-token (TTFT) for each query, enabling more informed routing decisions. Experimental results show that this approach can improve accuracy–cost utility by up to 40% compared to standard load-balancing methods, without increasing latency.
Why it matters: Incorporating latency into LLM query routing addresses a key challenge for real-time applications, enabling more efficient and responsive model serving.
A preprint investigates how different state compression methods impact constraint preservation in a two-agent LLM relay for travel planning. The study finds that schema-constrained JSON extraction yields the highest feasibility accuracy (0.96), while narrative summarization significantly reduces it (0.48). Embedding-based pruning achieves feasibility accuracy comparable to the uncompressed baseline (0.88) without requiring additional generative calls.
Why it matters: The work demonstrates that structured, auditable hand-off formats are crucial for maintaining constraints in multi-agent LLM systems, challenging the notion that brevity alone is sufficient.
Researchers introduce 'Fence,' a method that uses small language models (SLMs) trained on synthetic data as specialized guardrails for large language model (LLM) applications. The approach leverages a novel GAN-inspired synthetic data generation technique to produce high-quality training samples, enabling SLMs to address application-specific safety concerns such as hallucination and topic drift. Experimental results indicate that SLM guardrails outperform prompt-based LLM guardrails in these tasks.
Why it matters: This work presents a scalable and cost-effective strategy for enhancing the safety of LLM deployments by enabling tailored, application-specific guardrails.
Researchers present PEARL, a system that integrates solver feedback and iterative revision into the process of translating natural language descriptions into formal optimization models. PEARL learns when to test partial models and how to revise them based on solver diagnostics, operating in a multi-turn, tool-integrated setting. Experiments show that PEARL achieves higher verified solve rates than both one-shot and tool-augmented baselines, with the 4B-parameter PEARL-Qwen3 model outperforming the much larger 685B-parameter DeepSeek-V3.2 on optimization modeling tasks.
Why it matters: This work demonstrates that interactive, solver-in-the-loop approaches can significantly improve automated optimization modeling, enabling more practical and efficient AI-assisted decision-making.