A new arXiv preprint introduces Learn2Discern (L2D), a benchmark designed to test large language models' (LLMs) ability to weigh information from external sources. Evaluating 13 models across nearly 670,000 trials, the study finds that LLMs perform near chance at distinguishing reliable sources and updating beliefs toward the truth. While newer and larger models show some improvement in truth discernment, they do not improve at recognizing source reliability, highlighting a persistent limitation.
Why it matters: This finding raises concerns about the reliability of LLMs as they are increasingly used to access and evaluate information online.
A new preprint stress-tests leading medical AI models in open-ended clinical conversations with missing information, revealing that the choice of LLM judge significantly alters apparent safety ratings. The study finds only moderate agreement between judges and identifies a same-provider bias, where models appear safer when evaluated by their own provider's judge. LLM judges are also more lenient than human clinicians, indicating that observed safety gaps are due to calibration differences rather than knowledge deficits.
Why it matters: This work highlights that current evaluation practices for medical AI may overstate safety due to evaluator bias, underscoring the need for independent, human-anchored assessments before clinical deployment.
Researchers introduce SciHazard, a benchmark comprising 2400 hazardous and 600 oversafety questions across 12 scientific disciplines, grounded in real-world regulated entities and documented failure scenarios. They propose DeHarm-Score, a decomposed evaluation framework that improves agreement with expert annotations by 90.17% over the strongest baseline. Evaluation of 31 frontier LLMs and deep research agents shows that agents yield a 32.3% higher mean DeHarm-Score, indicating greater safety risks compared to standard models.
Why it matters: SciHazard provides a rigorous, domain-grounded method for evaluating scientific misuse risks in LLMs and agents, revealing that autonomous agents may pose significantly greater hazards than standard models.
AgentDebugX is an open-source framework that structures LLM agent debugging into a closed loop of Detect, Attribute, Recover, and Rerun. Its core component, DeepDebug, achieves 28.8% exact attribution accuracy on the Who and When benchmark and repairs 13 of 73 failed tasks on GAIA in a single rerun, outperforming several baselines. The toolkit is available as a Python library, CLI, web console, and installable agentic skill.
Why it matters: AgentDebugX provides systematic root-cause diagnosis and recovery for LLM agents, addressing a key challenge in improving agent reliability and robustness.
A new preprint proposes using semantic primes from the Natural Semantic Metalanguage (NSM) as foundational elements to explain and control emotions in large language models (LLMs). Experiments on four instruction-tuned LLMs demonstrate that NSM primes are more recoverable, controllable, and selective than appraisal-based directions, with interventions on NSM primes controlling emotion about three times as strongly and twice as selectively. The study suggests that NSM primes may serve as more effective explanatory primitives for emotion in LLMs than existing approaches.
Why it matters: This work introduces a potentially more interpretable and effective framework for understanding and manipulating emotions in LLMs, which could impact model alignment and safety.
A new framework called DWM (Decomposed World Model) is proposed to separate latent world model transitions into action-driven and action-invariant components. This approach enables the model to distinguish between changes caused by agent actions and those arising from intrinsic environment dynamics, such as gravity or inertia. In experiments on modified control benchmarks with persistent world effects, DWM achieved a mean absolute improvement of 13.1% in planning success compared to strong baselines.
Why it matters: This decomposition could lead to more transferable and interpretable world models, improving model-based control in environments with complex dynamics.
A new framework called TARA (Type-Aware Repair Allocation) is introduced for optimizing prompts in text-to-image generation. Unlike previous methods, TARA routes each failed proposition in a prompt to a type-specific repair operator, enabling more precise corrections without retraining the generator. Experiments show TARA achieves the highest semantic accuracy across eight benchmark-generator combinations, outperforming VisualPrompter by 5.6 and 2.6 points on DSG and TIFA benchmarks, respectively, and operates faster at 16.0 seconds per prompt.
Why it matters: TARA offers a novel, training-free approach to improving the reliability and accuracy of text-to-image generation, addressing heterogeneous prompt failures more effectively than prior methods.
Researchers have identified a novel attack surface in graph foundation models: the alignment layer that maps diverse graph inputs into a shared representation. They demonstrate that targeted perturbations in this representation space can severely degrade model performance, with some models—such as OpenGraph—being particularly vulnerable. Additionally, input-space attacks that modify edges, features, or text can eliminate at least half of correct predictions in three out of six tested models.
Why it matters: This work exposes a previously unexamined vulnerability in graph foundation models that can be exploited at inference time without access to training data, raising new security concerns for real-world deployments.
Researchers have introduced SkillSight, a training-free retrieval framework designed to improve skill selection for large language model (LLM) agents by calibrating shared descriptive background in skill libraries. SkillSight addresses the bias introduced by recurring descriptive patterns in skill descriptions, leading to more accurate retrieval. Experiments show that SkillSight improves Recall@10 by up to 20.21 percentage points and is up to 1,248 times faster than dense+reranker baselines.
Why it matters: SkillSight offers a significant advance in efficient and accurate skill retrieval for LLM agents, which is essential as skill libraries grow larger and more complex.
Researchers introduce Black-Mamba, a test-time adaptive forecasting architecture that uses evidence-gated state tracking to distinguish persistent distribution shifts from transient noise. The model updates its internal memory only when accumulated surprisal signals a regime change, reducing unnecessary updates while maintaining competitive or improved predictive performance on non-stationary forecasting benchmarks. This approach is inspired by biological mechanisms and is supported by mathematical analysis.
Why it matters: This work presents a novel, principled method for efficient and robust online adaptation in AI systems facing non-stationary environments.
Researchers present OntoBook, a method that transforms medical ontology structures into synthetic textbook-style prose using large language models, and uses this data to pretrain a French medical encoder (ModernCamemBERT). The approach leads to significant improvements on three French medical coding benchmarks, including up to +8.0 micro-F1 on Distemist-FR. The team also releases 1.3 million generated textbooks and pretrained model checkpoints.
Why it matters: This work offers a scalable way to inject structured medical knowledge into language models, potentially advancing clinical NLP in low-resource languages.
PhoenixRepair is a multi-agent framework designed to enhance automated software repair by systematically exploring multiple candidate edit locations and employing iterative reflection and refinement in patch generation. On the SWE-bench-Verified benchmark, PhoenixRepair achieves a 76.0% Pass@1 resolution rate under MiniMax-M2.5 and demonstrates a 7.8% relative improvement over SWE-agent under DeepSeek-V3.1. The framework also reports higher fault localization accuracy compared to existing methods.
Why it matters: PhoenixRepair expands the search space for automated software repair, leading to notable improvements in resolution rates and fault localization accuracy over prior approaches.
MAGE is a multimodal, multi-agent framework for macro placement refinement in chip design, combining structured floorplanning rules, visual checks, and iterative refinement. In tests across nine designs, MAGE achieved geometric-mean improvements of 11.1%-19.3% in Worst Negative Slack (WNS) and 70.0%-74.0% in Total Negative Slack (TNS) over commercial macro placers. On three designs with human-expert baselines, MAGE outperformed human experts by 18.3% in WNS and 72.5% in TNS, and also improved human-likeness metrics by 6%-48% over all baselines.
Why it matters: This work demonstrates a significant advance in automated chip floorplanning, showing that a multimodal, agentic approach can outperform both commercial tools and human experts in macro placement.
A new position paper argues that most AI/ML deepfake research is misaligned with the primary real-world abuse of generative AI: the creation of non-consensual sexualized imagery. Through a landscape analysis of highly-cited works, the authors show that technical interventions overwhelmingly focus on epistemic harms like fraud, while largely ignoring subject-centric dignity harms such as AIG-NCII. The paper recommends updating threat models and integrating AIG-NCII into AI safety research, and urges researchers to adopt safety guardrails and collaborate with sexual violence prevention experts.
Why it matters: This paper identifies a significant oversight in AI safety research, calling for greater attention to the most prevalent and harmful uses of generative AI.
Researchers introduce Evidence Chain Evaluation (ECE), a selective fact-checking framework for large language models (LLMs) that allows the system to abstain from issuing a verdict when supporting evidence is weak or inconsistent. On the ECE-Bench benchmark, ECE achieves 91.6% standard accuracy, 93.7% coverage, and 97.8% selective accuracy on claims it chooses to answer, deferring 6 out of 95 cases, mostly in low-reliability evidence settings. While ECE does not outperform the strongest retrieval baseline on aggregate calibration metrics, it demonstrates a strong trade-off between accuracy and selective abstention.
Why it matters: This work provides a practical safety mechanism for LLM-based fact-checking by enabling abstention on uncertain claims, addressing a key reliability challenge in automated verification systems.
Researchers fine-tuned Qwen3-0.6B and Qwen3-1.7B language models on a 21,250-example synthetic arithmetic dataset generated by GPT-5-mini. This fine-tuning improved exact-match accuracy on the GSM8K benchmark from 36.5% to 49.1% for Qwen3-0.6B and from 53.5% to 66.5% for Qwen3-1.7B. The fine-tuned Qwen3-1.7B model also showed strong transfer performance on MultiArith (98.9%) and SVAMP (73.0%), compared to much lower scores for the base model. The study demonstrates that structured synthetic reasoning data can substantially enhance arithmetic reasoning in small language models under consumer hardware constraints.
Why it matters: This work demonstrates a practical and effective approach to improving arithmetic reasoning in small language models, making them more capable for local use on consumer devices.
A new preprint introduces OrderBench, a benchmark designed to evaluate restaurant ordering agents powered by large language models (LLMs). The study finds that while LLMs can reliably produce outputs that are syntactically and schema-valid, these outputs often contain significant semantic errors. Even the strongest tested model achieved 100% schema validity but only around 80% semantic success, and weaker models exhibited double-digit rates of unsafe acceptances. The authors caution that structured output formats like JSON Schema do not guarantee transaction safety or correctness.
Why it matters: This work exposes a critical limitation in relying solely on schema-constrained outputs for LLM-driven transactional systems, emphasizing the need for additional domain verification to ensure safety and correctness.
A new arXiv preprint empirically identifies two critical failure modes in LLM-based autonomous agents: Safety Drift, where initial safety alignment erodes over multi-turn interactions, and Operational Hallucination, where agents repeatedly invoke tools due to flawed state perception. The authors introduce an Action-Aware Supervision Layer, a lightweight architectural addition that enforces intent-action consistency and runtime state tracking. Simulations on real failure cases show this layer can intercept unsafe actions without causing false positives on benign tasks.
Why it matters: The work exposes structural vulnerabilities in current AI agent architectures and proposes a practical, enforceable solution to improve reliability and safety in autonomous agent deployment.
A new diagnostic framework, SAAG, breaks down agent-calling evaluation into three sequential stages: registry conformance, structural completeness, and argument grounding. Experiments on a function-calling benchmark show that providing stage-specific feedback enables more precise argument selection and reduces hallucinated values compared to binary feedback. The framework also supports targeted self-repair of agent calls without exposing ground-truth values.
Why it matters: SAAG offers a more granular and actionable approach to diagnosing and improving agent-calling reliability, addressing limitations of traditional binary evaluation methods.
AlayaWorld is a 15-billion-parameter video diffusion transformer capable of generating interactive, persistent 24-fps video at 540p and 720p resolutions from text, image, or video inputs. It employs autoregressive latent chunk generation with bounded visual context and introduces a novel distillation method that reduces inference steps from about 30 to 4 per chunk. The model demonstrates state-of-the-art performance on long-horizon video generation benchmarks.
Why it matters: AlayaWorld represents a significant advance in interactive world modeling by enabling efficient, persistent, and customizable long-horizon video generation from minimal input.