Researchers propose Posterior Prefix Tuning (PPT), a method for steering the behavior of Bayes-filtered transformers (BFTs) by optimizing prompts using importance sampling from the latent posterior. PPT does not require forward passes or backpropagation through the transformer, and a single set of prior samples can be reused for multiple utility functions at minimal additional cost. The method is validated on Beta-Bernoulli and reinforced urn BFTs across three types of utility functions.
Why it matters: PPT provides a computationally efficient alternative to gradient-based optimization for adapting transformer models to diverse objectives.
Researchers introduce PDDIM, a new algorithm for solving linear inverse problems using diffusion priors. The method modifies standard DDIM updates with coordinate-wise adjustments based on signal-to-noise ratio, and is proven to converge to the Bayesian posterior. Empirical results demonstrate that PDDIM performs favorably compared to existing diffusion-based posterior samplers across various image restoration tasks.
Why it matters: This work offers a practical and theoretically grounded approach to posterior sampling in inverse problems, combining empirical effectiveness with provable guarantees.
Researchers have introduced a statistical framework for auditing privacy in synthetic data, capable of distinguishing true disclosures from phantom ones using hypothesis testing. The method requires only synthetic outputs and a held-out control set—no model access, canary insertion, or reference model training. It is model-agnostic and provides tighter empirical lower bounds on privacy leakage than previous data-based auditing methods, while being more resource-efficient.
Why it matters: This framework enables practical and efficient detection of privacy leaks in synthetic data, addressing a key safety concern in generative AI without requiring access to the underlying model.
Researchers propose partial fusion of neural networks, a method that interpolates between full ensembles and weight aggregation to balance computational cost and model performance. The technique aggregates only the most similar neurons across networks, using partial optimal transport for matching, and generalizes to a form of pruning that allows for isolating, deleting, or combining neurons. This approach provides a flexible framework for managing the tradeoff between accuracy and efficiency in neural network deployment.
Why it matters: This method could make neural network ensembles more practical by reducing their computational demands while preserving much of their performance.
FVRuleLearner is a new framework that applies Operator Reasoning Trees (Op-Trees) to automate the translation of natural language specifications into SystemVerilog Assertions (NL-to-SVA) for formal verification. By decomposing the alignment process into operator-aware reasoning steps, FVRuleLearner enables more accurate and interpretable rule generation. In evaluations, it outperformed state-of-the-art baselines in both syntax and functional correctness, and significantly reduced SVA functional failures across operator categories.
Why it matters: This work introduces a novel, interpretable approach that improves the reliability and efficiency of automated formal verification, which is essential for ensuring hardware correctness.
Researchers have introduced QuArch, the first benchmark specifically designed to evaluate large language models' (LLMs) knowledge and reasoning abilities in computer architecture. QuArch includes 2,671 expert-validated question-answer pairs spanning topics such as processor design and memory systems. Leading LLMs achieved between 34% and 73% accuracy on advanced questions, and fine-tuning on QuArch led to improved performance on a realistic memory hierarchy design task, yielding more area-efficient and viable hardware solutions.
Why it matters: QuArch addresses a significant gap in LLM evaluation by providing a domain-specific benchmark for computer architecture, enabling more rigorous assessment and development of AI capabilities in this critical field.
Researchers introduce a discrete denoising diffusion framework that leverages a sample-efficient estimator (NeurISE) for single-site conditional probabilities, eliminating the need for score function approximations. Experiments on binary datasets, including Ising models, MNIST, and quantum annealer data, show that this method outperforms existing approaches in several evaluation metrics.
Why it matters: This work presents a more sample-efficient approach to discrete diffusion, potentially advancing generative modeling for scientific and binary data.
Researchers propose latent distribution matching (LDM) as a new theoretical framework that unifies various self-supervised learning (SSL) methods, including contrastive, non-contrastive, and predictive approaches. The framework also leads to a nonlinear Bayesian filtering model for high-dimensional time series and demonstrates identifiability of latent representations under mild conditions. LDM clarifies the assumptions behind existing SSL methods and offers principled guidance for developing new approaches.
Why it matters: This work provides a unified theoretical foundation for self-supervised learning, potentially enabling more systematic and effective development of representation learning methods.
A new controlled longitudinal study investigates the impact of agent harness evolution on coding agent quality, finding that quality regressions often attributed to large language models (LLMs) are actually caused by updates to the agent harness. By evaluating 35 sequential releases of the Qwen Code CLI while keeping the underlying LLM constant, the study reveals that quality fluctuations are linked to specific development patterns and architectural changes in the harness.
Why it matters: This research challenges the common assumption that LLMs are primarily responsible for coding agent regressions, highlighting the significant influence of agent harness middleware and the importance of its careful management.
Researchers introduce SWE-Milestone, a benchmark designed to evaluate AI agents on streams of milestone-level coding tasks that simulate real-world software evolution. By testing 12 advanced models across 4 agent frameworks, they observe that performance drops from over 80% on isolated tasks to 38.03% in continuous, evolving scenarios. This highlights significant challenges for AI agents in maintaining system integrity and managing error propagation over time.
Why it matters: This benchmark reveals a major limitation in current AI coding agents: their inability to reliably handle long-term software maintenance, which is essential for real-world applications.
TraceDev is a multi-agent framework designed to convert natural language requirements into repository-level code using five specialized agents and a traceability graph. In evaluations on two datasets, TraceDev achieved success rates of 53.63% and 56.82%, outperforming state-of-the-art baselines by up to 340.80%. The framework emphasizes explicit requirement traceability, supporting more reliable code generation for complex software scenarios.
Why it matters: TraceDev introduces a novel approach to requirement traceability in automated code generation, addressing a key limitation in current LLM-based software development methods.
Researchers present Spaghetti Architect, a tool that generates code datasets in five programming languages (Python, JavaScript, Go, Java, C++) with controlled difficulty and resistance to data contamination. The tool produces correct-by-construction programs from a known-optimal reference, allowing for reliable evaluation of code models. It enables fine-grained control over code 'messiness' and labels each instance along intrinsic and incidental difficulty axes.
Why it matters: This tool addresses the persistent challenge of data contamination in code model evaluation by enabling the creation of fresh, controlled, and contamination-resistant datasets.
Researchers introduce LISA, a framework that leverages large language models (LLMs) to generate API sequences and program invariants for detecting functional bugs in software libraries. LISA achieves higher bug-detection rates and competitive code coverage compared to traditional fuzzing and previous LLM-based test generation methods.
Why it matters: LISA offers a new approach to uncovering non-crashing functional bugs that are often missed by conventional fuzzing techniques, potentially improving software reliability.
Researchers have released SciCodePile, a 128GB dataset of scientific code collected from 37,737 public repositories, along with a 200-task executable benchmark designed to rigorously test code generation by large language models (LLMs). Evaluation of 15 LLMs on this benchmark revealed that even the best-performing model achieved only 12.30% Pass@1 on executable tasks, indicating that scientific code generation remains a major challenge for current LLMs.
Why it matters: This work establishes a new large-scale standard for evaluating scientific code generation and demonstrates that current LLMs are not yet reliable for this critical application area.
A new preprint demonstrates that widely used benchmarks for evaluating LLM-generated GPU kernels can incorrectly certify buggy code as correct. The study shows that these benchmarks, which rely on fixed-shape, small-sample allclose checks, often miss subtle errors. By applying op-schema-aware seeded fuzzing with high-precision references, the authors catch all tested bugs without false positives across five different GPU classes.
Why it matters: This work reveals a critical flaw in current evaluation methods for LLM-generated GPU code, indicating that many kernels considered correct may actually harbor undetected bugs.
Researchers used large language models (LLMs) to generate Alloy formal specifications from both documentation and source code for two Python libraries, then derived executable tests from these specifications. Their pipeline uncovered a genuine bug in Flipper—a feature flag management system—where duplicate flag names were silently accepted, a defect missed by a direct LLM test generation baseline that achieved 68% branch coverage. Additionally, code-derived specifications showed lower variance in test generation compared to documentation-derived ones.
Why it matters: This study demonstrates that LLM-generated formal specifications can reveal constraint-level software defects that may be overlooked by coverage-focused test generation, suggesting a new avenue for improving automated software validation.
Researchers introduce 'falsifiable release gates,' a methodology requiring every new capability in self-improving AI systems to pass a pre-declared, machine-checkable acceptance suite while preserving fixed safety invariants. This approach was implemented in the Antahkarana runtime, where safety invariants held unchanged across six releases, and the acceptance suite expanded from 122 to 563 tests. The system demonstrated gated self-improvement on real hardware, increasing a small model's accuracy from 20% to 70% while automatically rejecting unsafe candidates.
Why it matters: This work offers a practical, verifiable framework for maintaining safety guarantees in self-improving AI systems, addressing a key challenge in AI alignment and deployment.
Researchers have introduced eight uncertainty and five quality metrics tailored for evaluating Vision-Language-Action (VLA) robots, moving beyond traditional binary success rates. In a large-scale study involving 908 successful task executions from three state-of-the-art VLA models, these metrics demonstrated moderate to strong correlation with human expert judgments. The findings suggest that these metrics can better capture task quality and model confidence, and can distinguish between varying levels of execution quality even when binary success is achieved.
Why it matters: This work offers a more nuanced and informative evaluation framework for VLA robots, supporting improved real-time monitoring and adaptive system improvement.
A new preprint formalizes 'binding drift' in tool-augmented language-model agents, where entity bindings that are correct at step 1 can silently become incorrect in later steps, distinct from simple error propagation. In a controlled testbed, the commonly suggested 'entity lock' fix actually amplifies wrong actions by 3x, while a lightweight LLM-based re-verifier reduces wrong actions by 79%, nearly matching an oracle upper bound.
Why it matters: This work identifies a previously underappreciated failure mode in multi-step tool-augmented agents and demonstrates a practical mitigation, with direct implications for the reliability of such systems.
A new preprint introduces Skillware, a software abstraction designed to formalize and manage persistent behavioral artifacts—called Skill Artifacts—in AI agent systems. The paper defines Skillware Units as managed software entities with independent identity and lifecycle, operationalized through three necessary conditions: behavioral primacy, independent software identity, and an Agent Host execution relationship. The work is supported by analysis of a large corpus of SKILL.md records and multiple engineering implementations, establishing a recurring artifact structure and lifecycle engineering considerations.
Why it matters: Skillware offers a formal ontology and lifecycle for agent skills, potentially enabling more robust, maintainable, and evolvable software artifacts in AI agent ecosystems.