Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation
A new preprint introduces Seer, a training-free framework that accelerates inference in diffusion multimodal large language models (DMLLMs) by detecting the valid semantic boundary at the first denoising step using MLP activation sparsity. Seer truncates redundant suffix tokens, eliminating unnecessary computation and achieving up to 31x throughput acceleration. The method maintains or slightly improves performance on benchmarks, including a small accuracy gain on DocVQA, and requires no model retraining.
Why it matters: This approach offers a significant advance in inference efficiency for DMLLMs, enabling faster and more practical deployment in real-world applications without sacrificing accuracy.
Full story at: arXiv AI/ML ↗