Diffusion models recover accurate mixture weights despite score function insensitivity
A new preprint addresses the paradox that score-based generative models can cover all modes of a multimodal distribution but may not learn the correct relative mode amplitudes (mixture weights). The authors show that generated samples can still recover mixture weights accurately if the score functions at intermediate noise levels are informative. They introduce the diffusion score sensitivity index (DSSI), which quantifies how changes in parameters affect the diffusion score matching loss and governs estimation accuracy. Empirical results demonstrate that DSSI predicts mixture weight recovery and that noise schedule choices can impact sensitivity and mode amplification.
Why it matters: This work provides a theoretical and empirical framework for predicting when diffusion models can accurately recover distribution parameters, informing model design and evaluation.
Full story at: arXiv Machine Learning ↗