Diffusion models may create by smoothing, not by magic

Generative AI’s “creativity” is often discussed as if it were either magic or plagiarism. Google Research’s July 15 post offers a more useful mechanism: diffusion models generate novel samples because neural networks learn a smoothed version of the score function used during denoising. The work, presented at ICLR 2026 as On the Interpolation Effect of Score Smoothing in Diffusion Models, also ships with linked code for the numerical experiments.

The setup starts with the standard diffusion recipe. A model corrupts training data with noise, then learns to reverse that corruption step by step. If it learned a perfect score function from the training samples, the denoising process would pull generated points back to exact training examples. That would make the model closer to a retrieval system than a generator. The practical result is different because neural networks do not usually learn sharp functions perfectly.

Google Research identifies that gap as score smoothing. Regularization such as weight decay, along with implicit regularization from gradient-based training, makes the learned score function smoother than the ideal one. In a one-dimensional example with two training points, the perfect score function sharply splits the space and drives each particle to one of the known samples. A smoothed score function slows particles in the middle, creating an interpolation zone where new plausible samples can settle.

The multi-dimensional version is the more interesting claim. Real images live on a hidden data manifold inside a much larger pixel space. Score smoothing does not simply slow movement everywhere, which would produce blurry junk. Instead, the effect appears direction-dependent: it mainly reduces collapse along directions parallel to the manifold while still allowing movement toward the manifold. That distinction helps explain how a model can preserve quality while producing outputs that are not direct copies of the training set.

The stakes are larger than an academic explanation of image generators. Memorization is now a product, safety, and copyright issue. If novelty is partly governed by score smoothing, model builders may gain a more precise lever for balancing fidelity, variation, and training-data leakage. Google Research is careful to frame the result as an initial effort rather than a final theory for every architecture and data distribution. Still, it gives the debate a concrete object to test: not whether diffusion models “understand creativity,” but how regularized denoising dynamics steer samples away from memorized endpoints.