Diffusion Models
andrewkchan.dev
andrewkchan.dev
[1] https://arxiv.org/abs/1503.03585 [2] https://arxiv.org/abs/2006.11239 [3] https://arxiv.org/abs/2105.05233
"Train a single diffusion model on every training sample x0x0 twice: once paired with its class label yy, and once paired with a null class label."
not doing exactly the same, and having the same problem that was deemed bad in the first paragraph of the same section:
"However, the label can sometimes lead to samples that are not realistic or lack diversity if the model has not seen enough samples from p(x∣y)p(x∣y) for a particular yy. So we often want to tune how much the model “follows” the label during generation."
Awesome post btw
That said I am not 100% convinced : first the underlying model is still the same and because you are effectively learning a eps(x, t, y). secondly the equivalence suggests (if you actually use the difference as gradient) you should end up in the same global min/max as if you were doing this directly no ?
Yes, when conditioned on a non-null label, the underlying model for classifier-free guidance can be seen as the same as a class-conditional model (in fact, the original CFG paper[0] discusses training a separate model - they didn't do that because it's extra work for not much benefit). And for a certain weighting of the guidance vector for each step, you can obtain the same sampling step you would get with a class-conditional model. But my understanding is that the point of guidance is that you can control this weighting, because you are able to estimate the unconditional score too.
I hope the author stuck with it anyway. The more AI encroaches on creative work, the more I want to tear it all down.
After all, photography didn't stop people from drawing or painting.
Not much to TL;DR for the comment lurkers. This post is the TL;DR of stable diffusion.
The loss function encodes information about the noise and, because the network sees the noised up image exactly, this is equivalent to learning about the actual sample images. It's worth noting that you could design a loss function measuring the difference between the output and the real images. This contains equivalent information, but it turns out that the properties of gaussian noise make it much more conducive estimating the gradient.
But point being, the information on the true images is in the loop albeit only through the lense of some noise.
xts = alpha_bar[t].sqrt() * x0s + (1.-alpha_bar[t]).sqrt() * eps
The only disappointment came when I hit Reader View—almost to prove “this page is semantically perfect!!”—and alas, the nav list has a line height less than one in that medium, and scrunches up all crazy. I’ll let it slide ;)