Yeah. Stable Diffusion itself is a bit on an oddball because there are so many actual dreambooth finetunes, not to speak of the merging culture.

Also, it all still runs in PyTorch eager mode last I checked. So it isn't even to the point where discussing optimized kernels like this is necessarily relevant.