I guess an interesting way to translate this technique to text-to-image would be to get an image from a database that matches the text query (via CLIP) and feed that + noise into a diffusion model that only does a few denoising iterations (and no clip guidance maybe). Would be a lot faster than from-scratch diffusion.

(Another way could be to redo the architecture to include a “inspired by this image” input, which is queried from an image server at inference time.) Anyone have other ideas?