What I can not wrap my head around: How are these models trained?
What training mechanism or model architecture provides the glue to go from human text to images?
Don't you need to have millions of really descriptively labelled images?
What training mechanism or model architecture provides the glue to go from human text to images?
Don't you need to have millions of really descriptively labelled images?
Slightly longer answer for older text to image models you teach them how to encode images and text into the same latent space. Then you simply do a conversion, take a text input, put it into latent space and then extract the image that latent space represents.
There are ML models that do the reverse and output image to text, which assist quite a lot.
The better the text represents the unique thing in the photo, the better the model understands what that text means.