CLIP is a such a nice paradigm shift. Historically, CV things were quite limited:
- You could predict a class (from a static list such as [dog,cat, ...]) or ...
- You could use image embeddings disconnected from text (you could tell image look-alikes but not what they actually represent). By "embedding" text and images in the same latent space, you can now query your images with text query (such a "a large dog") and find the relevant photos. CLIP understands semantics but also is not limited to a set list of classes (thanks to the ability to use of web data in training).
This is a list compiled by OpenCLIP of high performance models (some better than MetaCLIP) for those interested in using CLIP: https://github.com/mlfoundations/open_clip/blob/main/docs/op...