Hacker News Reader

Are there details on how it works? How is CLIP being used here?

The code is available in the repo: https://huggingface.co/spaces/osanseviero/draw_to_search/blo...

It first uses a simple CNN to get one of the 100 classes. It then uses that class to retrieve images using CLIP. The original CLIP Space that helped for inspiration is https://huggingface.co/spaces/vivien/clip.

Oof. That 100 class limit is less practical than just asking the user for arbitrary text input and using CLIP as a zero-shot classifier. You would have gotten better responses if the limitations and motivations were explained on the page.

Thanks for the feedback! This was just a demo I shared in Twitter and didn't expect it to be shared so widely by others. We're training a new CNN for more classes and we might switch to another dataset for the image retrieval part. It's really a very small pet project :D

Reply on news.ycombinator.com