I agree having a good vector is important to start with. However this is not very hard to make it work, you only need to finetune some of the clip models[1] to run it well.
Disclose: I have built a vector search engine to proof this idea[2]
Disclose: I have built a vector search engine to proof this idea[2]