This is a ridiculous rant. “ oh no! We have choices”. Then you list out every choice available for what is a new space people are exploring and the list is barely a half dozen long? It’s more like this is peak “claiming everything is peak”.
I think what I mean to say is that, in my experience, practitioners and vendors alike are overly focused on "just put embeddings somewhere and do cosine similarity" and that's the only problem to solve. In fact, that's a teeny tiny part of it. Hence "peak vector DB".
So I think the market needs some education that its harder than that. That part is my rant :). I've spoken / worked on enough problems now to see that disconnect between market and reality.
Though I think "vector DB" is actually a place for capital/brainpower to concentrate to solve these other problems. And I think we'll see the vector DB vendors pivot there. It's just taking a while for the market and investors to see this...
I agree, and as one who does exactly and only this on the search side, it's also something that falls flat on its face if you don't think a little more about the data and tasks involved.
I wrote about it here[0], but the gist of it for our use case is that if we don't intentionally include what may be considered "less relevant" data then we stand a good chance at failing our main generative task.
[0]: https://phillipcarter.dev/2024/01/15/three-properties-of-dat...