Hacker News Reader

That depends a bit on the scale and use case specifics. But commoditized billion-scale vector search is indeed a thing. We published this for Weaviate in December last year https://weaviate.io/blog/sphere-dataset-in-weaviate

We've seen Milvus used in a variety of recommender systems running in production.

They’re embeddings so they’re dense. There are few things easier than dense vector similarity.

Embeddings for retrieval don't have to be. It is not unheard of to transform the raw embeddings to optimize them for retrieval; e.g., through binarization or hashing.

I was more making a distinction between embeddings and bag of words which are very very sparse matrices. The embedding dimensionality will not be anywhere near as high so this level of sparsity is a minor inconvenience.

Edit: also CPUs for this, yikes…

Such as...?

Qdrant, milvus, weaviate

Vespa.ai is pretty crazy too, a bit unkown we run a huge vespa cluster serving 1k+ queries with <100ms latency ...

Typesense: https://typesense.org/docs/0.24.1/api/vector-search.html