Does anyone use OpenSearch for its knn and vector capabilities? Is it any good? It’s always hard to know with systems like this whether it works at scale until your team is fighting fires.
So if you want to build a very, very large index using HSNWs, you have to understand if you normally have many writes that accumulate evenly, or if your index is a mostly read-only thing that is rebuilt from time to time. Mass-insertion the first time is going to be very slow. You can parallelize it if you build N parallel HNSWs, since the searches can be composed as the union of the results (sorted by cosine similarity). But often the bottleneck is the embedding model itself.
What is really not super scalable is the size of HNSWs. They use of memory is big (Redis by default uses 8 bit quantization for this reason), and on disk they require seeks. If you have large vectors, like 1024 components, quantization is a must.
https://docs.opensearch.org/docs/1.2/search-plugins/knn/appr...
For whatever an endorsement from a random stranger is worth, we've been using opensearch for a vectordb for hybrid search across text and multimodal embeddings as well as traditional metadata and it's been great but we're not "full production" yet so I can't really speak to scale, but it's opensearch so I expect the scale to be fine most probably.
If you’re using lucene HNSW, it will scale but will eat lots and lots of Heap RAM. If you’re using FAISS or nmslib plugins keep an eye out for JNI RAM consumption as well as its outside the heap.
Overall, I’d say that it is a challenge to easily scale ANN past 100M vectors unless it’s given significant attention from the team.
The quality of your results will depend mostly on the quality of your embeddings