Hacker News Reader

Comment by rvrs | Hacker News Reader

This is tangential, but another issue I've had with existing vector databases is write throughput. Weaviate and Pinecone build an index immediately after the creation of a "table" (or rather, the analogue thereof) and this makes write throughput abysmal at times.

We have around 2 million 768-length embeddings + metadata -- writing too fast to Weaviate quickly results in OOMs. We circumvented this by very slowly trickling our data through to Weaviate, but this causes many-hours-long writes.

I'm experimenting with dumping our data into postgres and building an hnsw index with pgvector once all writes are done. This has reduced our time to maybe 2 hours at most (the bulk of that being index building time, of course).

View on HN

Hey @rvrs, I work on Weaviate and we are doing some improvements around increasing write throughput:

1. gRPC. Using gRPC to write vectors has had a really nice performance boost. It is released in Weaviate core but here is still some work on do on the clients. Feel free to get in contact if you would like to try it out.

2. Parameter tuning. lowering `efConstruction` can speed up imports.

3. We are also working on async indexing https://github.com/weaviate/weaviate/issues/3463 which will further speed things up.

In comparison with pgvector, Weaviate has more flexible query options such as hybrid search and quantization to save memory on larger datasets.

I once solved worst-case write performance in an open source MVCC data store (Apache Lucy, a “loose C” port of Lucene) like so:

1) Limit the maximum size of an update. This causes gradual degradation of the index, which is ordinarily dealt with through periodic consolidation — but now this this consolidation will be disabled in the main write mechanism.

2) Enable a background consolidation process which runs concurrently. At the end of the consolidation process a short write lock will need to be obtained while you replay all deletions that have occurred since consolidation started, but the time required isn’t problematic.

This fix required major surgery to the Lucy indexer core code, so it’s something that that would only be feasible if the data store itself supported it.

@rvrs, Have you tried Pinecone's S1 pods? Curious which indexing type you went with when experimenting (https://docs.pinecone.io/docs/indexes)

Disclaimer: I'm from Pinecone, hello!

Yeah basically all the vector "database" solutions in market have chosen data-dependent indexes, so you need the data upfront. Imagine if regular databases needed all data upfront before they could build indexes. It's kind of crazy...

Reply on news.ycombinator.com