Comment by ankit219 | Hacker News Reader

Comment by ankit219 | Hacker News Reader

Been following this space for a bit. In my view, the literature on the internet is distorted about these things because of the need for certain OSS project creators and (non OSS) companies to promote their own products. It typically goes like this:

[Insert technique name] is a good technique, but it can be further improved if you enable [some complex workflow]. [Some example of how it improved a particular case]. We at [Company] already do this at scale and you can use it out of the box if you use it. Go make better things.

Nothing against people doing this stuff. It just makes me think how difficult is it to promote open source projects today.

Regarding Vector DBs, most have advocated for weaviate and pinecone. Incidentally, all run into quantization problems at scale (1M+ vectors) if the clusters are to be adjusted, and handle it differently. i tried it for a 6000 vector product[1] and got the best results by using a pkl file rather than any of the dbs.

[1] Not sharing the product here as such. We just scraped a govt tax website to create a search engine on top of that. Helps as the data is not indexed by google, and we had no interest from customers in wanting to chat. They were interested in original source along w a summary.

Bit sad how it's going. People go straight to product and venture scaled growth.

Build protocols, not products! I tell people.

It's a terrible advice actually, unless you like eating noodles and scraping by. But it's still the right way.

We're still waiting for the antirez' Redis version of Vector DBs. Maybe he should do one! :)

The good news is that this already exists in the Redis search module [1], which allows you to do similarity search against indexed embeddings, among other features, and offers comparable performance against other ANN libraries [2], depending on your performance criteria.

I've been using it for a side project to do semantic search on books[3] and have been really happy with its performance. (Not affiliated with any of this, was mostly interested in exploring existing well-performing, fairly standard tools with low latency)

[1] https://redis.io/docs/interact/search-and-query/search/vecto... [2] https://ann-benchmarks.com/#redisearch [3] https://viberary.pizza/how

Bit of a side note on Redis Search. While performance is pretty good, it is not production quality yet, mostly around monitoring indexing errors and quirky query results. Silent indexing errors pretty much make it impossible to use it in production, especially for structured data.

Redis Enterprise edition does have Vector database.i haven't used it but am curious about other people's experiences.

https://redis.com/solutions/use-cases/vector-database/

> Regarding Vector DBs, most have advocated for weaviate and pinecone. Incidentally, all run into quantization problems at scale (1M+ vectors)...

Can you elaborate? We (at Pinecone) have dozens of customers with over a billion embeddings, still serving queries in under 300ms (p95) with index updates in <1s. If you saw performance issues with just 1M embeddings then something else is going on that we may be able to help with.

It's a specific case. Most customers won't run into it. It's to do with quantization or specifically what happens when you add enough vectors that the original clustering changes (use cases could be hit when you onboard a IM/consulting customer which has so much data to alter clusters amongst others). It is not a performance issue to be fair. Just that altering a cluster takes time.

I was just testing to see if there is a DB which does not run into such problems, before putting a business case. I am not discouraging the use, just saying that most cases do not need a vector DB. The reason we did not choose one was because our business case's number of vectors was small.

I may be misunderstanding, but I'll try to answer — quantization typically means retrieval will be slower (if referring to techniques like product quantization), but that is the case whether you're at 10K vectors or 1B vectors, afaik it doesn't really make a difference because you're only quantizing the query vector at query time (it has been awhile since I read anything on quantization, so I could be mistaken).

Maybe your question is referring to the need to have quantization at larger index sizes? In which case, yes would typically be true because you're wanting to either (1) minimize the index size when quantizing it, or (2) optimizing the query space (to search through less). Whether you want (1) or (2) will impact on the type of quantization being performed (basically 1 == product quantization, and 2 == inverted index)

So once you get to the 1M+ size, you need to consider quantization in some form - or you can go with graph-based retrieval, if you don't mind using a lot of disk space.

You likely won't see performance (speed) benefits if you have just tens of thousands of vectors and of course brute force (vector matrix multiplication) will have better precision & recall.

Vector databases make sense when you have a lot of data or some other requirements (e.g. persistence, online updates, complex filtering, etc) that you would rather not implement yourself.

exactly. brute force methods work great for thousands pieces of data of (almost) any kind. for vector data no vector db is needed; for structured data even a simple csv file with a python script will do the job and a sql db is not necessary.

The thing is, depending on the dimensions of the vectors your using the growth rate can be pretty drastic. Each openai vector is 5kb and 1500 dimensions. So sure searching 1k vectors, just brute force it. But what if you have 100k vectors and a ton of users hammering at the index searching through it? Each search is now 150,000,000 dot products. Go up one more order of magnitude and you definitely need an index. And I don't think 1 million vectors is that much.

Number of vectors are determined by (other than original dataset ofc) how you chose to chunk the data you have available. Bigger chunks work better in terms of search (empirically) and they also keep the number of vectors down. For openai, based on prevalant norms and their cookbooks, 1M vectors likely mean 1M (more like 700K) pdf pages of text (at a token size of 1000 per embedding). That is a lot of textual data for a decent size company. Enterprises might reach that stage. Consulting firms definitely would - though they already trained and announced their own models.

700k pdf pages is not a lot. Also, you might be a business serving other businesses and indexing their documents and at reasonable (again, not google scale), 700k pages is again not a lot.

Another way to look at 700k pages is 2333 300 page books.

Reply on news.ycombinator.com