VectorChord: Store 400k Vectors for $1 in PostgreSQL

Hi, now we adopted a dual license model (AGPLv3 and ELv2).

We don’t perform any reclustering. As you said, users would need to rebuild the index if they want to recluster. However, based on our observations, the speed remains acceptable even with significant data growth. We did a simple experiment using nlist=1 on the GIST dataset, the top-10 retrieval results took less than twice the time compared to using nlist=4096. This is because only the quantized vectors (with a 32x compression) need to be inserted into the posting list, and only quantized vector distances need more computations. And the quantized vector computation only accounts for a small amount of time. Most of the time is spent on re-ranking using full-precision vectors. Let's say the breakdown is approximately 20% for quantized vector computations and 80% for full-precision vector computations. So even if the time for quantized vector computations triples, the overall increase in query time would be only about 40%.

If the data distribution shifts, the optimal solution would be to rebuild the index. We believe that HNSW also experiences challenges with data distribution to some extent. However, without rebuilding, our observations suggest that users are more likely to experience slightly longer query times rather than a significant loss in recall.

I'm the project lead for VectorChord. I have tested ScaNN on AlloyDB Omni but have struggled to achieve reasonable recall on the GIST 1M dataset, with results peaking at only around 0.8. The limited documentation makes it challenging to understand the underlying causes of this performance.

Additionally, I couldn’t find any performance benchmarks for ScaNN integrated with PostgreSQL, particularly in comparison to pgvector or its standalone. The publicly available metrics focus exclusively on query-only indexing outside of the database.

On our side, we’ve implemented the fastscan kernel for bit vector scanning, which is considered as one of ScaNN’s key advantages.

The IVF indexing can be considered into two phases, computing the centroids (KMeans), and assigning each point to the centroids as the inverted lists. The most time-consuming part is at the KMeans stage, and can be greatly accelerated with GPU. 1M 960dim vec can be clustered in less than 10s. We did the KMeans phase externally, and the assignment phase inside postgres. The KMeans part depends only on the data distribution, not on any specific data. So we can do sampling on the data, and inserting/deleting the data won't affect the KMeans result significantly. For the update, it's just a matter of assigning the new vector to a specific cluster and appending it to the corresponding list. It's very light compared to inserting in hnsw

We will look into this. We've tried it before, but it took too long to build the index.

It's hard to compare the cost with the Serverless pricing model, as write and read have extra costs. On the pricing page, datastax costs $4000 to write 100M 768-dim vectors. And 10M query will cost $300, which is only 4 QPS. As comparison, VectorChord can achieve 100 QPS on $250 instance.

The pricing estimator is fake, the bill is real : ).

Looks like that there is recent work to make a pgvector example for this https://github.com/pgvector/pgvector-python/blob/master/exam...

There's no easy way to index ColBert multi-vectors in a scalable way that I know of. Vespa seems to rely heavily on binary quantization, which can cost a lot in recall loss. And for most cases, using ColBert as a reranker is good enough, as the pgvector example you posted.

I would like to throw our project in the ring that solves this problem: https://github.com/tjmlabs/ColiVara

1. Uses half-vecs, so you cut down everything by half with no recall loss 2. Uses token pooling with hierarchial clustering at 3, so, you further cut down things by 2/3rd with <1% loss 3. Everything is on Postgres and pgvector, so you can do all the Postgres stuff and decrease corpus size by document metadata filtering 4. We have a 5000+ pages corpus in production with <3 seconds latency. 5. We benchmark against the Vidore leaderboard, and very near SOTA

You can read about half-vecs here: https://jkatz05.com/post/postgres/pgvector-scalar-binary-qua...

Hierarchical token pooling: https://www.answer.ai/posts/colbert-pooling.html

And how we implemented them here: https://blog.colivara.com/

In the LAION-5M benchmark, we’ve compared our performance against ElasticSearch and OpenSearch. However, comparing ingestion performance is more challenging due to differences in architecture. Both ElasticSearch and OpenSearch, like most vector databases, use the concept of shards. Each shard represents a separate vector index, and queries aggregate results across these shards. Larger shards lead to faster queries but come with higher resource requirements and slower update speeds.

It’s also worth noting that ElasticSearch has implemented RaBitQ support for HNSW. So it's difficult to compare without running actual benchmarks. However, ElasticSearch typically requires at least double, if not triple, the memory size of the vector dataset to maintain system stability. In contrast, PostgreSQL can achieve a stable system with far fewer resources—for example, 32GB of memory is sufficient to manage 100 million vectors efficiently.

From my perspective, it would be faster in query comparing to ElasticSearch due to the extensive optimizations. And much much faster with the updates (insert and delete) due to using IVF instead of HNSW.

I'm the project lead for both projects. We're still in the process of supporting all the function from pgvecto.rs in VectorChord (int8, more than 2000 dim vec, etc.). We'll provide the migration docs for pgvecto.rs users to VectorChord. User will have better experience with VectorChord due to better integration with postgres storage system. We will stop supporting pgvecto.rs early next year when everything on VectorChord is ready.

First paragraph of tfa mentions "768-dimensional vectors"

That's because this product isn't for you then. My team has been evaluating vector databases for years and everything on the VectorChord page resonated with me. We run one of the world's largest vector databases and we'll likely benchmark vectorchord to see if it lives up to its promises here.

So they should start every one of their posts, on that same site, with a summary of what is available on the homepage?

And every hosting provider doesn't start by teaching you what is HTML