How We Made PostgreSQL a Better Vector Database | Hacker News Reader

How We Made PostgreSQL a Better Vector Database

timescale.com

Title is a little disingenuous? Isn't this a SaaS service? So not making PostgreSQL (OSS database) anything.

Blog author here. This index is built in PostgreSQL. We make it very clear that this is cloud-only in the post and never mention OSS. While we offer it via the Timescale cloud platform, everything under the hood is still PostgreSQL and works with the entire postgresql ecosystem.

I have been looking into Postgres as a vector database option for some of the reasons you outlined in this blog. One thing I noticed when digging into pg_embedding specifically is "ef_search" controls the MAX number of neighbors you can return, which I don't think is the case for other hnsw indexes. Upping this value greatly increases the time of indexing (indexing with ef_search of 1000 took like 3-4 hours)

Does the Timescale vector index limit the number of results you can return? Does that impact speed of indexing or search? I wrote a blog about our somewhat niche requirements, which includes that we need to return ~10K or more results. If curious: https://medium.com/earthrisemedia/finding-a-vector-database-...

Interested in whether that would be a possibility with Timescale.

Hah! This was actually one of the main algorithmic challenges of adapting DiskANN to PostgreSQL. Yes, I think it's common for these algorithms to assume you know how many results to return ahead of time. But in PostgreSQL that's not how things work -- because of things like post-index-retrieval-filtering the right interface for Postgres is one that just keeps on returning more and more results until all possible matches are exhausted. We solved this by creating a "streaming" version of the search algorithm that keep state like which nodes in the graph have been visited, which have been returned etc.

That's all to say -- Yes we've solved this, there are no arbitrary limits on the number of results returned.

Hi, we've solved the problem you mentioned! Please take a look on our open source postgres vector extension https://github.com/tensorchord/pgvecto.rs.

Our index building process is significantly faster than pgvector on hnsw because we can utilize all the cores, whereas pgvector can only use one core. And for the filter support, we do support pre-filtering, which will guarantee enough results no matter the condition is.

Lead engineer here, happy to answer any questions

It actually works if only using 2GB of RAM? Just looking at the pricing costs.

Personally, without a way to run it locally, I'd look at either using pgvector or something like ChromaDB.

Yup 2GB is fine (especially if using quantization), depending on the dataset size of course.

Totally fair about cloud-only. Many developers prefer developing on cloud, but some prefer local dev. YMMV.

By the way timescale vector offers pgvector as well so it's easy to test and compare. (Note: I work for Timescale)

This looks really cool! I'm excited to see a production deployment of DiskANN.

According to the single-threaded QPS experiments, your DiskANN solution should clock in at about 4.5ms latency (1000ms/224QPS) whereas pgvector is about 5.8ms latency (1000ms/173QPS). How is that possible? My (very shallow) knowledge of DiskANN vs HNSW tells me that DiskANN should generally have higher latency than HNSW — DiskANN needs to touch the SSD while HNSW only touches RAM.

Also, compared to pgvector and HNSWPQ in faiss, how much less RAM does your DiskANN-based solution use?

(Blog author here). Thanks for the question. In this case the index for both DiskANN and pgvector HNSW is small enough to fit in memory on the machine (8GB RAM), so there's no need to touch the SSD. We plan to test on a config where the index size is larger than memory (we couldn't this time due to limitations in ANN benchmarks [0], the tool we use).

To your question about RAM usage, we provide a graph of index size. When enabling PQ, our new index is 10x smaller than pgvector HNSW. We don't have numbers for HNSWPQ in FAISS yet.

[0]: https://github.com/erikbern/ann-benchmarks/

Reply on news.ycombinator.com