On-disk HNSW index for Postgres with pg_embedding

This is comparing apples and oranges. You've pointed to some technologies from vector database vendors. Neon isn't a vector database vendor. They're a Postgres vendor. The objective of pg_embedding, pg_vector, and others is to offer teams who deploy Postgres the opportunity to use their existing infrastructure for vector search. The added and important benefit here is hybrid search: using existing DB data to filter semantic search results.

What the work done by Neon, the pgvector team, Supabase and others points to is that "speed" isn't the only factor in vector database selection. Developer experience and existing infrastructure investment are too.

DiskANN construction is much much slower than HNSW, and the support around updates has been discussed in literature but has not been open sourced yet (They call this FreshDiskANN in the paper - DiskANN is from microsoft research btw).

HNSW-IF is an excellent extension to HNSW (that the vespa team has made easy to implement) that takes advantage of the speed/recall of HNSW in combination with the disk scalability of inverted indices - it is a hybrid approach.

DiskANN has some great ideas that will likely make it into pg_embedding. The most important implication is how many vectors it can support. DiskANN claims it can manage 1Bln vectors and it's a huge deal. Index construction does take a long time, but we are researching how we can scale it out.

I think it's good technology. Our team flagged it, we haven't tested it yet.

Can you explain: What does it mean to be constructed on disk?

Can you clarify:

Does the the initial construction (creation) of the index need to fit into RAM?

Also, should one expect the reading via Postgres's buffer cache to have better access patterns / need less memory / have other benefit vs having a non-Postgres HNSW index that's mmapped from disk?

We used 1M vectors of 1536 dimensions (OpenAI). The index size was ~6GB.

Different vector indexing algorithms under the hood, with different tradeoffs.

In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the differences.

tsvector is something else; it's not a numeric vector index, it's a different kind of data structure that stores the actual lexemes (keywords) with positions and weights to facilitate full text search (rather than using a text embedding model.)

pg_embedding is based on a different approximate nearest neighbors algorithm - HNSW, which is generally considered - and studied - to be faster as well as more accurate than pgvectors IVF (ignoring a lot of nuance).

However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data.

For example

Chroma -> Serves HNSW out of memory, persists to disk with a WAL.

Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability.

Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.)

QDrant -> Has a WAL, supports memory and mmap'ed indices.

It's under Apache license