Hacker News Reader

CEO of Neon here. After we built an in memory HNSW index for Postgres that allowed us to establish a baseline in performance and prove that it's the right approach to support vector search we now built it "the right way" and now it support restarts of Postgres, replication and the rest of the Postgres machinery.

View on HN

Can you clarify:

Does the the initial construction (creation) of the index need to fit into RAM?

Also, should one expect the reading via Postgres's buffer cache to have better access patterns / need less memory / have other benefit vs having a non-Postgres HNSW index that's mmapped from disk?

The index doesn't need to fit in the shared buffers to be built, but you can speed the queries if you load it to memory

Can you explain: What does it mean to be constructed on disk?

This means that the index is persisted in Postgres pages which are durable against restarting Postgres. Our previous version what fully in-memory and wasn't "durable".

Reply on news.ycombinator.com