GPU-Accelerated Indexing in LanceDB

If you are just getting into this tech, like me, then this article leads down the rabbit hole of different index types supported in different vector stores. Most easy-to-use vector stores only use HNSW indices, which is "good enough".

A few offer different types of indices. Milvus[1] and FAISS[2] are a couple that support IVF indices. It's great to see another lightweight tool that does this with GPU acceleration.

1. https://objectbox.io/vector-database/

2. https://www.pinecone.io/learn/series/faiss/vector-indexes/

Faiss is a library that supports indexing, not a fully-fledged vector database on its own. Milvus uses Faiss with a few other libraries to build a full vector database (https://github.com/milvus-io/milvus#acknowledgments).

They do predicate pushdown for filtering too. Noice! https://lancedb.github.io/lance/read_and_write.html#filter-p...

LanceDB is one of the few options for embeddable vector databases, and I have used it in my Electron application. If they could choose a less confusing npm package name than "vectordb," maybe I would be more forgiving towards them. Moreover, the documentation for LanceDB is also poorly written.