Investing in new vector database development vs enhancing existing databases
blog.det.life
blog.det.life
The existing DB + vector index option seems so obvious to me that I'm worried I'm missing something.
Performance, features, and scalability
run a search against ES => hydrate records from postgres
run a search against a vector database => hydrate LLM context from documents
Neither is a primary canonical datastore. Postgres is for ES and S3 is for documents.
Pgvector seems to perform very poorly compared to Qdrant: https://nirantk.com/writing/pgvector-vs-qdrant/
But if I was using Qdrant myself I'd treat it like ElasticSearch - I'd denormalize some of my data into it, but I'd still do most of my work in the relational database and treat Qdrant as effectively an external index for my stuff.
Maybe I'm getting hung up on the world "database" here as indicating that you use that instead of an RDBMS, when actually everyone selling a vector database expects you to use it as effectively an external indexing mechanism instead.
The embeddings are not primary data, they're derived from other content and can be recreated if necessary. The vectors will probably be fed or read in bulk by your ML pipeline at times.
I'd say you probably want the primary storage of your embeddings to be file storage like S3 or similar, and then feed them directly into a search engine like Vespa or Elasticsearch.
Every place where I'm deploying vector databases, though, I'm using them as an alternative index just for searches (indexing embedding/id only) and doing the full object value lookup in the RDBMS. It's easier to optimize their vector clustering / sort performance outside the constraints of postgres - which is perhaps why pgvector suffers more under higher load.
For (1), all of the indexing features of a typical RDBMS are overkill for IVF, where the only indexing you need is by IVF bucket, and then you simply sequentially scan and evaluate distances for everything in those buckets.
However, if you want to perform filtering based on other criteria (e.g., filtering based on metadata associated with each vector), then multiple column indexes could be useful with a RDBMS, as you can more efficiently extract just the vectors that meet criteria prior to a sequential scan (e.g., give me all vectors which have a date in the range between X and Y and whose name contains the string "red").
For (2), these algorithms would be inefficient to express in a typical RDBMS. While you can easily encode a graph in a RDBMS, walking the graph sequentially based on criteria is another matter, because each time you'd have to hit the disk (or in-memory cache) to find the next row to look at, then repeat. Latency could be an issue unless it's all in memory, and if it's all in memory, a RDBMS-type representation is overkill versus more simple pointer chasing in memory.
I guess it also depends upon latency / throughput needs (caching in memory on front of a database could help, but a pure in-memory vector index would be faster), persistence / fault tolerance / replication needs (in memory databases would need some kind of persistent store), or update needs (adding/removing vectors, though adding/removing vectors from an IVF or graph-based index becomes non-trivial if you add or remove too many vectors). Traditional databbases that have been around for a while might provide better guarantees on reliability.
(I'm the author of the GPU half of the Faiss library)
I don't see any fundamental reason why the index in Postgres would be slower than a specialized vector database. The query pattern of the vector database is simply a point query using an index, similar to other queries in an OLTP system.
The only limitation I see is scalability. It's not easy to make PostgreSQL distributed, but solutions like Citus exist, making it still possible.
(I'm the author of pgvecto.rs)
In this case, you get the robustness of Postgres (which has been around a while) and Faiss (which is one of the most mature vector indexes).
I’m curious about which use cases it enables.
I use txtai in my everyday work. While txtai is open source, I also support a number of consulting projects that are txtai focused. In most cases, I've used SQLite + Faiss for the vector database config paired with an LLM for RAG support.
This setup has worked well and it's what I would start with until it didn't scale.
Instead with DataStax's Vector Search, we designed a document style API and corresponding clients that give you a Vector Native experience to do CRUD of vector and meta-data as well CRUD of other data models. Here is one client ref for example https://docs.datastax.com/en/astra/astra-db-vector/clients/p...
This Faiss wiki article might help (https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v...).
For example, a partial Faiss configuration with 4-bit PQ quantization and only using 5% of the data to train an IVF index is shown below.
faiss={"components": "IVF,PQ384x4fs", "sample": 0.05}
I don't see this reflected in the comments here
1: https://m.youtube.com/watch?si=eUhNtghbiLRJ7yHB&v=MDxEXKkxf2...
EDIT>> Plus pgvector is enabled by default on AWS RDS databases. Huge bonus.
In this case, you get the robustness of Postgres (which has been around a while) and Faiss (which is one of the most mature vector indexes).
No they don't. Because they aren't trying to build general purpose databases.
And often trying to jam everything into one product just makes it worse not better.
I’ve worked in companies where we grabbed any tech we want, and I’ve worked in companies with one default DB unless you had an excellent reason. The latter meant a much simpler universe where entire teams knew how to fix things that broke at 2am. You just hire one type of DBA. It also meant dev teams really knew how to squeeze the one DB platform rather than learn a little about 10 database types.
«Good enough» is not a quantative metric by any conceivable measure. Furthermore, people in surrounding threads have reported inadequate performance of the pg_vector PgSQL extension (which, indubitably, can and will be fixed at some point).
An example of a useful quantative metric would: «compared to alternative vector processing/indexing engines/databases A, B and C and a dataset comprising N inputs and a total size of S Mb/Gb/Tb and using the M, N and P use cases in the performance testing scenario and pg_vector being the performance baseline, the following results have been obtained: X, Y and Z» – do you have anything similar to that you could share? Not only the comparison results would be insightful, they could also feed into someone's decision making process of whether X should be favoured over Y.
> […] and only going to get better.
Beyond doubt, it will. In the future. But people need to solve problems in the present.
txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. txtai can satisfy most vector database use cases such as being a knowledge source for retrieval augmented generation (RAG).
txtai is independently developed (not VC-backed) and released under an Apache 2.0 license.
I consider txtai to be a vector database and satisfy many of the vector database use cases.
A specialized vector database would be needed in just a few relatively rare use cases.
Regarding the dedicated solution, the difference is always in the details. That detail might significantly impact if you are looking for precision or working with large data. First of all, filtering the vector similarity search is quite tricky, and it does not work with pre and post-filtering because the graph's connectivity gets lost. That is why we introduced filterable HNSW at the very beginning. https://qdrant.tech/articles/filtrable-hnsw/ Here is another quite good explanation by James Briggs: https://www.pinecone.io/learn/vector-search-filtering/
Going further, it is not only about being able to search. It is also about being able to scale. Optionally, use disk for cold data as more affordable storage and keep hot data in expensive RAM. Or using one of the built-in compression functionalities. The recently introduced Binary Quantization makes it possible to compress vector embeddings up to 32 times and speed up the search up to 40 times in parallel. This makes billion-scale vector search affordable only for enterprise companies. https://qdrant.tech/articles/binary-quantization/
Native vector databases build all their features around vectors, and vectors are the first-class citizens in the architecture of the database core engine, not just another type of index support.
Vector search is not only about Text Search. See: https://qdrant.tech/articles/vector-similarity-beyond-search... It is also not even only about Search. Qdrant offers, for example, a dedicated Recommendation API, where you can submit negative and positive vector examples and get ready-made recommendation results back. https://qdrant.tech/articles/new-recommendation-api/ And the upcoming version of the engine will introduce even more functionalities like a new Discovery API, for example.
Ultimately, you do not need a vector database if you are looking for a simple vector search functionality. But you need one if you are looking to do more around it, and it is the central functionality of your application. It is just like using a multi-tool to make something quick or using a dedicated instrument highly optimized for the use case. Thank you for your attention.
Didn't read the rest but this looks like sound advice; same would go for graph DBs.
But I want to comment on another thing I often hear: "You don't need a vector database - just use Postgres or Numpy, etc". As someone who moved to Pinecone from a Numpy-based solution, I have to disagree.
Using a hosted vector database is straightforward. Get an API key from Pinecone, send them your vectors, and then query it with new vectors. It's fast, supports metadata filtering, and scales horizontally.
On the other hand, setting up pgvector is a hassle - especially since none of the Cloud vendors support it natively, and a Numpy-based solution, while great for a POC, quickly becomes a hassle when trying to append to it and scale it horizontally.
If you need a vector database, use a vector database. You won't regret it.
Adding vector support to our app was as easy as enabling the extension and created a table with vector columns. No additional database required and trivial to do mixed queries. Maybe Pinecone and friends have better scalability, but if you need basic vector support, you can do it easily on RDS.
See here for details:
https://aws.amazon.com/about-aws/whats-new/2023/10/amazon-rd...
I stand by my point: we often hear that adding a new tool to your stack is introducing unnecessary complexity, when more often not using the right tool is the path of most complexity.