We see ourselves a layer above vectorDB; we use Redis to index the data. We focused on building the ingest pipeline and operations on top of the embeddings, such as clustering and fine-tuning (embedding customization). Ultimately we want to provide the best developer experience possible, and we believe much work is needed here!
haha. That case you might actually wanna consider FAISS/Milvus instead of Redis.
We’ve looked into FAISS and Milvus. Milvus is possibly an excellent option for us in the future. What’s your experience with these so far?
hnswlib? Best of the bunch imho
Great to hear that you're considering Milvus. Feel free to reach out if you ever have any questions/comments/concerns.
Just took a look at your docs and product page as well. Keep up the great work!
I tried out milvus. Developer Experience is crap. Documentation lacks some major core concepts. I've been experimenting with it for hours. Eventually I turned my back and said: Why not use pg_vector and scale the fuck out of the cluster? That should bring.. equal performance, as the pg_vector implementation is written in c and the comparing algorithms wouldn't differ too much from milvus.
I think those assume you already have the embedding vector calculated, and they just store and retrieve the vectors.