- As a managed service there are some overheads. We need to auth, validate and parse the inputs, fetch the index that is getting queried as we then need to use the index’s model to generate the embeddings. Then if the index is fine tuned/customized, we need to transform the embedding to the new vector space, to then call our vector index. We then fetch the metadata of the results from the db and parse the response to send it back.
- We’ve only tested upwards of 1M vectors ~ 1500 dimensions. But, more formal testing is required here and we plan to do so. I’m particularly curious about pg_vector and how it stacks up with other players as keeping the data central is a significant upside. We started with these lower vector indices to get something out there and iterate as a startup. But, scalability is part of what we want long term.
- We see both; we’ve had to turn down a very early lead with 100M+ vectors because it would derail other engineering efforts while we were starting. We’re now much better positioned to tackle that challenge as we have all the foundations.
- We haven’t considered this, but it’s an excellent idea. We’re currently discussing this with the team.
We would love to chat more; we appreciate your questions and feedback. Always happy to riff with someone who has seen issues around these use cases, like yourself. Feel free to reach us at founders@getmetal.io !
We have some open-source tooling in the works! :) We understand that some users are sensitive to managed services, we’re starting with this, but we’re planning to open source tools to improve developer experience around information retrieval and memory.
You’re right! If you want to do that in a notebook, it’s pretty straightforward. But if you want to have it running in production, it’s a bit more complicated. Also, providing users with a gui to run these operations without a notebook has resonated with many less ml savvy users. Dilbert's boss probably didn't know much about ml... :)
Hey! I appreciate the comment, and we would love to hear about other pains you've encountered. I can't find a way to DM on HN, but please email us at founders@getmetal.io, and we can connect there!
Redis provides indexes for vector similarity. And we have a lot of experience with Redis. We have plans to expand into offering other data stores, like Qdrant
Chroma is awesome <3 - We have some overlap with them as we store the embeddings. But, we provide additional operations on top of the data, such as clustering/fine-tuning. We're also looking into open-sourcing some tools in the near future!
It does compare with them, but we want to lower the barrier of entry for any developer to build features that use embeddings. So we want to give regular software engineers superpowers in providing this technology within their stack and out of the box offering the infrastructure and high-level APIs to run operations on top of the vector db.
We store the vectors, but we also provide additional operations that would require additional code/infra if you just use a vectorDB. We also have the infrastructure in place to ingest all the data, generate the embeddings (we also take raw embeddings), and provide APIs for fine-tuning and clustering. Another big difference coming soon is index versioning, allowing developers to test multiple models/embeddings.
Thank you! We’ve looked into instructor-xl, and it’s really awesome! We also accept custom embeddings, allowing developers to use whatever model they want. But we want to keep adding models to allow for better experimentation.
There’s some overlap with information retrieval for chat GPT applications. As a managed service, we handle all of the infrastructure and maintenance. Also, we support additional use cases for web applications/backends, such as clustering and fine-tuning. We’re also working on an open-source alternative to the retrieval plugin.
This is great, and that makes a ton of sense! Would you want to define + experiment with these various configurations yourself explicitly, or would you expect a system to determine this automatically? I like the concept of rolling-up chunk scores!
We don’t support this use case yet, but we could by exposing an API to update the non-filterable metadata of the records. This is a cool use case; we would love to learn more about it. Would you want to create embeddings from the product name + description and then have the other attributes returned from the search results? We are very close to supporting this; just a matter of exposing a way to update those attributes
We don’t offer this through the API, yet! You can however run clustering in the UI. We are working on exposing classification so that you can generate clusters on specific topics. We plan to offer both in the API within the next week or two!
Qdrant is awesome :). Redis also supports metadata filtering we’re currently building. We are considering adding a different data store option and Qdrant might be our next choice.
Redis provides indexes for vector similarity. And we have a lot of experience with Redis. We see a future where we can offer more than one datastore, and we’ve been considering Qdrant as the next datastore to support.
We agree with the sentiment; we’re currently figuring out the pieces we want to open source, as much of it is just infra (like the ingest pipeline). But the search server and some of our future work around memory will get open-sourced first.
We agree; this is precisely the problem area we’re focusing on!! We’re currently working on the ability for users to specify chunking strategies while providing a ton of guidance on this selection based on their particular data.
Great question; while it’s still super early, we believe that some of the most critical problems to solve will involve making current APIs compatible with AI use cases. Products like ChatGPT Plugins are game changers, but they will still be limited by the APIs they interact with.
We see ourselves a layer above vectorDB; we use Redis to index the data. We focused on building the ingest pipeline and operations on top of the embeddings, such as clustering and fine-tuning (embedding customization). Ultimately we want to provide the best developer experience possible, and we believe much work is needed here!
Yes, we do! We allow users to run `metal.tune` to determine whether two vectors should be close to each other. Then we use that to recalculate the embeddings similar to the customized embeddings cookbook from OpenAI. Then the queries get embedded and transformed into the same space.
Search is one use case we support, but you can perform a few other operations on your data, like clustering or fine-tuning. We're also working on a classification feature. Are there other async jobs you'd like to see?
Hey! We do support multiple versions of an Index under an App. When you fine-tune an embedding, we autogenerate the new embeddings for the entire dataset into a unique index. We store the raw data uploaded to our system via text or file imports. Although we don’t allow you to easily re-embed this data today, we have this on the roadmap!
Great post -- one of our core offerings at https://getmetal.io is to manage the process of utilizing Custom Embeddings very similar to your approach here! We'd love to connect and talk through some of the pain points that you came across -- i'll send you a message in the w23 slack!