Show HN: Chroma Cloud – serverless search database for AI | Hacker News Reader

Show HN: Chroma Cloud – serverless search database for AI

trychroma.com

93 pointsjeffchuber40 comments

Hey HN - I’m Jeff, co-founder of Chroma.

In December of 2022, I was scrolling Twitter in the wee-hours of the morning holding my then-newborn daughter. ChatGPT had launched, and we were all figuring out what this technology was and how to make it useful. Developers were using retrieval to bring their data to the models - and so I DM’d every person who had tweeted about “embeddings” in the entire month of December. (it was only 120 people!) I saw then how AI was going to need to search to all the world’s information to build useful and reliable applications.

Anton Troynikov and I started Chroma with the beliefs that:

1. AI-based systems were way too difficult to productionize

2. Latent space was incredibly important to improving AI-based systems (no one understood this at the time)

On Valentines Day 2023, we launched first version of Chroma and it immediately took off. Chroma made retrieval just work. Chroma is now a large open-source project with 21k+ stars and 5M monthly downloads, used at companies like Apple, Amazon, Salesforce, and Microsoft.

Today we’re excited to launch Chroma Cloud - our fully-managed offering backed by an Apache 2.0 serverless database called Chroma Distributed. Chroma Distributed is written in Rust and uses object-storage for extreme scalability and reliability. Chroma Cloud is fast and cheap. Leading AI companies such as Factory, Weights & Biases, Propel, and Foam already use Chroma Cloud in production to power their agents. It brings the “it just works” developer experience developers have come to know Chroma for - to the Cloud.

Try it out and let me know what you think!

— Jeff

Open articleView on HN

Happy paying user of Chroma Cloud here. Congratulations Jeff, Hammad and team!

We were lucky to get early access and have saved a bundle.

Chroma looks cool. Congratulations on the Cloud version.

For my client I've "built" a similar setup with Supabase + pgVector and I give the AI direct SQL access.

Here is the hard part: Just last week did I index 1.2 million documents for one project of one customer. They have pdfs with 1600 pages or PPTX files of >4GB. Plus lots of 3D/2D architecture drawings in proprietary formats.

The difficulty I see is - getting the data in ETL. This takes days and is fragile - keep RBAC - Supabase/pgVector needs lots of resources when adding new rows to the index -> wish the resources scale up/down automatically. Instead of having to monitor and switch to the next plan.

How could chroma help me here?

> Supabase/pgVector needs lots of resources when adding new rows to the index -> wish the resources scale up/down automatically. Instead of having to monitor and switch to the next plan.

Many ways potentially - but one way is Chroma makes all this pain go away.

We're also working on some ingestion tooling that will make it so you don't have to scale, manage or run those pipelines.

I'll for sure take a deeper look. Ingestion has been by far the biggest pain and least fun. Those infra parts hold us back from the cool things -> building agents/search

Im using LanceDB (and pretty happy with it so far). I am looking at the chroma docs and have a few questions

1. I see the core is OSS, any chance of it being pushed up on crates.io(i see you already have a placeholder)

2. Is it embeddable or only as a Axum server?

Do you see all providers converging on similar alpha i.e cheap object storage, nvme drives,ssd cache to solve this?

Cheers and congrats on the launch

Hey there

Chroma is fully OSS - embedded, single-node and distributed (data and control plane). afaik lance distributed is not OSS.

We do have plans to release the crate (enabling embedded chroma in rust) - but haven't gotten around to it yet. Hopefully soon!

> Do you see all providers converging on similar alpha i.e cheap object storage, nvme drives,ssd cache to solve this?

It's not only a new pattern in search workloads, but it's happening in streaming, KV, OLTP, OLAP, etc. Yea - it's the future.

I appreciate the straightforward pricing calculator and the pricing seems very reasonable.

thank you!

Can Chroma handle combined structured+vector search? E.g. filtering by `category=X and value>Y` then finding top N matches by vec similarity?

Yes! It can - https://docs.trychroma.com/docs/querying-collections/metadat...

I run a fractional jobs site. Lots of non-technical customers are interested in ai. What problem does Chroma solve for these non-technical users?

your customers might have heard of RAG (retrieval augmented generation) before. chroma powers the “R” in RAG. It enables language models to dynamically pull in information to help them answer questions and solve tasks.

Not sure why this is voted down. The site doesn't really explain what problems of mine it'll solve (if any). I just came away thinking "oh, it's a thing. might have been a useful thing, who knows"

"Sell the sizzle, not the steak" is a real thing for a reason.

That is helpful feedback, thank you. We'll address this.

High-level, Chroma lets you incorporate your own data into AI. That can be business data, like your help docs, or customer data, such as private documents.

I think there's a misconception among many non-technical people that they should "fine tune" models - to make an LLM sound like their brand or know their internal data. But, the best practice today is "context engineering" - to use a commodity LLM, and add propriety information into the prompt.

The hard part of context engineering is knowing which data to incorporate. For example, all help docs in every context creates context rot [1] that hurts accuracy, so you need a way to find which information to put in which context. The solution is essentially an internal search engine, which is Chroma.

Chroma gives AI multiple ways to search, and you can either hard-code which one works best for you - or let an agent write its own queries to conduct research by itself. Vector search is a generation ahead of old search infrastructure, and is useful for relatedness queries - like "help docs about billing". Full-text search is useful for proper nouns, like "Next.js". Regex search is useful for code and laws. Metadata search is more nuanced, but becomes really important in document search (e.g., PDFs). Chroma lets you run all of these search methods against private data, and you can use it to even include citations in results.

So, the high-level answer is: Chroma enables you to incorporate your business or customer data into AI.

[1] https://research.trychroma.com/context-rot

Doesn't this does seem like a bit of an... exact rip off of turbopuffer?

Down to the style of the webpages and the details of the pricing?

Here's the pricing calculator, for example:

https://share.cleanshot.com/JddPvNj3 https://share.cleanshot.com/9zqx5ypp

As a happy turbopuffer user, not sure why I'd want to use Chroma.

Hi! Hammad here - Chroma’s CTO. First off, I have an immense amount of respect for the Turbopuffer team, they’ve build a solid product.

I understand your point. Chroma Cloud has been quietly live in production for a year, and we have been discussing this architecture publicly for almost two years now. You can see this talk I gave at the CMU databases group - https://youtu.be/E4ot5d79jdA?si=i64ouoyFMevEgm3U. Some details have changed since then. But the core ideas remain the same.

The business model similarities mostly fall out of our architecture being similar, which mostly falls out of our constraints with respect to the workload being the same. There are only so many ways you can deliver a usage based billing model that is fair, understandable, and predictable. We aimed for a billing model that was all three, and this is what we arrived at.

On aesthetics, that’s always been our aesthetic, I think a lot of developer tools are leaning into the nostalgia of the early PC boom during this AI boom (fun fact, all the icons on our homepage are done by hand!).

On differences, we support optimized regexes vs full-scans, lending better performance. We also support trigram based full-text search which can often be useful for scenarios which need substring matches. We also support forking, which allows for cheap copy-on-write clones of your data, great for dataset versioning and tracking git repos with minimal cost. We've been building with support for generic sparse vectors (in beta) which enables techniques like SPLADE to be used, rather than just BM25. You can also run Chroma locally, enabling low-latency local workflows. This is great for AI apps where you need to iterate on a dataset until it passes evals, and then push it up to the cloud.

Chroma is Apache 2.0 open source - https://github.com/chroma-core/chroma and has a massive developer community behind it. Customers can run embedded, single-node and distributed Chroma themselves. We've suffered from depending on closed-source database startups and wanted to give developers using Chroma confidence in the longevity of their choice.

Lastly, we are building with AI workloads front and center and this changes what you build, how you build it and who you build for in the long term. We think search is changing and that the primary consumer of the search API for AI applications is shifting from human engineers, to language models. We are building some exciting things in this direction, more on that soon.

didn't the ceo of turbopuffer work for you? doesn't have to bias you, but probably worth calling out?

I still remember when the website said Chroma Cloud was coming end of 2023. What took so long?

very fair!

cloud has been in private beta for a year now.

we chose to not release it to the public until we were extremely confident in the system and its characteristics.

databases are a serious business. developers trust us with their mission critical data.

I haven't heard of Chroma before (sorry about that), but how does it compare to Weaviate?

no worries!

Chroma and Weaviate are both popular OSS projects, 21k and 14k stars respectively.

Weaviate has some features that Chroma does not (yet) - notably vector quantization. Weaviate is a single-node DB that can be run in cluster mode.

Chroma focuses on reliability and operational simplicity. The fully distributed architecture means there are no knobs to tune or config to set.

There's no right tool for everyone. Try both!

I'm excited to see Chroma be fully available! They're already differentiated from Turbopuffer/S3-vector with regex support and I think they will only get more so with sparse vector support and some kind of API surface which enables LTR.

what place do vector-native databases have in 2025? I feel using pgvector or redisearch works well and most setups will probably be using postgres or redis anyway.

if you want or need to optimize for speed, cost, scalability or accuracy.

dedicated solutions have more advanced search features enable more accurate results. search indexing is resource intensive and can contend for resources with postgres/redis. the cost and speed benefits are naturally more pronounced as data volume scales.

for example - chroma has built in regex+trigram search and copy-on-write forking of indexes. this feature combo is killer for the code-search use case.

Philip here from the Chroma engineering team.

Chroma supports multiple search methods - including vector, full-text, and regex search.

Four quick ways Chroma is different than pgvector: Better indexes, sharding, scaling, and object storage.

Chroma uses SPANN (Scalable Approximate Nearest Neighbor) and SPFresh (a freshness-aware ANN index). These are specialized algorithms not present in pgvector. [1].

The core issue with scaling vector database indexes is that they don't handle `WHERE` clauses efficiently like SQL. In SQL you can ask "select * from posts where organization_id=7" and the b-tree gives good performance. But, with vector databases - as the index size grows, not only does it get slower - it gets less accurate. Combining filtering with large indexes results in poor performance and accuracy.

The solution is to have many small indexes, which Chroma calls "Collections". So, instead of having all user data in one table - you shard across collections, which improves performance and accuracy.

The third issue with using SQL for vectors is that the vectors quickly become a scaling constraint for the database. Writes become slow due to consistency, disk becomes a majority vector indexes, and CPU becomes clogged by re-computing indexes constantly. I've been there and ultimately it hurts overall application performance for end-users. The solution for Chroma Cloud is a distributed system - which allows strong consistency, high-throughput of writes, and low-latency reads.

Finally, Chroma is built on object storage - vectors are stored on AWS S3. This allows cold + warm storage tiers, so that you can have minimal storage costs for cold data. This "scale to zero" property is especially important for multi-tenant applications that need to retain data for inactive users.

[1] https://www.youtube.com/watch?v=1QdwYWd3S1g

I wonder how chroma collections compares to using Postgres partitioning. I haven't done this personally, but you should theoretically be able to add a `PARTITION BY collection_name` to have the same effect as sharding between chroma collections.

object storage here do you mean the recent released S3 Vector?

We use S3, but not S3 Vector.

Best of luck. Do you guys have anything for startups?

Philip here from the Chroma team.

I see your email in your profile, so I'll reach out.

We work closely with any startup that wants help, ranging from whiteboarding architectures to a shared slack channel.

we want to create vector search for people — what's the best way to use Chroma Cloud for that?

can you tell me more about your use case?

Why does it say open source but then try to charge me money

Chroma's open-source under an Apache 2.0 License: https://github.com/chroma-core/chroma

Single-node Chroma is very easy to self-host for free: https://docs.trychroma.com/guides/deploy/docker

Typical FOSS model to offer a super useful core then charge for ancillary services and consulting to pay for development of the core.

It's a great model for people who want to have grasp beyond the typical software reach.

“Free as in freedom, not as in beer.”

https://en.m.wikipedia.org/wiki/Gratis_versus_libre

What about comparison with Qdrant?

Qdrant team is also smart!

The main reason we have seen people switch from Qdrant to Chroma is for operational simplicity and reliability. With Qdrant you have to size nodes, manage sharding, be oncall, etc - with Chroma there is literally 0 config or operational pain. Search, that just works!

Hack

Reply on news.ycombinator.com