Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

I'm curious why you would say this. The main use cases for LLMs involve things like customer service chatbots, knowledgebase search, document summarization, co-pilot/code generation, and content generation for product descriptions and marketing emails. The main data sources would be things like document repositories on Sharepoint and transcripts of customer calls. Not a bunch of historical sales data or financial data sitting in the typical data warehouse. I think there's a big misconception about this. Data sources for Generative AI are very different than data sources for 2013 era data science projects (which perhaps not coincidentally is what led to the development of things like Databricks).

While Databricks definitely started with Spark, and it’s still a significant foundation, there’s done much more on top and around it. For instance:

- MLFlow for ML lifecycle management from experiments to real-time ML serving;

- Delta Storage format, extending parquet to leverage cloud storage and enable efficient updates and very fast access;

- SQL Warehouses, which expose Databricks as a SQL engine for Analytics;

- Jobs/Workflows, which is one of the most used orchestration engines in the world;

- Unity Catalog, which will replace (at least in Databricks) Hive Metastore for metadata, access control, lineage and data governance tooling in general.

And now LLMs, on top of the data and ML capabilities mentioned above, much extended by the Mosaic deal (still to be approved).

The interesting thing is that yes, Databricks can do the whole “Data Warehousing” thing, but it can also do very large scale streaming, machine learning, process unstructured data like text, audio or images, support BI applications, etc - all accessing the same data with compatible tooling. So, it’s a full blown multi-workload data platform for any kind of use case and company size.

One can argue that most components are open-source and can be deployed independently - and Databricks has open-sourced Spark, MLFlow and Delta. It’s just that most companies simply don’t have enough (if any) staff with skills to deploy and operate all these things, let alone as one integrated platform. With Databricks, I’m used to deliver a demo where it takes me about 20 minutes from having a new cloud account to be running data workloads against a cluster or SQL, with all the functionality above.

As such, they’ll be forced to do a lot more services oriented work rather than product / platform oriented work, because it pays well. Their sales team is also excellent. I see a similar fate for them as Cloudera.

It may surprise HN to learn, but most companies don't have top-tier technical talent.

Consequently, how do you do {cutting edge thing board and investors are demanding company do} with the people you have/can afford? Use an offering that decreases the necessary skill level by providing powerful prebuilt components.

From this perspective, LLM integration sounds like a perfect fit. It's something every company is being asked to have a plan on, but one few are technically staffed to execute on their own.

Databrick cost is stupid high, like snowflake. Any companies looking to chop budget would put those contracts up first.

Most of these startups (AI and others) have to offer a compelling product before even being notable.

Besides, AWS top level doesn’t care if you use sagemaker or not. There’s a premium but if you’re still using EC2 via another startup, they’re still capturing lions share of value.

Mosaic is one of the better providers for this. AWS is nowhere near ready at this current point in time, it is pretty much a "dumb" infra provider in large LLM training at this point. (Of course they won't be standing still and will prob acquire that capability one way or another)

I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.

However, a year later, their SaaS has been discontinued. The open-source repository is now stagnant, with hundreds of unresolved merge requests. On a positive note, they recently shared the repository with some other open-source maintainers, so there's hope that Redash will be reborn.