Hacker News Reader

Comment by quickthrower2 | Hacker News Reader

Looking at the Mosaic website, and looking at many AI startups, I can't help feeling like they are all small AWS Sagemaker, Bedrock, Trainium, ... (etc.) competitors, and not sure how they will compete with the sheer capital that Amazon has, and potential to offer that compute power cheaper (at least long enough to kill off most of the competition). Maybe it is it the off chance that one of these companies might become a big player worth billions that makes the odds worth it?

View on HN

A lot of AWS services (especially SageMaker) aren’t very good in customer experience. People buy them for nominal capabilities and AWS core bread and butter — short-term and long-term reliability.

Most of these startups (AI and others) have to offer a compelling product before even being notable.

Besides, AWS top level doesn’t care if you use sagemaker or not. There’s a premium but if you’re still using EC2 via another startup, they’re still capturing lions share of value.

Sagemaker is probably a bad example since most folks who run ML workloads on AWS don't use it (from what I've seen talking to many ML teams). It's partially a scattered focus on their customer type (are they for experts/non-experts/something in between?, what exact use cases are they covering?) and partially I think just bad UI/UX (related to customer focus). AWS will converge on what is working for ML platforms and just build their own as time goes on. First mover advantage won't matter and these earlier ML platforms will be consumed imho

The devil is in the details. Training large LLMs requires a lot of custom infra (handling GPUs going down, efficiently pushing data to keep the accelators busy, deciding on which mechanism of parallelizing model training is better - data vs model parallelism or both, tuning hyperparams of optimizers which can be different for larger batch sizes, etc)

Mosaic is one of the better providers for this. AWS is nowhere near ready at this current point in time, it is pretty much a "dumb" infra provider in large LLM training at this point. (Of course they won't be standing still and will prob acquire that capability one way or another)

Reply on news.ycombinator.com