> If you have to sell the hardware and support people buying it, that is a world of trouble.
What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription?
What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription?
Margins.
Pricing for cloud compute is much higher and servicing and management for the provider is much cheaper.
If I sold hardware directly, then I'm often on the hook for support contracts which can get pricy with hardware and distract from shipping future facing product features, as customers who purchase directly have longer upgrade windows due to logistical overhead.
It also isn't just the hardware capex, it is everything involved under the covers. Market pricing also factors into that as well. This is something I've struggled with myself quite a bit. I know all of my costs and what I'd like to charge, but because my offering is so brand new, until my competitors announced their pricing, I wasn't sure what the market would tolerate.
Selling hardware directly is hard for exactly what you state though. Service contracts are a pain in the butt. All of this latest AI hardware has high rates of failures too. Up until recently with AMD coming to market with a great offering, the only thing people want any sort of quantity on are nvidia products. Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable.
Yep! You explained it better than me!
> Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable
Yep! And if would have bogged them down by slowing down R&D cycles and even fulfilling orders as they obviously are not placing orders the same size as Nvidia or AMD.
I wonder what this portends for SambaNova and other similar vendors as well.
Time will tell. This is definitely an interesting development.
They're focused on services based full stack deployments afaik.
They come in with a rack and sell you on models as well.
Knowledge/training.
If you're shipping a brand new hardware arch, exposed as raw hardware, then you're on the hook for training everyone in the world and fixing all their weird edge case uses.
I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support?
If you're exposing a PaaS (or even IaaS), then you have some levers you can tweak / mask behind the scenes, so only your team need be experts at low-level operations.
For a fast-paced company, the latter model makes a lot more sense, at least until hardware+software stabilizes.
At least in our experience, the first line of support is from the chassis vendors. You don't go to a store and buy MI300x. You buy them from someone like SMCI/Dell, who provides the support. Of course, behind the scenes, they might be talking to AMD. Even those chassis companies often have other providers of their gear (like Exxact) as another line of defense as well.
In the case of Groq, it would have been death by 1000 cuts to have to support end users directly, especially if they are selling small quantities. It is much easier to just build data centers full of gear, maintain it yourself and then just rent the time on the hardware.
Good thing that I'm a glutton for punishment.