I wonder how this will impact consumer gpus? If nvidia can print money making h100's you have to wonder if they will continue to invest on the retail side or if they'll funnel most of their efforts to server and let desktop parts languish.
If they were to give up their consumer gpu lead and let AMD overtake them that will lead to more AMD support in ML, enabling AMD to eventually compete in the server gpu space too.
I think many, maybe even most, would claim this is now best done on cloud services, like huggingface.co.
Stable Diffusion blew up when people are able to run it at home.
The moat is their high quality libraries for speeding up common operations on Nvidia gpus.
AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.
If you start having physical space guarantees, then the calculus changes...
"The break-even point for a desktop vs a cloud instance at 15% utilization (you use the cloud instance 15% of time during the day), would be about 300 days ($2,311 vs $2,270):"
And that was written before the current GPU crunch and assumes availability in the cloud, which currently is not a given at all.
[1] https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
In Feb they artifically constrained 4090 volume to try and push 4080s no one wanted. The 4070 was dead on arrival and had production cut by April.
They've near stopped production now - theoretically because the AI stuff is so lucrative, but if they were really "shipping a ton of units" of the 40 series and didn't have excess inventory i doubt they'd have done that.
- https://www.dexerto.com/tech/nvidia-rtx-4070-production-repo... - https://www.dexerto.com/tech/nvidia-rtx-40-series-production...
Rarely does Nvidia sell a GPU for 2000 dollars or more, and when they do its relatively low volume.
An H100 sells for tens of thousands of dollars and is sold out to 2024 - companies are literally fighting over them - I can absolutely see conditions might arise Nvidia might want to use some fab capacity for their much more profitable H100 parts vs consumer GPUs.
It's good they are selling shovels while there's a gold rush, but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well.
I mean, we can just go ahead say "Intel" instead of beating around the bush.
They may end up in a position similar to US Steel or Standard Oil at the start of the 20th century if they continue to meet market demand as they do now, and AI is goes on to have the economic impact that steel and oil had in the last century.