Hacker News Reader

Comment by light_hue_1 | Hacker News Reader

That's not their moat at all. Most students don't have a capable gpu in their laptop. And these days everyone who is learning doesn't care about running on Nvidia, AMD, a cpu, etc. It's easy to move from one to the other (nominally).

The moat is their high quality libraries for speeding up common operations on Nvidia gpus.

AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.

View on HN

The students who are really interested in AI will buy desktop PC anyway. And the fact that they can use the same GPU for model training and for playing Diablo IV makes the whole investment easier to justify.

I dunno man, stuff like google collab is ubiquitous for students these days. I asked about building a pc on the AI/ML subs on reddit and people basically and the default advice was that few did that these days, and they're right, compute prices have been really low until recently even if you decided to go with premium offerings.

If you don't have a house with space that you expect to stay in for awhile, laptop+cloud is too compelling to make anything else worth it.

If you start having physical space guarantees, then the calculus changes...

Last time I did the math, by the time I spent 4090 money on compute, I could have been done with a PHD. Now, if all you need is a 3060? Yeah you could probably make that work, but you wouldn't be doing LLM inference even quantized

Either you only use the GPU sporadically or your math is very different from Tim Dettmers` [1]:

"The break-even point for a desktop vs a cloud instance at 15% utilization (you use the cloud instance 15% of time during the day), would be about 300 days ($2,311 vs $2,270):"

And that was written before the current GPU crunch and assumes availability in the cloud, which currently is not a given at all.

[1] https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

Thanks for the source! Very helpful for future reference.

Reply on news.ycombinator.com