Hacker News Reader

Comment by wongarsu | Hacker News Reader

That would be foolish. A large part of their moat is that lots of people already have Nvidia cards in their computers, making development really smooth and allowing hobbyists to easily get started and contribute to the ecosystem.

If they were to give up their consumer gpu lead and let AMD overtake them that will lead to more AMD support in ML, enabling AMD to eventually compete in the server gpu space too.

View on HN

That's not their moat at all. Most students don't have a capable gpu in their laptop. And these days everyone who is learning doesn't care about running on Nvidia, AMD, a cpu, etc. It's easy to move from one to the other (nominally).

The moat is their high quality libraries for speeding up common operations on Nvidia gpus.

AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.

The students who are really interested in AI will buy desktop PC anyway. And the fact that they can use the same GPU for model training and for playing Diablo IV makes the whole investment easier to justify.

I dunno man, stuff like google collab is ubiquitous for students these days. I asked about building a pc on the AI/ML subs on reddit and people basically and the default advice was that few did that these days, and they're right, compute prices have been really low until recently even if you decided to go with premium offerings.

If you don't have a house with space that you expect to stay in for awhile, laptop+cloud is too compelling to make anything else worth it.

If you start having physical space guarantees, then the calculus changes...

Last time I did the math, by the time I spent 4090 money on compute, I could have been done with a PHD. Now, if all you need is a 3060? Yeah you could probably make that work, but you wouldn't be doing LLM inference even quantized

Either you only use the GPU sporadically or your math is very different from Tim Dettmers` [1]:

"The break-even point for a desktop vs a cloud instance at 15% utilization (you use the cloud instance 15% of time during the day), would be about 300 days ($2,311 vs $2,270):"

And that was written before the current GPU crunch and assumes availability in the cloud, which currently is not a given at all.

[1] https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

Thanks for the source! Very helpful for future reference.

> allowing hobbyists to easily get started and contribute to the ecosystem

I think many, maybe even most, would claim this is now best done on cloud services, like huggingface.co.

I think that's still a minority. Everyone has heard of cloud platforms giving people huge surprise bills and other such limitations. Another factor is availability and potential T&S concerns when you're doing something they don't want you to do.

Stable Diffusion blew up when people are able to run it at home.

Nah, when you ask around on AI/ML subs, the default advice is to use google collab. Even shelling out for their premium offerings, you could pay for all the compute you would need through a PHD for the price of a 4090.

Reply on news.ycombinator.com