Hacker News Reader

Comment by whalesalad | Hacker News Reader

Everyone has been writing cuda code. Hopefully we continue the shift towards using higher level frameworks/abstractions that can determine the optimum machine code for the underlying hardware. I don’t really get it either.

Nvidia is pouring $$$ into protecting their advantage as anemic as it might be. They give free compute and gpu resources to universities etc to cement the dependence on cuda. When you’re a beginner or someone who needs to just “get shit done” you’ll go nvidia at the expense of the greater good.

View on HN

What the hell is the greater good here? I'm a researcher in a R1 university, I can quickly try stupid things on my personal 3080, then if it works I can virtually git clone the codes onto my lab's server and bam it chops through the problems. After a few weeks I'll get (hopefully) some publisable results that (hopefully) advance my field a little. Do you really expect most researchers to use half of their time debugging the unfinished ROCM stack that is just not there, sacrificing their output, and maybe even their tenure prospect? We love Nvidia solutions, at least in our field, not because it's a monopoly, but because it's actually the best solution right now.

I really like that you can prototype on your own consumer level card - the rocm stack has not matched that historically and has over promised and underdelivered for years now.

It's getting closer, and well, what matters is price points and what, billions of people crunching away on the various problems.

There's huge momentum on the "user" side of the equation that just getting cheap hardware and passable performance means Nvidia losing dominance.

>Nvidia is pouring $$$ into protecting their advantage as anemic as it might be.

This description isn't fair. Nvidia was pouring money into CUDA for 16 years. AMD/Intel only woke up once ChatGPT launched and nvidia started printing money. It's not fair to call it "protecting their advantage" when nvidia invested into the ecosystem for years and AMD couldn't even bother to ship working examples with OpenCL. I'm not sure why choosing the company that decided to invest in the platform for years is at the expense of the "greater good." What "greater good" is there above actually working software?

Nvidia had boatloads of money to spend, and 6X the number of engineers. AMD was skating on the edge of bankruptcy. AMD bet the farm on Zen, and that bet paid off so now they have money to spend on other things.

If they would have split resources off of Zen to spend on CUDA they would have failed at both and AMD would be a bankrupt husk.

You can get a consumer laptop with AI: https://www.amd.com/en/products/apu/amd-ryzen-7-7840hs

The thing AMD doesn't have is the software stack.

Also, the full consumer (RTX) stack is supported. Lots of people who are gamers, or want a hobbyist rig can go out to the shops and start with a RTX.

AMD's ML / ROCm support for their consumer cards is still awful.

They did add official support for the latest gen rdna cards to rocm.

Not on Linux!

What? It was only for Linux.

https://rocm.docs.amd.com/projects/radeon/en/latest/index.ht...

Why only the latest gen?

Because they want to recover their investment

Actually because ROCm ships native machine code, as opposed to a bytecode, so supporting any additional architecture is a tonne of effort.

I expect support will eventually be extended to a generation or two prior GPUs, but nothing beyond that.

> They give free compute and gpu resources to universities etc to cement the dependence on cuda.

It's actually quite brilliant. It never occurred to me before that the resources we had access to at university weren't the most popular because that's what the library wanted. They were the most popular because the provider made them cheap/free to widen their moat over the competition.

It reminds me of Thomson Reuters launching Eikon, a superior alternative (imho) to Bloomberg terminals. Every business school has Bloomberg terminals. But few/none had Eikon, which is why nobody knows how to use one.

It's the same reason that students get free MSDN access and unlimited windows keys.

It's a pretty solid strategy, but naturally favors the current incumbent who has the $$$ to throw around.

Try Hidet - open source deep learning compiler written in Python, generating optimized CUDA https://pypi.org/project/hidet/

The obvious answer is to just let the LLM boostrap use better abstractions.

Reply on news.ycombinator.com