But
He has a history of being an arrogant prick. That will color people's perceptions of you even if it's not relevant to the immediate interaction.
He is a serial shit stirrer and this is no exception.
Did he try Windows?
As much as I hate that AMD is not really supporting consumer GPUs for compute, presenting his problems as some variant of production drivers breaking is a stretch at best.
[1]: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...
They need to be better than nvidia, not "you'll take what you can get." We can get nvidia. That's the bar. NVidia charges a premium but their shit works (comparatively speaking). AMD has been half-assing their compute offerings for 15 years, it finally became important, and now they need to play catch-up not drag their heels and toss out excuses as to why that's OK. My prediction: AMD won't be second place in this race for long, someone who actually wants the #2 spot will take it from them and AMD will sit around wondering how they blew a 15 year head start.
So far, with all that money, he has compiled their driver, ran it in an unsupported configuration, and then had a complete public mental breakdown because it didn't work. Something he already knew going in.
Should AMD support ROCm on its consumer grade GPUs? Probably. But that's really not geohot's choice to make, unless he wants to get his hands dirty and actually write the software he promised to write.
Having worked on GPU drivers, it's not a couple-lines fix situation, it would be a pretty big investment to add stable ROCm support for consumer GPUs. AMD higher-ups responding with anything other than "lol what it's unsupported what did you expect" is extending a pretty long olive branch here.
I am amazed that Geohot reached out to AMD to extend an olive branch so far as to force their failure of a product to work despite itself, because frankly I'd just have expected AMD to spin excuses like you did. I am encouraged to see that they took a higher road; hopefully that translates to actual execution. We'll see.
Having not followed this, I don't understand what the promise was. ROCm works just fine on at least some consumer cards like the 6xxx, and by fine I mean "as bad as on AMD's pro cards", but at least it works out of the box.
Certainly it's not supported, and so therefore they are not shipping precompiled binaries, but it does seem to work...
That said, I don't think people realize that there is literally no more large scale tier 3 (redundant) data center power in the US. Even if you are sitting on 1000 or 10k NVIDIA cards, you can't deploy them anywhere.
They also need to be in the same data center for speed. You can't just colo in 2-3 data centers to get what you want. If you want to train a large model across 1000 gpus? You're screwed.
Think you can just go to the cloud? Go try to sign up for coreweave. They are full and not taking any more customers. A lot of the other sites advertising nvidia gpus are just reselling coreweave under the covers.
Forget the software problems. There are far bigger issues and they are not getting better any time soon.
Google and Amazon offer their own hardware that is both better supported and actually available for customers. Apple has fast inference hardware in every computer and mobile device they offer.
I raised this issue with AMD 8 years ago in a technology conference, and the answer I got back then was a shoulder shrug "we don't think this is an important market." 8 years later and they have all but lost the war.
Apple is Apple. Not public. Let's also not mix consumer needs with enterprise.
You're correct, 8 years ago and up until recently, AMD only cared about gamers. They are waking up fast though.
ROCm 5.6 is a visible first step in that regard. MI300 will blow A/H100's out of the water.
But again, hardware/software isn't the problem here. The problem is much deeper than that... even if you have those things resolved, you can't put them anywhere.
Your shit isn’t so good that google is going to peek under the covers and steal your shit, because that would actually implode their business when they got caught doing it. The net present value of all of google’s future decades of operation is a lot higher than your hot dog detector app, or even critical F500 business operations.
NVidia mixed them and now everything is written in CUDA. Lol.
https://www.mosaicml.com/blog/amd-mi250
Waiting on the updates.
I'll add that I have learned over time to not discount motivation. If AMD it motivated, they can do it. This has been proven already with their dominance over the server cpu market.
That said, ROCm does indeed work with machine code instead of IR. You can compile fat binaries with more than one type of machine code and they'll work on any of the chips you compiled for, at the cost of the binaries becoming, well, basically obese if you want a decent range of hardware supported.
And yet, that's what a lot of the big AI startups are doing. Granted, it's not what everyday business are doing (yet). But TPUs offer pretty impressive perf/cost ratio, so I'd be surprised if it's actually "nobody".
They don't have any other choice or they are just dumb...
Real evidence that Google or Amazon actually introspected the contents of their cloud platform customer’s VMs, Databases, GPUs, disks, blob storage buckets etc. would be far more convincing, but such evidence doesn’t exist - because it doesn’t happen.
This is interesting to me, what is the constraining factor? Raw generator output? Transmission lines getting power to the right places?
All the large FAANGs have been sucking up availability.
Keep in mind this post is around two months old now, and since it was published AMD have already officially announced plans to support (at least some) consumer GPU's in ROCm.
George being a prick one time = suddenly working driver appears?
My perception is indeed colored but in the opposite direction.
Should someone who will just drop a GPU vendor if they get flustered really be leading a ML framework?
His post is mostly hot air. AMD driver development is very open, excepting only ROCm. Not knowing where things are discussed or how the various driver systems, excluding Nvidia, interact on Linux is not an excuse to rant.
The corporate amd kernel development starts amd-staging-drm-next, with internal AMD pull requests available on the mailing list or here: https://patchwork.freedesktop.org/project/amd-xorg-ddx/serie..., before going through airlied, the maintainer of drm in the main kernel, then to Linus.
Everything user-space regarding OpenGL/Vulkan and some rusty OpenCL, aside from the proprietary amdgpu-pro driver which should almost never be used, is in Mesa.
ROCm is the only thing with huge code dumps, obviously because it's a new effort, and it's said outright that it's unsupported for consumer GPUs. Yes, it's a buggy mess. Did his issue warrant the corporate response he got? No.
But yes, this one sits squarely in “nobody involved looks real good” territory. AMD puts out shit software in general, and ROCm is exceptionally far behind the standard in this field, and often difficult to set up and broken even in “supported” configurations. And the ROCm team have a habit of doing this closed-door development, splitting the actual work into the AMDGPU-PRO driver while neglecting the open one (even in “supported” configurations), and generally just having a process that is not compatible with delivering high-quality open software.
And Hotz is being a manchild who melts down about a driver he knew was bad and berates the people who are trying to get him rush patches while they do their dayjob getting the ROCm release out (which hopefully fixes a lot of the complaints around lack of windows support, lack of consumer gpu support, and general (near-total) lack of polish and stability.
To their credit AMD finally posted a job listing for like 30 engineers for ROCm. It’s just something they should have done like 4 years ago when ROCm first got rolling. And we are well past the “poor lil amd couldn’t afford that” in 2019 or whatever. If you can’t afford it then don’t start the ROCm initiative and make the promises.
Even 2-3 years ago it was clear that the foundation work was done and that if you ever wanted it to go bigger you’d have to do some hiring. And frankly it’s been just as bad with OpenCL before it - support was never good, the runtime constantly had bugs and paper features, etc.
AMD didn’t care about GPGPU compute until AI turned into a billion-dollar industry overnight. And that’s a 15-year story there.
I remember the HSA push too...
Calling that out publicly is probably needed at this point, or they will keep putting AI in their earnings reports without anyone actually being able to train a model on an AMD chip.