I really hope this breaks Nvidia's stranglehold on deep learning. Some competition would hopefully bring down prices at the compute high-end.
AMD don't seem to even be trying on the software-side at the moment. ROCm is a mess.
AMD don't seem to even be trying on the software-side at the moment. ROCm is a mess.
With modern tooling.
Instead of forcing devs to live in the pre-historic days of C dialects and printf debugging, provide polyglot IDEs with graphical debugging tools capable of single step GPU shaders and a rich libraries ecosystem.
Khronos got the message too late and now no one cares.
CUDA wiped out OpenCL, because it went polyglot as of version 3.0, while insisting that everyone should write in a C dialect.
They also provide great graphical debugging tools and libraries.
Khronos waited too long to introduce SPIR, and in traditional Khronos fashion, waited for the partners to provide the tooling.
One could blame NVidia, but it isn't as the competition has done a better job.
https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a...
The issue is that AMD has split their compute into two categories:
* RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.)
* CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers.
------------
ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs.
Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days.
Still handicaps them compared to Nvidia where you can just buy anything recent and expect it to work. Suspect it also means they get virtually no open source contributions from the community because nobody can run or test it on personal hardware.
Each assembly language from each generation of cards changes. PTX recompiles the "pseudo-assembly" instructions into the new assembly code each generation.
---------
AMD has no such technology. When AMD's assembly language changes (ex: from Vega into RDNA), its a big compiler change. AMD managed to keep the ISA mostly compatible from 7xxx GCN 1.0 series in the late 00s all the way to Vega 7nm in the late 10s... but RDNA's ISA change was pretty massive.
I think its only natural that RDNA was going to have compiler issues.
---------
AMD focused on Vulkan / DirectX support for its RDNA cards, while its compute team focused on continuing "CDNA" (which won large supercomputer contracts). So that's just how the business ended up.
This makes absolutely no sense to me, and I have a Ph.D:
"* RDNA -- consumer cards. A new ISA with new compilers / everything. I don't think its reasonable to expect AMD's compilers to work on RDNA, when such large changes have been made to the architecture. (32-wide instead of 64-wide. 1024 registers. Etc. etc.) * CDNA -- based off of Vega's ISA. Despite being "legacy ISA", its pretty modern in terms of capabilities. MI100 is competitive against the A100. CDNA is likely going to run Frontier and El Capitan supercomputers. ROCm focused on CDNA. They've had compilers emit RDNA code, but its not "official" and still buggy. But if you went for CDNA, that HIP / ROCm stuff works enough for the Oak Ridge National Labs. Yeah, CDNA is expensive ($5k for MI50 / Radeon VII, and $9k for MI100). But that's the price of full-speed scientific-oriented double-precision floating point GPUs these days.
I neither know nor care what RDNA, CDNA, A100, MI50, Radeon VII, MI100, or all the other AMD acronyms are. Yes, I could figure it out, but I want plug-and-play, stability, and backwards-compatibility. I ran into a whole different minefield with AMD. I'd need to run old ROCm, downgrade my kernel, and use a different card to drive monitors than for ROCm. It was a mess.
NVidia gave me plug-and-play. I bought a random NVidia card with the highest "compute level," and was confident everything would work. It does. I'm happy.
Intel has historically had great open source drivers, and if it give better plug-and-play and open source, I'll buy Intel next time. I'm skeptical, though. The past few year, Intel has a hard time tying their own shoelaces. I can't imagine this will be different.
Its right there in the ROCm introduction.
https://github.com/RadeonOpenCompute/ROCm#Hardware-and-Softw...
> ROCm officially supports AMD GPUs that use following chips:
> GFX9 GPUs
> "Vega 10" chips, such as on the AMD Radeon RX Vega 64 and Radeon Instinct MI25
> "Vega 7nm" chips, such as on the Radeon Instinct MI50, Radeon Instinct MI60 or AMD Radeon VII, Radeon Pro VII
> CDNA GPUs
> MI100 chips such as on the AMD Instinct™ MI100
--------
The documentation of ROCm is pretty clear that it works on a limited range of hardware, with "unofficial" support at best on other sets of hardware.
(1) There are a million different ROCm pages and introductions
(2) Even that page is out-of-date, and e.g. claims unofficial support for "GFX8 GPUs: Polaris 11 chips, such as on the AMD Radeon RX 570 and Radeon Pro WX 4100," although those were randomly disabled after ROCm 3.5.1.
... if you have a Ph.D in AMD productology, you might be able to figure it out. If it's merely in computer science, math, or engineering, you're SOL.
There are now unofficial guides to downgrading to 3.5.1, only 3.5.1 doesn't work with many modern frameworks, and you land in a version incompatibility mess.
These aren't old cards either.
Half-decent engineer time is worth $350/hour, all in (benefits, overhead, etc.). Once you've spent a week futzing with AMD's mess, you're behind by the cost of ten NVidia A4000 cards which Just Work.
As a footnote, I suspect in the long term, small purchases will be worth more than the supercomputing megacontracts. GPGPU is wildly underutilized right now. That's mostly a gap of software, standards, and support. If we can get that right, every computer have many teraflops of computing power, even for stupid video chat filters and whatnot.
It seems pretty simple to me if we're talking about compute. The MI-cards are AMD's line of compute GPUs. Buy an MI-card if you want to use ROCm with full support. That's MI25, MI50, or MI100.
> As a footnote, I suspect in the long term, small purchases will be worth more than the supercomputing megacontracts. GPGPU is wildly underutilized right now. That's mostly a gap of software, standards, and support. If we can get that right, every computer have many teraflops of computing power, even for stupid video chat filters and whatnot.
I think you're right, but the #1 use of these devices is running video games (aka: DirectX and Vulkan). Compute capabilities are quite secondary at the moment.
For tasks which require that much GPU, I'm using cloud machines. My dev desktop would like a working GPU 24/7, but it doesn't need to be nearly that big.
If I had my druthers, I would have bought an NVidia 3050, since it has adequate compute, and will run once available <$300. Of course, anything from the NVidia consumer line is impossible to buy right now, except at scalper prices.
I just did a web search. The only card from that series I can find for sale, new, was the MI100, which runs $13k. The MI50 doesn't exist, and the MI25 can only be bought used on eBay. Corporate won't do eBay. Even the MI100 would require an exception, since it's an unauthorized vendor (Amazon doesn't have it).
Combine that with poor software support, and an unknown EOL, and it's a pretty bad deal.
> I think you're right, but the #1 use of these devices is running video games (aka: DirectX and Vulkan). Compute capabilities are quite secondary at the moment.
Companies should maximize shareholder value. Right now:
- NVidia is building an insurmountable moat. I already bought an NVidia card, and our software already has CUDA dependencies. I started with ROCm. I dropped it. I'm building developer tools, and if they pick up, they'll carry a lot of people.
- It will be years before I'm interested in trying ROCm again. I was oversold, and AMD underdelivered.
- Broad adoption is limited by lack of standards and mature software.
It's fine to day compute capabilities are secondary right now, but I think that will limit AMD in the long term. And I think lack of standards is to NVidia's advantage right now, but it will hinder long-term adoption.
If I were NVidia, I'd make:
- A reference CUDA open-source implementation which makes CUDA coda 100% compatible with Xe and Radeon
- License it under GPL with a CLA, so any Intel and AMD enhancements are open and flow back
- Have nominal optimizations in the open-source reference implementation, while keeping the high-performance proprietary optimizations NVidia proprietary (and only for NVidia GPUs)
This would encourage broad adoption of GPGPU, since any code I wrote would work on any customer machine, Intel, AMD, or NVidia. On the other hand, it would create an unlevel playing field for NVidia, since as the copyright holder, only NVidia could have proprietary optimizations. HPC would go to NVidia, as would markets like video editing or CAD.
Hopefully the RDNA3 software stack is good enough that AMD decides that CDNA2 (or CDNA-3) can be based off of the RDNA-instruction set.
AMD doesn't want to piss off its $100 million+ customers with a crappy software stack.
---------
BTW: AMD is reporting that parts of ROCm 4.3 are working with the 6900 XT GPU (suggesting that RDNA code generation is beginning to work). I know that ROCm 4.0+ has made a lot of github checkins that suggest that AMD is now actively working on the RDNA-code generation. Its not officially written into the ROCm documentation yet, its mostly the discussions with ROCm github issues that are noting these changes.
Its not official support and its literally years late. But its clear what AMD's current strategy is.
yeah, but that's exactly what OP said - Vega is three generations old at this point, and that is the last consumer GPU (apart from VII which is a rebranded compute card) that ROCm supports.
On the NVIDIA side, you can run at least basic tensorflow/pytorch/etc on a consumer GPU, and that option is not available on the AMD side, you have to spend $5k to get a GPU that their software actually supports.
Not only that but on the AMD side it's a completely standalone compute card - none of the supported compute cards do graphics anymore. Whereas if you buy a 3090 at least you can game on it too.
Also, just flat-out using your own developers to write the basic code for libraries and frameworks and so on. There are tons of basic libraries like CuRAND and CUB (Cuda UnBound) and I'm sure tons of others that NVIDIA wrote not because a customer wanted that, but because it was building blocks that others could leverage into products that would sell NVIDIA hardware. In contrast AMD has fallen behind in that area, and has further fragmented their development work across a bunch of "flavor of the week" frameworks and implementations that usually get discarded within a couple years (remember HIP? remember Bolt? etc)
That's exactly the point. ML on AMD is a third-class citizen.
Now don't get me wrong: $9000 is a lot for a development system to try out the software. NVidia's advantage is that you can test out the A100 by writing software for cheaper GeForce cards at first.
NVidia also makes it easy with the DGX computer to quickly get a big A100-based computer. AMD you gotta shop around with Dell vs Supermicro (etc. etc.) to find someone to build you that computer.
I have not seen any problems with those cards in BLAS or SOLVER, though they don't get tested as much as the officially supported cards.
FWIW, I finally managed to buy an RX 6800 XT for my personal rig. I'll be following up on any issues found in the dense linear algebra stack on that card.
I work for AMD on ROCm, but all opinions are my own.
Why? Because as-is, most people still believe support for gfx1000 cards is non-existent in any ROCm library. Of course that's not the case as you've pointed out here, but without any good sign of forward progress, your average user is going to assume close to zero support. Vague comments like https://github.com/RadeonOpenCompute/ROCm/issues/1542 are better than nothing, but don't inspire that much confidence without some more detail.
Long term, AI (& a lot of other interests) need to serve themselves. CUDA is excellently convenient, but long term I have a hard time imagining there being a worthwhile future for anything but Vulkan. There don't seem to be a lot of forays into writing good all-encompassing libraries in Vulkan yet, nor many more specialized AI/ML Vulkan libraries, so it feels largely like we more or less haven't started really trying.
I see a lot a lot a lot of resistance to the idea that we should start trying to align to Vulkan. Here & elsewhere. I don't get it, it makes no sense, & everyone else using GPU's is running fast as they can towards Vulkan. Is it just too soon too early in the adoption curve, or do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit? It still feels inevitable to me. What we are doing now feels like a waste of time. I wish ya'll wouldn't downvote so casually, wouldn't just try to brush this viewpoint away.
They had no choice. Getting a bunch of HPC people to completely rewrite their code for a different API is a tough pill to swallow when you're trying to win supercomputer contracts. Would they have preferred to spend development resources elsewhere? Probably, they've even got their own standards and SDKs from days past.
> everyone else using GPU's is running fast as they can towards Vulkan
I'm not qualified to comment on the entirety of it, but I can say that basically no claim in this statement is true:
1. Not everyone doing compute is using GPUs. Companies are increasingly designing and releasing their own custom hardware (TPUs, IPUs, NPUs, etc.)
2. Not everyone using GPUs is cares about Vulkan. Certainly many folks doing graphics stuff don't, and DirectX is as healthy as ever. There have been bits and pieces of work around Vulkan compute for mobile ML model deployment, but it's a tiny niche and doesn't involve discrete GPUs at all.
> Is it just too soon too early in the adoption curve
Yes. Vulkan compute is still missing many of the niceties of more developed compute APIs. Tooling is one big part of that: writing shaders using GLSL is a pretty big step down from using whatever language you were using before (C++, Fortran, Python, etc).
> do ya'll think there are more serious obstructions long term to building a more Vulkan centric AI/ML toolkit
You could probably write a whole page about this, but TL;DR yes. It would take at least as much effort as AMD and Intel put into their respective compute stacks to get Vulkan ML anywhere near ready for prime time. You need to have inference, training, cross-device communication, headless GPU usage, reasonably wide compatibility, not garbage performance, framework integration, passable tooling and more.
Sure these are all feasible, but who has the incentive to put in the time to do it? The big 3 vendors have their supercomputer contracts already, so all they need to do is keep maintaining their 1st-party compute stacks. Interop also requires going through Khronos, which is its own political quagmire when it comes to standardization. Nvidia already managed to obstruct OpenCL into obscurity, why would they do anything different here? Downstream libraries have also poured untold millions into existing compute stacks, OR rely on the vendors to implement that functionality for them. This is before we even get into custom hardware like TPUs that don't behave like a GPU at all.
So in short, there is little inevitable about this at all. The reason people may have been frustrated by your comment is because Vulkan compute comes up all the time as some silver bullet that will save us from the walled gardens of CUDA and co (especially for ML, arguably the most complex and expensive subdomain of them all). We'd all like it to come true, but until all of the aforementioned points are addressed this will remain primarily in pipe dream territory.
The end is decrying how impossible & hard it is to imagine anyone ever reproducing anything like CUDA in Vulkan:
> Sure these are all feasible, but who has the incentive to put in the time to do it?
To talk to the first though: what choice do we have? Why would AMD try to compete by doing it all again as a second party? It seems like, with Nvidia so dominant, AMD and literally everyone else should realize their incentive is to compete, as a group, against the current unquestioned champion. There needs to be some common ground that the humble opposition can work from. And, from what I see, Vulkan is that ground, and nothing else is remotely competitive or interesting.
I really appreciate your challenges, thank you for writing them out. It is real hard, there are a lot of difficulties starting afresh, with a much harder to use toolkit than enriched spiced up C++ (CUDA) as a starting point. At the same time, I continue to think there will be a sea-change, it will happen enormously fast, & it will take far less real work than the prevailing pessimist's view could ever have begin to encompassed. Some good strategic wins to set the stage & make some common use cases viable, good enough technics to set a mold, and I think the participatory nature will snowball, quickly, and we'll wonder why we hadn't begun years ago.
To end things on a somewhat brighter note, there will be no sea change unless people put in the time and effort to get stuff like Vulkan compute working. As-is, most ML people (somewhat rightfully) expect accelerator support to be handed to them on a silver platter. That's fine, but I'd argue by doing so we lose the right to complain about big libraries and hardware vendors doing what's best for their own interests instead of for the ecosystem as a whole.