Hacker News Reader

AMD has a really weird view on how the programming world works internally. Some years ago they had some ARM server chips. Did they allow people access? No. They only allowed it for "Serious customers" and the whole thing failed as no software was available for it.

Same applies to their compute stuff. They really really want you to purchase their Pro cards and whatnot, and if you have a consumer card well too bad. Go back to playing games. Whereas with my Nvidia card CUDA runs flawlessly which leads me to doing test projects at home and fixing a bug here and there.

AMD doesn't understand how much of development happens on either home computers or university labs with small budgets which means consumer cards. They seem to think that everything happens using big teams and big money. Even though in reality it's the small developers doing stuff that then drives the purchase of the very lucrative large clusters.

For all of it's downsides NVidia actually understand the grassroots approach. Which is why their stuff just works on consumer stuff too.

I feel there's a large disconnect at AMD between the CPU division and the RTG (Radeon Technologies Group) which feels more insular than the rest of the company. I feel Lisa Su had her hand deeper in propping up the wailing CPU division and kind of forgot about RTG that's been loosing market share to Nvidia for the last 10 years not just in gaming but in ML. Maybe she should take a break from Ryzen and look into leading the RTG.

My impression is that they largely seem to focus on Big Money HPC contracts and other very large deals where bespoke engineering on the software stack is worth their time and can be paid off. So if you're paying them a shitload of money in a massive supercomputing-style deal, they'll get you software for their otherwise-unobtainium accelerators. But normal "day to day" stuff is literally not on their radar. At best it's an afterthought.

I think the timing of when they entered into the field is also relevant. When Nvidia first came on the scene swinging, they really needed to see what would stick; so you have to try fishing in every available pond. You don't know what will necessarily pan out or catch on. So you need the tech to be available to all your consumers at every level, consumer, enterprise, HPC, everything. And it turns out, they found that there are customers at every single one of those levels, and providing for all of them provides both a good on-ramp, and a path to bigger things.

In contrast AMD is so late to the game, and there's so much money sloshing around in the field right now, and the trajectories of what big buyers want is so much more clear -- that they can just afford to mostly ignore the lower end. They know where the money is. The field and buyers and use cases are just much more well understood from the business "make money in the easiest way" POV.

As for Intel, they seem to actually be dedicating tons of money to the software stack for oneAPI, to work everywhere, which is not surprising IMO. They have the money to dump into this (the most money intensive) task and know it's critical for their products to reach anything other than pork-belly HPC contracts.

It's still peanuts compared to what semiconductor research&engineering costs. They can afford it to make the most out of their market share. AMD must be rational and prioritize to repay investors for their trust. Time will tell whether that strategy works out.

Also AMD seems to underestimate how many people may just be curious to use compute functionality with their consumer based card. I was ultra excited to get a rdna2 card to make a dedicated linux system with a friendly gpu. Gaming works, but davinci resolve doesn’t, can’t use AMF for screen recording or streaming. I have a 3060ti back in a box abd parts for a “work computer,” and I thought I would work a bit with my system with the 6700xt but can only really game, screen record, and edit video with kaden live using my intel cpu.

I watched a video over the weekend about how a new amd apu using rdna3 will make entry level/cheap graphics cards useless, but thats not really going to be the case if you want to dip your feet into utilizing an entry level card for workloads. Gamers turn their nose up at dinky gtx 1650s but at least you can use it to edit video. Theres no wonder intel doesnt see AMD as real competition in the space.

> Nothing actually works on non-NVidia hardware. "Actually works" is different from "theoretically works." A lot of what I use theoretically supports ROCm, but in practice, when you try to use it, you have enough bugs big and small, and things crash or don't work right.

I was surprised when I started playing around with Stable Diffusion a couple of weeks ago and it just kind of worked with ROCm. That being said, I did run into a couple of very annoying issues... almost running out of VRAM seems to trigger GPU timeouts but as far as I can tell the GPU itself is fine and the memory allocation crawled to a halt. This also seems to be much more likely when running at half precision, which is a shame given that the relatively large 16GB VRAM effectively gets cut in half when running at full precision...

But this is already a big improvement over "ROCm doesn't support RDNA2" [0] which was still the case ~1.5 years ago. I just really hope that AMD keeps up the effort and fixes all the small and big issues that are still around...

[0]: https://github.com/RadeonOpenCompute/ROCm/issues/1180

> I was surprised when I started playing around with Stable Diffusion a couple of weeks ago and it just kind of worked with ROCm.

Stable Diffusion is literally the single highest-profile application of machine learning. There has been extensive effort to make it work in all sorts of places. If it were the year 2000, someone would port it to a Beowulf cluster...

I'm more concerned about SpaCy, CuPy, the hundreds (thousands) of models on Hugging Face, etc. I do use a lot of that stuff. Buying a GPU or accelerator which only runs the high-profile stuff is a bad buy.

The other models on Hugging Face I tried also worked fine... I'm not saying that everything is running perfectly or even that buying an AMD card for compute is a good idea at the moment, but there has been quite some progress even compared to 2-3 years ago so I'm cautiously optimistic...

I still remember trying to train a model with TensorFlow on AMD maybe 2 years ago. The only option was OpenCL and it was disappointingly slow (and barely any load on the GPU) and in the end I switched back to my old 1060 which worked much better.

ROCm also has significant startup latency, and for Stable Diffusion additional latency compiling the new operations whenever you switch resolutions.

I found that running out of memory would lock up my X session, spam kernel logs about GPU ioctls timing out, and require a reboot. That's much less friendly than CUDA reporting an error to the requesting application and carrying on!

Another thing that could happen is a fundamental algorithmic advance to replace backprop. The brain doesn't use it, and backprop doesn't have answers for how to do long term credit assignment and continuous learning. It works poorly with RL as well.

Better algorithms would require writing new ML libraries which would weaken the stranglehold GPUs have on ML. New niches opening would inevitably be bad for Nvidia, but good for upstarts.

Geoffrey Hinton recently came up with a new "forward forward" algorithm that seems to fit the bill:

https://bdtechtalks.com/2022/12/19/forward-forward-algorithm...

I recently migrated a lot of my CUDA code to SYCL using Intel's compatibility tool [1]. I'm trying to get my applications to work irregardless of hardware, so it simply needed to be done despite the pain.

It actually wasn't as hard as I was expecting. Don't get me wrong, it will definitely depend on the size of your code base as to how effective of a tool it is. However, I managed to port over most of mine in the course of the month, and it's showing dividends now I'm making back performance gains using FPGA hardware acceleration. I'm now vendor neutral, and can run my code on team Red, Blue, or Green. I'd argue that heterogeneous computing is a good enough reason to make the switch.

[1]https://www.intel.com/content/www/us/en/develop/documentatio... [2]https://www.khronos.org/sycl/

I'm only an interested observer of the space, but a bunch of work is being done getting at least some of this stuff to run on Apple Silicon in recent months.

I've been running Stable Diffusion on my M1 MBP since...I dunno, November or something? and it's working great.

Slow as heck tho compared to on a dedicated card. I was running it on my M1 pro till I bought some old miners to resell and tried out a 3070. That thing gets me a 512 x 512 DDIM 20 steps photorealistic image with protogen 5.3 in 2.85 seconds. Its insane to watch it churn out images like that. My on my MPB I had to downgrade to the older SD model that was trained only one 256 x 256 to get decent feedback on my prompts.

Really? Mine usually only takes about 30s per image for 50-step 512x512 images (which is perfectly reasonable for my purposes)...I wonder if it has to do with the RAM (I've got an M1 Max with 64GB)? Or maybe I'm just not doing it enough to care about the speed so much.

Compared to the 2.85 seconds on my dedicated card? Yes 30 seconds is slow as heck. Youre talking about 10 times longer and waiting this time to adjust your prompt.

I dont find that acceptable for a workflow where I need to tweak prompts.

> Slow as heck tho compared to on a dedicated card.

It's important to mention that there is a huge difference in performance between a new MacBook with 8 GB or 16 GB of RAM when it comes to this memory-intensive ML tasks.

I got the 16gb version, though its too slow for a real workflow even on the heaviest model. Its literally a magnitude of difference going from 30 seconds to less than 3.

I dont think the m1 pro comes with 8gb either, so kinda unwarrented assumption. I love the kneejerk reaction on me saying anything bad about apple though.

Critically, there's also cost. The formula is $W Nvidia for N-flops vs $X ROCm for N-flops + $Y engineering cost of recoding.

If $W > $X + $Y, then people will go with AMD.

That's roughly right only if the people paying $W vs $X and $Y are the same people. The people paying $X and the people paying $Y are often not the same people across large swaths of the GPU-buying ecosystem. ($W and $X also change over time and with volume of units shipped and volume shipped varies with how useful and widely applicable the cards are.)

I'd love to get an AMD [6650XT-ish] for my teen's gaming PC. But they also do some programming with me and I slightly worry about "oh, your PC can't do CUDA, therefore can't do XYZ that's interesting to you". I probably will still just go AMD and then have us use rented GPUs if we want to do any CUDA demos together.

While the hardware and software budgets are not the same, some companies do have tricks to make the software team care about hardware at least in theory.

I have never ordered any server hardware, but there's a dashboard where I can see some abstracted infrastructure CAPEX and OPEX numbers for my team, org, division, etc.

These numbers get reported up the food chain and initiatives to reduce costs are rewarded if the savings are significant enough.

Initiatives to support ARM servers or AMD's CUDA equivalents can take a long time and be expensive to achieve full compatibility, but experiments in areas where the effort is not terribly high can demonstrate enough value to fund teams to do so.

I suspect that's most difficult in mid sized organizations where the hardware costs are high enough to get some savings but not high enough to fund a team to work on it, especially once the opportunity cost of new features is taken into account.

Universities, mid sized companies, big companies without a software focus, etc. spring to mind as the ones with the biggest disconnect between hardware costs and ability to fund engineers to cut them down.

For the long tail of ML uses, $Y dominates.

That's actually most of the market. AMD will do fine for 3d gaming, stable diffusion, and a few similar uses. And price/performance wins for giant data centers doing something specialized (but if NVidia is about to lose a major data center deal because $W > $X + $Y, they can just cut a deal on $W).