Hacker News Reader

Comment by brucethemoose2 | Hacker News Reader

That limitation is totally artificial.

There is nothing stopping AMD from allowing the sale of an affordable 48GB 7900 (or 32GB 7800) right now... In fact they already do in the form of the W7900, for $4500. They just don't sell those at competitive consumer prices, as they are playing the anticompetitive artificial segmentation game right alongside Nvidia, because they somehow think their tiny workstation market is more valuable than seeding AI development on their cards.

Technically speaking, AMD could sell "cheap" 96GB+ GPUs by simply swapping the 7000 series memory controller die. This would take some time and money, but not an eternity + fortune like Nvidia would need to tape out a whole new GPU die.

Intel has no workstation market to lose. I suspect the Arc A770 was kinda late and skimpy for a 32GB "AI" SKU, but the next generation could be very interesting.

View on HN

You have no idea how deep the Nvidia lock-in goes.

AMD only supports small fractions of the deep custom eco-systems built around models such as Stable Diffusion. Good luck getting half of the shit built for ComfyUI or Automatic1111 to run at all, and frequently those features are the killer ones which motivate folks to want to do AI in the first place. There's a reason why Nvidia is a first class citizen among all of the AI clouds. AMD is even behind Intel in this regard, because at least Intel had an existing robust ecosystem of accelerators (i.e. intel optimized sklearn/pytorch, OpenVino, etc)

AMD can be 4x better in every way compared to Nvidia hardware wise but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future.

Which makes it even more stupid that AMD does not at least try to compensate for their shortcoming by selling large memory GPUs, there would be a reason to develop for team red

Yeah.

If there was a 40GB-48GB 7900 SKU, I would have bought it over a 3090 in a heartbeat, even at a big premium over the 24GB card. And I'd be debugging ROCm support in ML projects left and right.

But I am not. The 24GB SKU is just not worth the trouble over a 3090, which is more RAM efficient out of the box. I can't pay $5K for a W7900.

> but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future.

Dec 6th was an announcement from Lisa Su about the MI300x. It also included a full on commitment to ROCm as well as other projects like pyTorch. They are clearly aware of what you're talking about, and working on it. It isn't something that can magically be fixed over night, but it is something that people can complain about on HN over and over and over again...

Some of us have been complaining for close to a decade and there's been multiple promises that have gone no where. It's better to keep complaining until the problem is actually fixed. CUDA is 16 years old and AMD has been "aware" of the issue for at least 10.

How could they promise to fix something that didn't exist 10-16 years ago?

In my eyes, the incentives weren't there before. But, they sure are now!

And nvidia is killing it right now because they predicted the need _before_ the incentives were there.

Nobody is arguing that AMD didn't get caught with their pants around their ankles or that they have a great reputation with developers. What matters is their statements and more importantly, actions, moving forward.

Right now, they are cranking out ROCm releases on schedule, contributing to a bunch of AI open source projects, listening to crazy people like George Hotz and have released a GPU product that leapfrogs the H100 and is built on OAM-UBB, at 1/3 the FLOPS/$, while also promoting fast ethernet standards.

I'd call that a pretty good start.

OpenCL did exist 10 years ago! Rocm has been around for 7 years. Pytorch is 7 years old and had CUDA support on day one and still doesn't have RocM support. Before the LLM craze people still wanted an alternative to nvidia and could see the writing on the wall. There was a bunch of complaining about Rocm and OpenCL didn't deliver. GeoHotz tried to get a RocM backend working and the examples just didn't work.

You can't build a software platform by just willing it into existence. It means sitting down with developers, writing documentation and support users. It means tackling use cases other than llama so that users feel confident in building the next greatest thing on your platform. At it is today, AMD will always be playing catch up and people will continue to push nvidia because the latest and greatest will be on Cuda.

I guess for those of spending money on real gpus for the last 10 years, nvidia was incentivized to take our money but amd wasn't.

Before the current crazy of AI, what else generated large sums of money for GPUs?

Games, rendering and crypto.

If I was a hardware company, that's all that I would have focused on. Certainly AMD could have done better on the software/developer front, nobody is arguing that.

I tend to live in the present and what matters is what they are saying and doing today. You can choose to be pissed off about the past, or you can work towards the future.

This is a pet peeve of mine, but A1111 is not a great example of a ML project. The backend codebase is a mess compared to diffusers, its slow even on Nvidia, its history is filled with controversy (including license breaking and outright piracy in the codebase), the frontend is buggy and unstable. I appreciate how it was a pioneer in 2022-H1 2023, but I am somewhat annoyed its still the de facto SD backend.

Better maintained inference projects are picking up considerable AMD/Intel support. Sometimes even Apple Silicon support.

The only real competitor is ComfyUI, because without the huge ecosystem around your other front end, you lose MASSIVE capabilities, for example, losing this is a big mistake: https://github.com/ljleb/prompt-fusion-extension

Is there something like that extension for comfy yet?

At least on Nvidia, A1111 has had near identical performance to ComfyUI on SDXL and SD1.5 for me. Besides that, there were typical minor problems that happen when some guy's hobby project goes viral and suddenly is under massive scrutiny. It was the first "good enough" interface and now there is massive community inertia with extensions, plugins, etc.

ComfyUI has found a place in the image pipeline design niche, but A1111 still seems like the best choice for new users who just want to spin up Stable Diffusion and generate some art for a D&D campaign or whatever.

Thats because ComfyUI is also based on the old SAI backend, not diffusers like, say, VoltaML.

AMD plays that artificial segmentation game with SR-IOV as well, unfortunately.

Intel is supporting SR-IOV in Linux with the latest Xe GPU driver they are developing, but it isn't clear to me that they'll allow it in the consumer cards. If they do, and if I can run a Linux host and Windows guest simultaneously on one card, I'll be on it like stink on a monkey.

I didn't know about SR IOV! I took would love to see that adopted on consumer cards.

>as they are playing the anticompetitive artificial segmentation game right alongside Nvidia

Product segmentation might be anti-consumer (in the sense that they can charge higher prices overall), but it's hardly "anticompetitive".

>than seeding AI development on their cards

*ceding

I understood "seeding AI development on their cards" as in "making sure AI development in the future will happen on their cards". "ceding" sounds like the opposite of that (i.e. giving up).

The anti competitiveness is from the pricing in the face of competition.

W7900 margins are enormous. In a competitive market, AMD would massively undercut the 48GB RTX A6000 and the 24GB 4090, which Nvidia is price gouging even more. They absolutely can... But they don't.

Separately, its also (IMO) a bone headed business decision.

Reply on news.ycombinator.com