Comment by schmidtleonard | Hacker News Reader

Comment by schmidtleonard | Hacker News Reader

Parent Full thread

I don't have the inside baseball but I have seen those weird as hell interviews with Lisa Su where she gets asked point blank about the software problems and instead of "working on it, stay tuned" -- an answer that costs nothing to give -- she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. No, the horsepower of your F1 racecar doesn't matter if the engine doesn't start and there's a wheel missing! You need to fix those problems before the horsepower can matter! Please tell me you are fixing the starter and the wheel!

Hopefully I am reading too much into this. Hopefully she doesn't have any weird hangups over investing in software and it all just takes time to Do It Right after GPGPU got starved in the AMD winter. But if it is a weird hangup then yeah, 100%, ownership needs to get management in line because whiffing a matmul benchmark years into a world where matmul is worth trillions just ain't it.

> she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having.

It's not a deflection, but a straightforward description of AMDs current top-down market strategy of partnering with big players instead of doubling down to have a great OOBE for consumers & others who don't order GPUs by the pallet. It's an honest reflection if their current core competencies, and the opportunity presented by Nvidia's margins.

They are going for a bang-for-buck right now aiming at data center workloads, and the hyperscalers care a lot about perf/$ than raw performance at. Hyperscalers are also more self-sufficient at software: they have entire teams working on PyTorch, Jax, and writing kernels.

Engineers at hyperscalers are struggling through all the bugs too. It's coming at notable opportunity cost for them, at a time when they also want an end to the monopoly. Do they buy AMD and wade through bug after bug, regression after regression, or do they shell out slightly more money for Nvidia GPUs and have it "just work".

AMD has to get on top of their software quality issues if they're ever going to succeed in this segment, or they need to be producing chips so much faster than Nvidia that it's worth the extra time investment and pain.

> Engineers at hyperscalers are struggling through all the bugs too

[citation needed]

It's in the article. Meta don't use AMD for training and write their own kernels for inference. You can't train with AMD, full stop, because their software stack is so buggy.

> It's in the article

The same article also states that AMD provided custom bug-fixes written by Principle Engineers to address bugs in a benchmark - this is software that will only become part of the public release in 2 quarters. I ask again, do you think AMD will not expedite non-public bug-fixes for hyperscalers?

> You can't train with AMD, full stop, because their software stack is so buggy.

Point 7 from the article:

>> The MI300X has a lower total cost of ownership (TCO) compared to the H100/H200, but training performance per TCO is worse on the MI300X on public stable releases of AMD software. This changes if one uses custom development builds of AMD software.

I have an inkling that Meta does not obtain MI300 drivers from https://download.amd.com

AI labs don't want to train models using a stack build some guy hacked up on his desktop last night that's been through no proper QA process. The cost of a training run that fails or results in a garbage model due to numerical errors are huge.

Which is why, as they say clearly, nobody is training models on AMD. Only inference, at most. I'm not sure why you keep claiming they are training using private drivers. They clearly aren't.

Now I i see how where talking past each other. I 100% agree that none of if the hyperscalers are currently (publicly) training on AMD silicon. I disagree with forward-looking statements like it "can't" happen, because I can guarantee you several of them are actively working on making it possible to train on AMD chips - that's just too juicy a target for a bonus packets all the way up to directors: "Our team saved the org $x0 million in TCO vy enabling training on MI300/MI400X in our new clusters"

Oh, I see. "Can't" means in the present tense in my previous sentence, it wasn't meant to be a definitive statement about the entire future.

Lolol 100% accurate. Go trawl through PRs to Triton by FB people to the AMD portion of the codebase.

Sorry, but NDAs mean I can't say any more.

That's the excuse used by every big company shitting out software so broken that it needs intensive professional babysitting.

I've been on both sides of this shitshow, I've even said those lines before! But I've also been in the trenches making the broken shit work and I know that it's fundamentally an excuse. There's a reason why people pay 80% margin to Nvidia and there's a reason why AMD is worth less than the rounding error when people call NVDA a 3 trillion dollar company.

It's not because people can't read a spec sheet, it's because people want their expensive engineers training models not changing diapers on incontinent equipment.

I hope AMD pulls through but denial is _not_ the move.

What exactly are they in denial about? They are aware that software is not a strength of theirs, so they partner with those who are great at it.

Would you say AMD is "shitting the bed" by not building it's own consoles too? You know AMD could build a kick-ass console since they are doing the heavy-lifting for the Playstation, and the XBox[1] , but AMD knows as much as anybody that they don't have the skills to wrangle studio relationships or figure out which games to finance. Instead, they lean hard in their HW skills and get Sony Entertainment/the Xbox division do what they do best.

1.and the Steam Deck, plus half a dozen Deck clones.

There is probably one employee - either a direct report of Su's or maybe one of her grandchildren in the org chart - who needs to "get it". If they replaced that one manager with someone who sees graphics cards as a tool to accelerate linear algebra then AMD would be participating more effectively in a multi-trillion dollar market. They are so breathtakingly close to the minimum standards of competence on this one. We know from the specs that the cards they produce should be able to perform.

This is a case-specific example of failure, it doesn't generalise very well to other markets. AMD is really well positioned for this very specific opportunity of historic proportions and the only thing holding them back is a somewhat continuous stream of unforced failures when writing a high quality compute driver. It seems to be pretty close to one single team of people holding the company back although organisational issues tend to stem from a level or two higher than the team. This could be the most visible case of value destruction by a public company we'll see in our lifetimes.

Optimistically speaking maybe they've already found and sacked the individual responsible and we're just waiting for improvement. I'm buying Nvidia until that proves to be so.

It would go nowhere, games history is full of great hardware that died because they failed to have a profitable ecosystem.

Even Steam Deck is only a success, because it depends on Windows ecosystem, and the moment Microsoft decides it is enough, lets see how long it holds.

Steam Decks run on Arch Linux

As means to avoid paying for Windows licenses.

All the games that matter are Windows games running via Proton, as Valve has failed to actually build a GNU/Linux native games ecosystem, in spite of UNIX/POSIX underpinnings of Android NDK, PlayStation, the studios hardly bother.

The day Microsoft actually decides to challenge Proton, or do a netbooks move on handhelds with XBox OS/Windows, the SteamDeck will lose, just like the netboooks did.

Additionally, it is anyone's guess what will happen to Valve when Gabe steps down.

As a means of control

If they cared about control, they wouldn't depend on Windows ecosystem, rather foster GNU/Linux native games.

Everyone is quite curious what Microsoft will drop at CES 2025, and which OEMs will be on their side, it is going to be netbooks all over again.

They are literally fostering Linux games by selling and endorsing a platform where those games would be native as well as having native releases of their own games. They aren't gonna force any third-party devs to do the same, but they're showing that there is a market while also growing it.

The ones buying gpus in pallets _really_ need the software to not be a worry. Software issues would ruin the entire operational life

>Hyperscalers are also more self-sufficient at software: they have entire teams working on PyTorch, Jax, and writing kernels.

None of this matters because AMD drivers are broken. No one is asking AMD to write a PyTorch backend. The idea that AMD will have twice the silicon performance than nvidia to make up the performance loss for bad software is a pipedream.

> None of this matters because AMD drivers are broken

Do you honestly think the MI300 has show-stopper driver bugs, or that Meta/Amazon doesn't have a direct line to AMD engineers?

>Do you honestly think the MI300 has show-stopper driver bugs

Yes

>Meta/Amazon doesn't have a direct line to AMD engineers?

I don't even think AMD engineers have a direct line to AMD.

The fine article says they had the ears of VPs and Principal Engineers, just to help with a benchmark. Meta/Amazon will get white-glove service

> None of this matters because AMD drivers are broken

How do you know that the problems arise from broken drivers rather than broken hardware? Real world GPU drivers are full of workarounds for hardware bugs.

Reply on news.ycombinator.com