Comment by nemothekid | Hacker News Reader

Comment by nemothekid | Hacker News Reader

Parent Full thread

>their fab advantage might have let them really go after Nvidia.

I think Cantrill spelled it out in the article, and I think people are overlooking (and for some reason really continue to overlook, or just have no idea) that nvidia's advantage is Software.

I remember Larabee as a teenager - I just graduated high school, the forums were talking about it, and I had finally saved enough money to buy an 8800GTX. The 8800GTX was also coincidentally, the first consumer GPU to support CUDA. Almost 2 decades later and it feels like nvidia has an insurmountable moat in AI due to CUDA. This wasn't an accident, nvidia spent a ton of time, with developers, getting it right. FTA:

>that NVIDIA had an indisputable lead, even in 2009; that for Intel to dominate NVIDIA it would have required conjuring software expertise and focus with which Intel has famously struggled

Ignoring this piece is huge in claming that nvidia would be a quarter of the size. Can you even imagine Intel investing in a non-x86 developer platform for 15 years while they were in the lead?

Not the Intel we have. But if the graphics card stuff had really started to show promise, maybe they would have become an Intel that would.

You’re dead right that Nvidia is where they are because of their software story and all the work they put into it.

I don’t know if Intel could have learned that lesson.

> I don’t know if Intel could have learned that lesson.

Intel invented this model with x86.

Really? It doesn't seem to me like much Intel-written software has been key to x86's success. The things I can remember from Intel are compilers, performance analysis tools, and threading and math libraries. But none of them were must-haves, or differentiated x86 chips from competing CPU architectures. Most of the software that made x86 successful came from third parties.

And more recently, the reviews I read of the Archmage graphics drivers didn't make it sound like Intel had understood the importance of the software that was essential to a pretty big hardware release.

Your examples are simply too recent. They've been doing it since the 70s: https://en.wikipedia.org/wiki/Intel_system_development_kit - that they forgot what made them the success they are is unfortunate, but also wholly self-inflicted.

Those are dev boards -- hardware, not software.

not sure why people give so much credit to Cuda, at least now.

AI hasn't cared about Cuda for years - sure, Torch on NV will use it, but AI is all about Torch/TF, not details below that.

What? This is really wrong. The name of the game these days is optimizing memory and FLOPs usage on these GPUs. For example, Hopper (H100) cards introduced the WGMMA (Warp-group Matrix-Multiply Accumulate) instruction, and anyone who does LLM training jumped to use it ASAP because without it you can't fully utilize the FLOPs. Anyone training AIs at the cutting age cares a lot about CUDA details. https://hazyresearch.stanford.edu/blog/2024-05-12-tk

That’s not what OP meant though. That’s caring about _hardware_ capability. You could do that yourself for different hardware. And (OP’s point) any alternative hardware provider could do that for you. It just happens to be really, really hard

I think "AI is all about Torch/TF, not details below that" directly contradicts the fact that ML people very much care about the details, to squeeze performance out of the hardware.

>AI is all about Torch/TF, not details below that

This could be 100% true, and the best ML engineers in the world could be completely oblivious to CUDA and it wouldn't change why nvidia is leading right now.

Anyone who thinks it's "trivial" to use AMD cards should help geohot and his tinygrad boxes. Despite emailing the CEO several times, it was a clear uphill battle to get something working, and even now the sales page lists driver quality as "mediocre".

If you open a GitHub issue or a forum post about your weird CUDA application that isn't working right, you will have an engineer help you. This is in nvidia's DNA and they did this before AI started printing money. On the flipside, you can email Lisa Su today and still not have problems fixed in AMDs drivers. I'm sure AMD is trying, this isn't to say they aren't, but having decent software culture at these large companies is hard.

If AMD can't do it, I feel even less confident about Intel. If the CUDA moat truly didn't matter, we would see AMD cards in demand. However it doesn't and it isn't for lack of trying.

I literally sat at a table where the argument was made: The Intel accelerator is faster. It provides more FLOPs. But you can’t get actual users to use it outside of our LINPack runs, since it just doesn’t provide bindings to all the software actually intended to run. Granted that was in the HPC space (ie FP64 performance), but that is literally the argument.

That doesn’t mean, that ML people don’t care about some hardware specifics btw. Of course they do! Because CUDA is very, very good at exploiting the specific features of the newer cards (I’ve forgotten, what exactly this was though). But take away the simple integration into their favorite AI-SDK and it would be completely irrelevant.

The point is just that, if Intel were to build strictly better hardware, but not provide the interface to torch, then they would still loose to Nvidia. Yes, yes, this is not completely true. Given enough advantage someone else would build the IntelCUDA. But this is literally the bet that those academics make. And it’s so risky, that only academics are doing that

Because Torch only exists on CUDA-platforms. At least in any useful form.

This was, if anything, the main theme at Supercomputing: “there is not even a point in talking about Nvidia’s new benchmarks. You all are going to buy it anyways” coupled with very few “we are betting on buying cheap hardware (Intel/AMD GPUs) and hope that we can build the relevant parts of CUDA ourselves” and the latter is pretty much a desperation move of labs/sites that simply cannot get NVIDIA GPUs (either price or availability).

And yes that is probably the seed of the end of Nvidia’s dominance. But it will take 20 years and multiple fuck ups. Just as it did with Intel

Maybe not now (I’m not in a position to know), but wasn’t it a HUGE factor in them becoming the player they are in non-graphics stuff? Because they had great APIs/libraries/documentation to play in that space?

Reply on news.ycombinator.com