My point is these things don't seem to last in tech.
What makes Nvidia seem unbeatable is that Nvidia does the best job on hardware design, does a good job on the software for the hardware and gets its designs out quickly such that they can charge a premium. By the time the competition makes a competitive design, Nvidia has the next generation ready to go. They seem to be trying to accelerate their pace to kill attempts to compete with them and so far, it is working.
Nvidia just does not do the same thing better in a new generation, but tries to fundamentally change the paradigm to obtain better than generational improvements across generations. That is how they introduced SIMT, tensor cores, FP8 and more recently FP4, just to name a few. While their competitors are still implementing the last round of improvements Nvidia made to the state of the art, Nvidia launches yet another round of improvements.
For example, Nvidia has had GPUs on the market with FP8 for two years. Intel just launched their B580 discrete GPUs and Lunar Lake CPUs with Xe2 cores. There is no FP8 support to be seen as far as I have been able to gather. Meanwhile, Nvidia will soon be launching its 50 series GPUs with FP4 support. AMD’s RDNA GPUs are not poised to gain FP8 until the yet to be released RDNA 4 and I have no idea when Intel’s ARC graphics will gain FP8. Apple’s recent M4 series does have FP8, but no FP4 support.
Things look look less bad for Nvidia’s competitors in the enterprise market, CDNA 3 launched with FP8 support last year. Intel had Gaudi 2 with FP8 support around the same time as Nvidia, and even launched Gaudi 3. Then there is tenstorrent with FP8 on the wormhole processors that they released 6 months ago. However, FP4 support is no where to be seen with any of them and they will likely not release it until well after Nvidia, just like nearly all of them did with FP8. This is only naming a few companies too. There are many others in this sector that have not even touched FP8 yet.
In any case, I am sure that in a generation or two after Blackwell, Nvidia will have some other bright idea for changing the paradigm and its competition will lag behind in adopting it.
So far, I have only discussed compute. I have not even touched on graphics, where Nvidia has had many more innovations, on top of some of the compute oriented changes being beneficial to graphics too. Off the top of my head, Nvidia has had variable rate shading to improve rendering performance, ray tracing cores to reinvent rendering, tensor cores to enable upscaling (I did mention overlap between compute and graphics), optical flow accelerators to enable frame generation and likely others that I do not recall offhand. These are some of the improvements of the past 10 years and I am sure that the next 10 years will have more.
We do not see Nvidia’s competition put forward nearly as many paradigm changing ideas. For example, AMD did “smart access memory” more than a decade after it had been standardized as resizeable bar, which was definitely a contribution, but not one they invented. For something that they actually did invent, we need to look at HBM. I am not sure if they or anyone else I mentioned has done much else. Beyond the companies I mentioned, there are Groq and Cerebras (maybe Google too, but I am not sure) with their SRAM architectures, but that is about it as far as I know of companies implementing paradigm changing ideas in the same space.
I do not expect Nvidia to stop being a juggernaut until they run out of fresh ideas. They have produced so many ideas that I would not bet on them running out of new ideas any time soon. If I were to bet against them, I would have expected them to run out of ideas years ago, yet here we are.
Going back to the discussion of Intel seeming to be unbeatable in the past, they largely did the same thing better in each generation (with occasional ISA extensions), which was enough when they had a process advantage, but it was not enough when they lost their process advantage. The last time Intel tried to do something innovative in its core market, they gave us Itanium, and it was such a flop that they kept doing the same thing incrementally better ever since then. Losing their process advantage took away what put them on top.
This is the most important point. Everyone seems to think that Nvidia just rests on its laurels while everyone and their dog tries to catch up with it. This is just not how (good) business works.
~"Good fortune favors those who are prepared to take advantage of it", etc.
But he also made sure that resting won't ever happen again. Andy Grove from Intel once said that only the paranoid survive and I bet Jensen is the most paranoid CEO alive. You won't see him in public that way because Jensen in public is Nvidia marketeer.
Nvidia has also understood early on how important marketing and brand recognition is. They learned it the hard way with the utter failure of their first chip the NV1 which was a technological master piece which no one wanted. Witht the bad GeForce FX Nvidia even made marketing videos to make fun of themselves and that helped. ATI didn't crush Nvidia as much as expeceted because of such activities despite having the ultra superior Radeon 9700/9800 series back then.
BK failed to understand the moat Intel had was the Fab. The moat is now gone and so is the value.
It's able to get the best process node from /whoever is willing to sell it to Nvidia/: it's vulnerable (however unlikely) to something very similar -- a competitor with a process advantage.
In summary the software story was very surprisingly better than I expected (no Jax though).
There were also ISA extensions. Even if Intel had trouble competing on existing code, they would often extend the ISA to gain a temporary advantage over their competitors by enabling developers to write more optimal code paths that would run only on Intel’s most recent CPUs. They have done less of that ever since the AVX-512 disaster, but Intel still is the one defining ISA extensions and it historically gained a short term advantage whenever it did.
Interestingly, the situation is somewhat inverted as of late given Intel’s failure to implement the AVX-512 family of extensions in consumer CPUs in a sane way, when AMD succeeded. Intel now is at a disadvantage to AMD because od its own ISA extension. They recently made AVX-10 to try to fix that, but it adds nothing that was not already in AVX-512, so AMD CPUs after Zen 3 would have equivalent code paths from AVX-512, even without implementing AVX-10.
Thats where Nvidia learned to "optimize" Cuda software path. Single threaded x87 FPU on SSE2 capable CPUs.
https://arstechnica.com/gaming/2010/07/did-nvidia-cripple-it...
https://www.realworldtech.com/physx87/3/ "For Nvidia, decreasing the baseline CPU performance by using x87 instructions and a single thread makes GPUs look better."
They doubled down that approach with 'GameWorks' crippling performance on non Nvidia GPUs, Nvidia paid studios for including GameWorks in their games.
Nvidia is helping power the next generation of big brother government programs.
And what does pytorch et al. use under the hood? cuBLAS and cuDNN, proprietary libraries written by NVidia. That is where most of the heavy lifting is done. If you think that replicating the functionality and performance that these libraries provide is easy, feel free to apply for a job at NVidia or their competitors. It is pretty well paid.
Anyone that wants off the shelf parts at scale is going to turn to Nvidia.
I never claimed it was easy. I meant in my opinion it is in the order of 10s of millions dollars of investment, not a trillion dollar CUDA moat that people comment here.
And we just gave them billions in tax dollars. Failing upwards...
Rich webapps hadn't been invented. Smartphones? If you're lucky your flip phone might have a colour screen. If you've got money to burn, you can insert a PCMCIA card into your Compaq iPAQ and try out this new "802.11b" thing. Java was... being Java.
Almost all the software out there - especially if it had a GUI, and a lot of it did - was distributed as binaries that only ran on x86.
A vast amount of code was only intended to compile and run on a single OS and architecture (circa 2000, that was usually x86 Win32; Unix was dying and Wintel had taken over the world). If some code needed to be ported to another platform, it was as good as a from-scratch re-write.
[0] in case you wanted to use the thing in Visual Basic, which you very well might.
"had". That's what helped prop up their monopoly but it didn't last. These days if can't run your software on another architecture, like ARM, you can run at least on AMD. AMD can basically run the same software as Intel. This isn't the situation for NVIDIA vs everyone else, so far.
I'm thinking about buying Nvidia
(this is bullshit)