Hacker News Reader

Comment by crazygringo | Hacker News Reader

Seriously. I can't think of any other company that has managed to "luck out" this much.

GPU's were designed for graphics. But then they just happened to be the right tool for crypto, and then the right foundation for ML.

I can't think of any other company that has managed to benefit so much from a product designed for one thing, that turned out to be the solution to other things as well.

(Edit: obviously they've done a ton of work to capitalize on these two trends and plan for them as they saw them coming, but that was always leveraging their pre-existing massive investment in GPU design and expertise.)

View on HN

> I can't think of any other company that has managed to benefit so much from a product designed for one thing, that turned out to be the solution to other things as well.

CUDA was first released in 2007:

* https://en.wikipedia.org/wiki/CUDA

* https://developer.download.nvidia.com/compute/cuda/1.0/NVIDI...

Two years before the Bitcoin paper (2009):

* https://en.wikipedia.org/wiki/Bitcoin

They had a presentation called "The Era of the Personal Supercomputing" at SIGGRAPH 2007:

* https://dl.acm.org/doi/10.1145/1281500.1281647

* https://www.nvidia.com/content/events/siggraph_2007/supercom...

Ian Buck (co-?)creator of CUDA speaking in 2008:

> Ian Buck talks about his background developing Brook for GPUs at Stanford university and what paths were taken for developing a C platform for GPUs.

* https://www.youtube.com/watch?v=Cmh1EHXjJsk

> In 2003, a team of researchers led by Ian Buck unveiled Brook, the first widely adopted programming model to extend C with data-parallel constructs. Ian Buck later joined NVIDIA and led the launch of CUDA in 2006, the world's first solution for general-computing on GPUs.

* https://developer.nvidia.com/cuda-zone

* http://graphics.stanford.edu/~ianbuck/

Nvidia purposefully went after parallel computing. Specific applications (cryptocurrency, ML/AI) appeared later.

A strong argument can be made that Nvidia helped these technologies gain traction.

In the early 2000s it was a common theme in the CS literature that the Von Newman architecture was about to run out of steam and some kind of parallel processor was going to become mainstream, out of all that work the GPGPU became the most famous. If NVIDIA hadn’t done it, someone else would have.

Heh, I remember that too. Von Neumann architectures being legacy was all the rage, and many students were excited to be able to build the next thing.

Indeed,

The thing is, Nvidia has "done it" and still no one else is "doing it". As the gp points out, Nvidia launched CUDA a while and this made Nvidia's products general purpose parallel processors.

They have succeeded as that and yet still no one is selling chips that either have an equivalent interface or (better) are drop-in Cuda compatible (since we know interfaces and programming language copyrightable).

The situation seems a bit like early PC days when companies sold (roughly) PC compatible computers were appropriately shoved aside by drop-in compatible clones/white-boxes. A company that makes a thing "kinda like" the leader but with it's own sauce is just trying to carve a piece of an existing market. The company that makes something exactly compatible with the leader has an incentive to go further. I hate to say "embrace and extend" but there you have it.

Not crypto. Bitcoin had a very short GPU phase that spanned for a few months, before moving to FPGAs and finally to ASICs.

Ethereum made a concerted effort to sabotage ASIC mining, and remain GPU minable. Such is the case for all other GPU minable coins.

So no, I don't think crypto needed GPUs.

Related to that, I feel like AI should start moving to FGPAs/ASICs soon.

My own personal history has to do with it, because when I was doing my PhD work they defunded our supercomputing center. Parallel computing went from being “the future” to a boring backwater dominated by the military industrial complex and a handful of ‘boondoggle’ projects like figuring out how proteins fold in water which turned out to be a waste of time because we found out proteins don’t “fold” but instead they get folded. I diagonalized a 65 spin cluster on an IBM AIX machine if I used a supercomputer I could have diagonalized a spin 90 maybe and it wouldn’t have made any difference in the results.

Turned out having a powerful computer on your desktop was revolutionary and having to fill out paperwork to run your job was just… a tired old throwback.

I got a lot more excited about commercial “big data” developments in the 2000s as these completely escaped the gravity of the national labs and the handful of problems they do over and over again.

I don't think it's exactly "luck", the trends are being driven by the technology, which they have done a good job of inventing. For better or worse, low-watt parallel compute is the primary area of innovation in compute hardware, so naturally the areas of innovation in software are more likely to leverage this innovative hardware. In other words, when all you have is a ton of tiny processors, every problem looks like a massively parallel process. It does raise the question of what we are missing because we don't have e.g. super-fast single-threaded processors or whatever new tech exists in an alternate universe.

This is a very uninformed comment.

As much as I hate praising big corp, NVDA played the game right. They saw the value of GPGPUs and invested heavily in CUDA, sponsored universities and research labs when the DL boom happened, helped all ML frameworks into incorporating CUDA (dl or otherwise), has done a lot of work on cudNN, sparsity, and has published countless papers.

It didn't simply "turn out", and claiming such is doing a disservice to all the people who made that reality.

Intel could have very well built accelerators and GPUs, but they didn't. AMD also didn't even though they had a GPU division as well.

Hardly , this is not about nvidia doing the foundation to make their own luck .

It is about those industries taking off in a big way.

Nvidia has zero contribution towards tranformers which was the driver behind the rise of AI or towards all the money that flowed in crypto during the low interest era.

The point has all to do with how they have been part of two cycles of boom unrelated to their product.

They are not AWS to have driven the cycle by their product itself , they have only capitalized on it with their good work .

A different lens is that GPUs are designed for parallel computing, of which graphics is the first primary use case. Crypto and AI/ML are perfect suits for parallel computing, and not the last ones. And nvidia is in the perfect spot there for the future of parallel computing as a whole.

Luck is the combination of preparation and opportunity. Cuda has played a huge role in Nvidia's ML success. They were working on this for years before the AI wave took off.

I recall that all the way back during the early days of their GPUs they were already talking about their compute capabilities, so I rather than luck, it's really just a lot of foresight, with crypto and AI emerging because of GPUs becoming good enough for the task, rather than happening to fit the problem.

They didn’t luck out, they developed CUDA for decades at this point

Well, parallelization was to be the stairway to heaven from Moores plateau..

Reply on news.ycombinator.com