Hacker News Reader

Comment by justinator | Hacker News Reader

When there's a gold rush, sell the pick axes and dynamite.

Nvidia has been the pick axe vendor for 2 gold rushes in a row now. First crypto, then AI. It's an extremely nice position to be in. Regardless of how something like OpenAI shakes out, Nvidia will keep making money.

Seriously. I can't think of any other company that has managed to "luck out" this much.

GPU's were designed for graphics. But then they just happened to be the right tool for crypto, and then the right foundation for ML.

I can't think of any other company that has managed to benefit so much from a product designed for one thing, that turned out to be the solution to other things as well.

(Edit: obviously they've done a ton of work to capitalize on these two trends and plan for them as they saw them coming, but that was always leveraging their pre-existing massive investment in GPU design and expertise.)

Luck is the combination of preparation and opportunity. Cuda has played a huge role in Nvidia's ML success. They were working on this for years before the AI wave took off.

This is a very uninformed comment.

As much as I hate praising big corp, NVDA played the game right. They saw the value of GPGPUs and invested heavily in CUDA, sponsored universities and research labs when the DL boom happened, helped all ML frameworks into incorporating CUDA (dl or otherwise), has done a lot of work on cudNN, sparsity, and has published countless papers.

It didn't simply "turn out", and claiming such is doing a disservice to all the people who made that reality.

Intel could have very well built accelerators and GPUs, but they didn't. AMD also didn't even though they had a GPU division as well.

Hardly , this is not about nvidia doing the foundation to make their own luck .

It is about those industries taking off in a big way.

Nvidia has zero contribution towards tranformers which was the driver behind the rise of AI or towards all the money that flowed in crypto during the low interest era.

The point has all to do with how they have been part of two cycles of boom unrelated to their product.

They are not AWS to have driven the cycle by their product itself , they have only capitalized on it with their good work .

I recall that all the way back during the early days of their GPUs they were already talking about their compute capabilities, so I rather than luck, it's really just a lot of foresight, with crypto and AI emerging because of GPUs becoming good enough for the task, rather than happening to fit the problem.

I don't think it's exactly "luck", the trends are being driven by the technology, which they have done a good job of inventing. For better or worse, low-watt parallel compute is the primary area of innovation in compute hardware, so naturally the areas of innovation in software are more likely to leverage this innovative hardware. In other words, when all you have is a ton of tiny processors, every problem looks like a massively parallel process. It does raise the question of what we are missing because we don't have e.g. super-fast single-threaded processors or whatever new tech exists in an alternate universe.

> I can't think of any other company that has managed to benefit so much from a product designed for one thing, that turned out to be the solution to other things as well.

CUDA was first released in 2007:

* https://en.wikipedia.org/wiki/CUDA

* https://developer.download.nvidia.com/compute/cuda/1.0/NVIDI...

Two years before the Bitcoin paper (2009):

* https://en.wikipedia.org/wiki/Bitcoin

They had a presentation called "The Era of the Personal Supercomputing" at SIGGRAPH 2007:

* https://dl.acm.org/doi/10.1145/1281500.1281647

* https://www.nvidia.com/content/events/siggraph_2007/supercom...

Ian Buck (co-?)creator of CUDA speaking in 2008:

> Ian Buck talks about his background developing Brook for GPUs at Stanford university and what paths were taken for developing a C platform for GPUs.

* https://www.youtube.com/watch?v=Cmh1EHXjJsk

> In 2003, a team of researchers led by Ian Buck unveiled Brook, the first widely adopted programming model to extend C with data-parallel constructs. Ian Buck later joined NVIDIA and led the launch of CUDA in 2006, the world's first solution for general-computing on GPUs.

* https://developer.nvidia.com/cuda-zone

* http://graphics.stanford.edu/~ianbuck/

Nvidia purposefully went after parallel computing. Specific applications (cryptocurrency, ML/AI) appeared later.

A strong argument can be made that Nvidia helped these technologies gain traction.

In the early 2000s it was a common theme in the CS literature that the Von Newman architecture was about to run out of steam and some kind of parallel processor was going to become mainstream, out of all that work the GPGPU became the most famous. If NVIDIA hadn’t done it, someone else would have.

Heh, I remember that too. Von Neumann architectures being legacy was all the rage, and many students were excited to be able to build the next thing.

Indeed,

The thing is, Nvidia has "done it" and still no one else is "doing it". As the gp points out, Nvidia launched CUDA a while and this made Nvidia's products general purpose parallel processors.

They have succeeded as that and yet still no one is selling chips that either have an equivalent interface or (better) are drop-in Cuda compatible (since we know interfaces and programming language copyrightable).

The situation seems a bit like early PC days when companies sold (roughly) PC compatible computers were appropriately shoved aside by drop-in compatible clones/white-boxes. A company that makes a thing "kinda like" the leader but with it's own sauce is just trying to carve a piece of an existing market. The company that makes something exactly compatible with the leader has an incentive to go further. I hate to say "embrace and extend" but there you have it.

Not crypto. Bitcoin had a very short GPU phase that spanned for a few months, before moving to FPGAs and finally to ASICs.

Ethereum made a concerted effort to sabotage ASIC mining, and remain GPU minable. Such is the case for all other GPU minable coins.

So no, I don't think crypto needed GPUs.

Related to that, I feel like AI should start moving to FGPAs/ASICs soon.

My own personal history has to do with it, because when I was doing my PhD work they defunded our supercomputing center. Parallel computing went from being “the future” to a boring backwater dominated by the military industrial complex and a handful of ‘boondoggle’ projects like figuring out how proteins fold in water which turned out to be a waste of time because we found out proteins don’t “fold” but instead they get folded. I diagonalized a 65 spin cluster on an IBM AIX machine if I used a supercomputer I could have diagonalized a spin 90 maybe and it wouldn’t have made any difference in the results.

Turned out having a powerful computer on your desktop was revolutionary and having to fill out paperwork to run your job was just… a tired old throwback.

I got a lot more excited about commercial “big data” developments in the 2000s as these completely escaped the gravity of the national labs and the handful of problems they do over and over again.

Well, parallelization was to be the stairway to heaven from Moores plateau..

A different lens is that GPUs are designed for parallel computing, of which graphics is the first primary use case. Crypto and AI/ML are perfect suits for parallel computing, and not the last ones. And nvidia is in the perfect spot there for the future of parallel computing as a whole.

They didn’t luck out, they developed CUDA for decades at this point

If OpenAI doesn’t achieve AGI, which it won’t, demand probably won’t continue to grow. LLM’s will find their limit and massive training budgets won’t be needed.

Since I'm being rate limited for one downvote:

> All science is brute force of guess and test, and OpenAI does a lot of that.

Nope. Science looks to answer a critical question. It doesn't conduct trillions of experiments at one time.

> You don't think there's any possibility of emergent properties? Given that we understand physically how the human brain works, but not how consciousness emerges from that, I'm pretty damn unconfident.

Possibility? Why? Because Altman & co are hyping things up? They have no proof of a breakthrough. It's just posturing.

Always interesting to see the hubris of certainty. You don't want to use ANY qualifiers?

If you accuse me of hubris, are you willing to take a $10,000 bet? It's clear that there is no science happening at OpenAI. It's pure brute force. You won't learn anything with that approach.

You don't think there's any possibility of emergent properties? Given that we understand physically how the human brain works, but not how consciousness emerges from that, I'm pretty damn unconfident.

LOL.

> It's pure brute force.

All science is brute force of guess and test, and OpenAI does a lot of that.

Nothing like being lucky and good.

This completely ignores the cash cow that is gaming, which has made them far more money than crypto ever did.

AI is a little different for now, we'll see how it holds up.

That article says that gaming cash has been eclipsed by A.I. in the short term but you could say the gaming market is more of a sure thing in the long term.

Not at these GPU prices. Over time, that might force people away from PC gaming, and Nvidia is not powering high-end consoles.

GeForce Now is really solid though, I've already moved most of my gaming to that. They could ride that wave as well.

It could let them push for higher gfx card pricing since the cards would have a higher duty cycle and deliver more value in a cloud environment but unfortunately I am in the wrong side of a DSL connection.

I am disappointed that nobody in cloud gaming has made a game that using the monsterous GPU instances in the cloud to synthesize a super-complex world and generate video for multiple players simultaneously but I think a system like that would not really beat highly optimized single players and furthermore optimizing it to the point where they could really deliver a “skip three generations” kind of experience would be expensive and risky.

I don't think ML is driving consumer GPU prices, although I could be wrong. They are selling 3080s and below primarily for gaming. Maybe an argument could be made for the 3090/4090, but even then that is extreme solo enthusiast territory.

It's clearly the higher end chips that are bringing in the AI revenue.

That’s a great business model, selling pickaxes and dynamite on credit though can backfire if your are the one loaning out the money. Nvidia is a bit at risk it this gold rush ends suddenly

we saw that with crypto and they took a hit for a quarter or so and moved on

They didn’t lend out money to the crypto industry last time

Reply on news.ycombinator.com