Hacker News Reader

Comment by agumonkey | Hacker News Reader

I'm still stumped by nvidia rise and how gaming propelled them into HPC/Supercomputing. Who knew playing video games who amplify research that much ?

View on HN

>I'm still stumped by nvidia rise and how gaming propelled them into HPC/Supercomputing. Who knew playing video games who amplify research that much ?

Jensen Huang did. When people think of tech visionaries they think of Jobs or Musk, but Huang is just as great. He's been bang on the money on the future of this industry since he founded Nvidia, which is how they managed to not just consistently stay ahead of their competition (ATI) or put them out of business (3dfx), but leapfrog them (AMD) by branching in several fields (AI/ML, PhysX, computer vision, self driving, compute etc.) He saw early on that GPUs should push into general-compute and not just be for video games, and he executed well on that.

There are interviews on Youtube with Huang at Standford IIRC, where he discusses his vision of the GPU industry from the early days of Nvidia. Check them out, the guy's not your typical CEO suit focused on the share price, but he's basically a tech genius.

So, to answer your other question about why only Nvidia manage to win compute and not the other GPU companies, it's simple. Huang had the vision for the entire ecosystem from GPU chips, to drivers, to APIs and SW libraries, to partnerships and cooperation with the people and the companies who will use them. Building great GPUs for compute is not enough if you're just gonna throw them on the market without the ecosystem and support behind them, and expect it to be a success. That's what Nvidia gets and the rest (AMD/Intel) don't. So while ATI/AMD had tunnel vision and was focused only on building gaming chips, Huang was busy building a complete GPU-compute ecosystem for their gaming chips with the rest of the industry.

Same applies to Khronos APIs, they behave just like that, throwing the APIs out there and hoping for the best.

No wonder they are always a shadow of what proprietary ones bring in the box.

To be fair, Nvidia hired a lot of people who at some point worked at SGI and 3dfx, so there was already a lot of HPC/server/supercomputing talent working there. There are articles (e.g. https://www.extremetech.com/gaming/239078-ten-years-ago-toda...) showing the transition from having fixed vertex/pixel shaders in the GeForce 7000 GTX series to the generalized stream processors starting with the GeForce 8000 GTX series going forward.

And it's easy to see that Huang and Nvidia put their money where their mouth was. The first GTC was 2009. That's 3 years before the famous AlexNet paper that's often credited with kicking off the current AI on GPUs trend.

Not to take anything from what you've said because Huang is really all that and more. Find his interview with Morris Chang to get some more insight. Nvidia, ATI and other players for the most part were seeded by ex-SGI crew however. SGI had an instrumental role in those companies that ate it.

The guy really is a visionary - although I gotta say he really needs to diversify his wardrobe. How long are we gonna see him in the same exact look with the black leather jacket?

His wardrobe is a meme at this point, and Elon proved memes make a great cult following.

Recent interview: https://stratechery.com/2022/an-interview-with-nvidia-ceo-je...

>He's been bang on the money on the future of this industry since he founded Nvidia

by wasting two years on Quad rendering? Wonder how much VC money got burned in NV1 and cancelled NV2 debacle.

Simple.

Rendering 3D graphics for games and the supercomputing used by AI/ML/research both need the same thing: Embarrassingly parallel math calculations with little or no branching in the code.

For example, a feed-forward neural network is just a whole lot of multiplication and addition. Transforming a 3D vertex in space to a 2D screen coordinate is matrix multiplication, which is just a whole lot of multiplication and addition. If you've already designed silicon to perform those operations in a single clock cycle, making it do HPC/SC rather than gaming isn't that big of a switch.

Aight but then why no other company managed to compete on the numerical core array ? still funny, as if the wealthy gaming market funded nvidia venture into serious computing enough to choke the potential competition.

You need huge money to develop the tech, but the HPC industry is tiny. People are always impressed at these million dollar machines, but only a handful are built every year. There is a lot more money in gaming selling millions of $200-$1000 cards every year.

Silicon is expensive to design, and even more expensive to build. The startup costs are crazy high. By the time people realized it could be a thing, nVidia and AMD already owned the market.

There certainly are some smaller companies attempting to compete on making AI/ML chips. Tenstorrent comes to mind: https://tenstorrent.com/grayskull/

Neural networks have non-linearities. There were hacks for doing some sort of general purpose computation using the GPU fixed rendering pipeline, but they're not enough for neural networks especially as understood today. You need general shaders, which were a relatively late development.

>. Who knew playing video games who amplify research that much ?

Unix was born to play a video game on a different platform. The Curses library for terminals (Windows users: "widgets" for the console") where born for Rogue.

Today, a damn serious OS like OpenBSD gives the BSD games set by default on the base install.

Text adventures for the Z-Machine had top tier grammar recognition parsers.

Then, a lot of simulation games overlapped the serious simulation software features, such as Sim City.

Nvidia is truly special because their first product was a total market miss. They pushed for Quad rendering architecture and as a result could only do _seven_ crappy Saturn game ports. Its amazing they were given 2 more tries! Cancelled NV2 doubling down on quad rendering, and finally Riva 128 almost two years later.

every single teenager whose parental units asked where gaming skills would come into use later in life as an adult. at least the were hoping for something like that.

I bet zero 90s teenager said "mooom, how do you expect to cure cancer if you stop me from buying nv chips ?? how ?"

I bet I can stack boxes in a truck better than any pre-Tetris playing person before me!

Unix was created to play SpaceWar, and now you have Linux, OSX and BSD's based on it.

And here I thought it was created to control the phone system.

Wasn’t it for editing texts?

Reply on news.ycombinator.com