An History of Nvidia Stream Multiprocessor | Hacker News Reader

An History of Nvidia Stream Multiprocessor

fabiensanglard.net

161 pointsingve34 comments

Fabien, how come you skipped Volta? (Presumably because it wasn’t interesting for Display, but it’s a huge step up from Pascal)

The most interesting thing to me in the progression of NVIDIA’s compute offerings is that until Volta, the “SIMT” claims were just a programming model and the hardware was actually just like any other SIMD hardware (though with automatic mask stacks). In Volta, they finally added “we can run different ops at different program counters”.

For the casual reader, the hot chips 2017 presentation is probably the easiest skim:

https://www.hotchips.org/wp-content/uploads/hc_archives/hc29...

> Since Intel proved that there is still room for miniaturization with the 7nm of Ice Lake, there is little doubt Nvidia will leverage it to shrink its SM even more and double performance again.

Intel calls their process 10nm and it's weird to mention it since it was a total disaster.

> This fragmented design reminds of the Pre-Tesla layered architecture, proving once again that history likes to repeat itself.

So true.

On the Hardware Wheel of Reincarnation [1] we are definitely swinging back into the realm of special-purpose processors and processor extensions all over the place. The gigahertz scaling free lunch has been dead for over a decade. We're still adding more transistors, but finding it harder to speed up day-to-day serial code. So, instead we're adding dedicated hardware for the common, hard stuff and throwing special-purpose chips into every nook and cranny of everything around the periphery of the main thread on the main CPU.

[1] https://www.computerhope.com/jargon/w/wor.htm

We are also inventing new ways to do computation. Massive deep neural nets require different hardware, and it seems there’s plenty of room to improve and experiment there.

The SVG doesn't render at all right in Firefox, for me.

Thanks for telling me, I had no idea. It looks like Firefox and Edge don't render in the same way Inkscape/Chrome do.

It seems converting the whole SVG from "Object to Path" fixed the issue everywhere.

I know next to nothing about SVG; do you think this was a mistake in the SVG, or is it a bug in Firefox's rendering? If the latter, ideally, we would file a bug report, although i don't actually know how you do that for Firefox these days.

It think it is a bug in Firefox rendering since Inkscape and Chrome render it properly.

If I remove the "writing-mode:tb-rl" firefox renders it properly. It seems FF does not support that attribute.

I have kept both versions of the SVG and intended to fill a bug report.

http://fabiensanglard.net/cuda/g71.svg http://fabiensanglard.net/cuda/g71_org.svg

I have this weird guts feeling it will be closed as "we know. we don't support it yet. Come back later". So I abandoned the idea.

Interestingly, a documentation overview says that tb-rl is an allowed value:

https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/P...

But the detailed documentation it links to does not:

https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/w...

I had a look in Bugzilla and couldn't find anything specific to tb-rl, but did find an ancient meta bug for writing-mode which links to millions of other bugs:

https://bugzilla.mozilla.org/show_bug.cgi?id=writing-mode

I wish I understood more of this, what a cool field of engineering.

I'm curious about the addition of Tensor Cores. It sounds like both the regular Shader Units and Tensor Cores can carry out GEMM, just that the latter are more efficient at it. I'm guessing that for ML applications, both the Tensor Cores and Shader Units are used.

Are tensor cores idle during 3D gaming? How do they differ from the regular Shader Units? Why are they more efficient?

The tensor cores are basically more efficient because they don't implement a 'read from register file; execute one instruction; write to register file'-style architecture:

They are basically a big array of ALUs hard-wired in a grid to do a matrix multiply: The connectivity between ALUs is baked into silicon: no general purpose register file to hold the intermediate results, just wires directly from the output of one ALU into the input of the next (well, and registers for pipelining).

No overhead for register files, instruction decoding, instruction scheduling, load/store units, cache, etc. (load/store is handled by the usual SM instruction set).

This is basically the same concept you find in any "AI acceleration" hardware (Google's TPUs, various neural-thingys you find in Phones, etc).

As for using in games, this hardware is generally idle, except for a few cases (typically because Nvidia did the work to light them up):

- upscaling from lower resolution rendered products to higher resolution (Nvidia markets this under the name "Deep Learning Super Sampling"), which is basically taking a derivation Temporal Anti-aliasing and mapping it onto the tensor cores to exploit that hardware

- Doing CNN-like denoising and reconstruction of a noisy ray traced image.

> Are tensor cores idle during 3D gaming?

Nv is making a use for them via https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling

> How do they differ from the regular Shader Units? Why are they more efficient?

They basically implement a special-purpose, single-instruction 4x4 matrix multiply operation with 16-bit floats. https://www.anandtech.com/show/12673/titan-v-deep-learning-d...

> Are tensor cores idle during 3D gaming?

The tensor cores are used for real-time raytracing, ML-based anti-aliasing during gaming, and other features. The Turing whitepaper has a lot of information on that.

https://www.nvidia.com/content/dam/en-zz/Solutions/design-vi...

A more generic answer to this question is that everything NVIDIA puts in their GPUs is for the gaming market. Other use cases are incidental.

I think the ray tracing cores may be separate, they accelerate BVH traversal and intersection.

The tensor cores are used to denoise the raytraced result.

Between the 30% framerate hit and the (to me) barely-perceptible visual difference, real-time raytracing still feels like a gimmick.

The new Minecraft stuff is the very first time I've seen raytracing produce graphics that make my brain conceive this could be a "realistically lit" scene, and it doesn't even take advantage of things like significant amounts of reflection and re-reflection. HOWEVER, I see zero benefit to the actual game from that.

In the case of Minecraft no new graphics feature is really going to benefit the "actual game", but players who obsess over their created spaces might really appreciate having better lighting that they can control through things like reflectivity. It's something you'd have to poll the average teen-who-still-plays-Minecraft about though.

Have you tried the many advanced shader packs that've been available for years and don't need raytracing?

https://youtu.be/JJeb1eEZYhI

If you haven't seen any screenshots of it at work in Remedy's Control, it's pretty nice there. Nothing groundbreaking but it's the first time I went 'oh!' and realized that I was seeing stuff that would've been invisible without raytracing. It's used for shadowing, reflections on glass, and a bunch of other stuff that ties the scene together and ensures things are where you expect them to be.

I expect we'll see some more compelling uses of it in the next few years, given that it took until the last year or so to see games start using it in a way that wasn't just tacked on. (It makes sense that Control was one of the first titles to use it extensively, since they were publishing talks about GPU raytracing well before the hardware was in anyone's hands)

We're in an awkward transition period where hardware raytracing is now available but not very useful for most people as the hardware is still niche and getting the most incredible results from it doesn't run at 60+ fps. But, if we ever want games/VR/AR/etc. to match the photorealism of modern visual effects, full-blown raytracing / path tracing will be required. It's going to take a while but it will eventually happen. The results will look far different than the before/after screenshots in the Turing paper.

Also, once the hardware is more ubiquitous (e.g. in consoles), some of the features provided by raytracing will make the game development process much simpler even if the results are only marginally better for some of the presented hybrid use cases.

The title led me down this somewhat interesting rabbit trail of "why an?": https://www.merriam-webster.com/words-at-play/everything-you...

The same here. Been meaning to sort it out for myself for some years now. Finally done that: "A history ..." is how I was taught, and it shall remain that way.

Fixed. Thank you!

Oh, sorry, wasn't saying it was wrong. People seem split on it. Just curious.

Nice comparison. If I could offer one bit of feedback: I think that graph at the end should be TFLOPS rather than GFLOPS.

Good catch! Fixed.

One of the things I had a fit of laughter was after reading that Jensen used to work at AMD before!!

I heard from a first-hand account that in the late nineties Jensen tried to sell nVidia's IP to ATI. They refused, even though the offer was good. The rest, as they say, is history.

Didn't he want to be made head of ATI at the time though, as part of the deal?

That was AMD. They wanted to merge with Nvidia for their graphics IP, but didn't want to accept the condition, so they bought ATI eventually.

He's also Lisa Su's uncle.

Reply on news.ycombinator.com