A trip through the Graphics Pipeline (2011) | Hacker News Reader

A trip through the Graphics Pipeline (2011)

fgiesen.wordpress.com

56 points1_player24 comments

This is a fantastic resource and a must read but it is quite out of date in many respects. Especially in respect to the introduction of low level APIs, mesh shaders etc etc.

It would be nice to see a modern version of this. Low level APIs largely change resource binding, state management and command execution, when they originally launched there weren't many changes to the core pipeline from say... DX11. But as you said with the introduction of mesh shaders and raytracing shaders there are two new, radically different alternative pipelines.

Updating the compute section to include WaveIntrinsics/subgroups, async compute queues, copy queue. Maybe even explore ML hardware (Tensor Cores)

I was expecting this to link to Jim Blinn's very similarly named pair of excellent books on 3D graphics fundamentals. I wonder if this title is an homage to this (older) work.

I've just started learning WebGL+Three.js. Does anyone know how similar this pipeline is to how it is done in a browser? (and how much of this is still true 9 years later?...)

The pipeline is essentially the same since what the article describes is the architectural abstraction modern GPUs implement. WebGL is essentially only a layer over the system GPU APIs (DirectX, OpenGL, Vulkan, btw the fact you are using WebGL does not mean you are using the OS OpenGL since, for example, on Windows most browsers use a SW component called ANGLE to translate WebGL calls into DirectX commands, due to the fact that the DirectX path on Windows is often more robust/performant) which are implemented by the driver, and thus are the main way to communicate with the HW and drive the graphics acceleration.

This being said, you can learn WebGL and especially Three.js without knowing the pipeline as described there, but knowing it will help you understand some subtleties or if you ever want to move to a lower level.

I haven't read this yet (and I'm not a graphics expert), but I expect this blog-series / WebGL to apply to DirectX11-era and OpenGL-era code (which still exists today).

DirectX12 and Vulkan have changed the pipeline by making it more flexible, but still largely follow the vertex -> geometry -> pixel pipeline.

Because the DirectX11 (and OpenGL) immediate-mode pipelines were used for so long, I expect that for the DECADES to come, the information in this blog will remain relevant (even if more efficient techniques advance the state of the art).

I am seeking a first principles approach to graphics. From poking a chip to manipulating bits and drawing pixels on the screen in the most raw form. Anyone know of any such books/tutorials? I need to study how the first terminals worked.

Assembly as the highest level abstraction. Anything equivalent to assembly anyway - a forth would be fine.

Maybe something like Ben Eater's video card series?

https://youtu.be/l7rce6IQDWs

If i remember correctly he assembles a gfx card on a bread board. So its fairly fundamental

I don't know how well it applies, but the Coursera course(s) NAND2TETRIS could be what you're looking for. You essentially learn how a computer works from the lowest level (logic gates), to the OS, where the final project is to make a little game. On the way, you also make a simple asslembly language, and a programming language that compiles to the assembly language.

Maybe check out this book:

https://www.amazon.com/Designing-Video-Game-Hardware-Verilog...

This one might be interesting to you: https://youtu.be/l7rce6IQDWs “The world’s worst video card?” Ben builds a framebuffer display device on a bread board from scratch, you’ll understand what every transistor does at a high level in an hour (don’t miss part 2).

That is abstractly what first principles graphics looked like some time ago - pixels were represented by a block of main memory, and the video system simply scanned that memory at the same frequency as the CRT monitor’s beam.

I’d say it’s pretty important to remember that what counts as “first principles” graphics has changed over time, and there was a time before frame buffers (the first Pong game was displayed on an oscilloscope) and the way we use framebuffers today is completely different - there is no more direct access to pixels in the same way old terminals used to work. The article here is detailing how we use a higher level API to draw higher level primitives, and the hardware takes care of the pixels.

I’d love to hear a little more about your goal... what’s driving you to study the old terminals?

One difficulty is that the "principles" aren't really pixels anymore, but things like triangles, rasterization, and SIMD programs that run. Understanding the historical context of super old "graphics adapters" might not equip you for the (quite ingenious, imo) insights that underlie their transition into the "GPU".

Not even SIMD on recent GPUs, it's SIMT.

SIMT has always just a fancy disguise on top of SIMD hardware, but don't tell the GPU manufacturers I said that :)

No one has ever been able to tell me the difference between SIMT and 1980s style SIMD on the Connection Machine 5 (CM-5).

Researchers like Blelloch have been writing in SIMD-parallel style "parallel Lisp" or "C-Star" for decades before CUDA. https://www.cs.cmu.edu/~guyb/papers/Ble90.pdf

Thread masking is one of the big differences - threads in a warp today can be enabled or disabled independently to enable limited amounts of divergent flow control, and conversely super cheap syncronization. I don't know what the CM5 did - was it limited to always-on sets of vector instructions? Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model.

> I don't know what the CM5 did - was it limited to always-on sets of vector instructions?

EDIT: Woops, I mean CM2. CM5 was MIMD and a slightly different architecture. I've edited this post to say CM-2 instead, but my previous post about CM5 will remain in error.

Individual thread masking was available on CM-2, for all 4096 cores that were executing in SIMD-parallel.

CM2 was a 4096 x 1-bit SIMD processor. Very limited compared to modern GPUs, but the execution model that CM2 experimented with eventually became the modern GPU architecture. Yes, you can have effective execution masks even with only 1-bit cores.

EDIT: See the C-Star manuals, a "per-thread if statement" required a "where" construct instead of if. But yes, the "where" statement performed similarly to "CUDA if": http://people.csail.mit.edu/bradley/cm5docs/AReferenceDescri...

The MIMD thing that CM-5 did didn't seem to stick. CM-2's SIMD execution seems to be a bigger influence. But C-Star ran on both... and it was the high-level languages like C-Star or Parallel-Lisp that influenced GPU languages like CUDA.

> The where statement is involved with setting the context, a process known as contextualization. The context is a parallel boolean mask (i.e., each element of it is true or false) that controls the execution of parallel operations position by position. A different context is associated with each shape object, and the context associated with the current shape is always applied to operators.

A CUDA-block was roughly equivalent to a CStar-Shape. A CUDA-if is roughly equivalent to a CStar-where.

> Current GPUs also support certain amounts of fully independent thread scheduling, e.g., per-thread instruction counters, etc. Maybe that is beginning to leak out of the SIMT model.

NVidia was calling SIMT back in 2010, long before per-thread instruction counters were available in Pascal (GTX 10xx series of GPUs)

Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison from someone who's used both C-star and CUDA, about the plusses and minuses of each... have you used both?

> Very limited compared to modern GPUs

Yeah, I'd expect the real true differences between CM5 SIMD and Pascal SIMT is just buried in the subtleties of those limitations compared to today's hardware, and that fundamentally they aren't super different ideas. In my mind, that doesn't mean that SIMT is meaningless or the same thing as SIMD.

> NVidia was calling SIMT back in 2010, long before per-thread instruction counters were available in Pascal (GTX 10xx series of GPUs)

Right; I'm saying the "SIMT" of 10 years ago has to do with masking and maybe other per-thread control (even though the Connection Machines might have had it as well), and that per-thread counters are now going beyond what we used to know as SIMT, perhaps deserving of some other acronym. Or maybe SIMT is becoming more appropriate and more differentiated from SIMD over time? Is it possible that CM5 should be called SIMT and we simply didn't have that acronym at the time?

> Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison from someone who's used both C-star and CUDA, about the plusses and minuses of each... have you used both?

Nope. I'm looking at C-Star through a historical lens, I was curious at the overall development of CUDA and was researching the "line of influence", so to speak.

> Right; I'm saying the "SIMT" of 10 years ago has to do with masking and maybe other per-thread control (even though the Connection Machines might have had it as well), and that per-thread counters are now going beyond what we used to know as SIMT, perhaps deserving of some other acronym. Or maybe SIMT is becoming more appropriate and more differentiated from SIMD over time? Is it possible that CM5 should be called SIMT and we simply didn't have that acronym at the time?

CM2 was just another SIMD machine, there were plenty of others before CM2 (its just that CM2 was one of the more popular ones). IIRC, Cray had similar SIMD machines that competed against them. The overall SIMD-methodology reaches back until the 70s at least, maybe earlier.

Just noting that CM5 was MIMD at the lowest level, trying to correct a mistake I made a few posts back.

------------

I think what "happened" was that Intel adopted SWAR: "SIMD With A Register" when Intel created the MMX / SSE / AVX instruction sets. (compared to dedicated SIMD-computers like the CM-2).

At some point, Intel's SWAR approach became colloquially known as SIMD (even though SWAR was much more limited compared to "proper" SIMD computers like the CM2). NVidia creates a new acronym called SIMT to differentiate Intel's SIMD (aka: SWAR) from a proper SIMD machine, even though the 80s-style SIMD was substantially similar to NVidia's new SIMT term.

just a few minor comments:

the cm-1 and cm-2 both supported up to 64k single bit processors, and as many 'virtual' processors as you wanted until their memories got too small

the cm-5, while it did have a potentially mimd model, each mimd node was a sparc that was lashed to 4 simd vector units, with their own memory bandwidth. so you would be giving up a lot to just run on the sparcs. it was also pretty clumsy - I think there was just a message library one called from C?

I think that its not entirely fair to compare the CMs to the Cray vector products. although somewhat similar, the programming model for the Crays was basically loop mining fortran and the CM really presented a model of 2^n fine grained processors. the precedent I always heard was the Iliac

Note that Blelloch's favorite(?) primitive- the parallel prefix operation was used quite widely in the CM world, but equivalent 'horizontal' operations are pretty much completely lacking on the Intel/AMD instruction sets. Haven't looked at the GPU ones, but its absence is pretty frustrating

I've written and open-sourced a book (soon to be published by No Starch Press) that might be close to what you're looking for. It doesn't poke chips directly, but drawing a pixel to the screen is the highest-level abstraction it uses (and builds on top of it).

http://gabrielgambetta.com/cgfs

You could also check out James Bowman's "Gameduino" line of graphics boards [0:2] (which includes a Forth-programmable CPU, the J1 [3], implemented on FPGA)

[0] https://excamera.com/sphinx/gameduino/

[1] https://excamera.com/sphinx/gameduino2/

[2] https://excamera.com/sphinx/gameduino3/

[3] https://excamera.com/sphinx/fpga-j1.html

Maybe this would be a good start! https://github.com/ssloy/tinyrenderer

It covers how graphics libraries like OpenGL go from triangles to screen coordinates, and how they "shade" pixels in those triangles to create an image.

On the hardware side (there are plenty of public resources about implementing graphics in software for the ground up), Jaymin Kessler's Jaystation2 and Jaystation3 dev blogs where he makes a GPU that runs on a FPGA might be useful http://maisonikkoku.com/. This course also looks very useful from the hardware design standpoint https://courses.cs.washington.edu/courses/cse467/15wi/. GPU hardware design, unlike CPUs, seems to be much more esoteric with very little shared info due to the key players in the space and possibly all the patents.

For graphics programming in general Graphics Codex looks really good (I haven't done the course, but I know people that have). https://graphicscodex.com/

Reply on news.ycombinator.com