I Am Graphics and So Can You
fasterthan.life
fasterthan.life
The reason I think it deserves to be called "arcane" is that, to even reach an RGB triangle (the shader equivalent of "Hello World"), you have to have dozen different things working all at once:
- Writing / compiling the shaders (vertex and pixel/fragment, at least) - Loading the shaders - Describing the vertex data - Creating vertex data - Describing constant buffers - Creating constant buffers for vertex + world/view/perspective - Filling all the buffers with data - Giving the right data for view / perspective matrices (the camera, basically) - Calling the rendering of a mesh - Getting the output to draw on the screen - + a ton of other drawing descriptions I don't even remember, like raster descriptions
For all of those tasks there is an insane amount of boilerplate that needs to be written. This is okay in itself, since it's all stuff that's worth tweaking for various use cases, but it creates a pretty insane barrier to entry. Not everyone needs to learn all the quirks of setting up vertex buffer descriptions right up front. I personally wasn't going to read a huge DirectX bible just to see how to get started.
It's a problem very much deserving of a copy+paste solution, with some clear instructions on how to tweak for individual needs, but most tutorials (MSDN or otherwise) concentrated on specific features or were in a series building on an already opinionated codebase / pipeline. And if any part of it doesn't work, and you'll be second guessing yourself and asking a ton of questions with every line of code, you have no real way of knowing what went wrong.
However, once I had a pipeline set up (and then started organizing it in a way that it managed itself and could host various shader types) I started having a lot of fun. I'm working on a 3D NES emulator [http://n3s.io] and felt super inventive when I found a way to get the models to render using the NES palette.
In the graphics world it's been the other way around. Earlier libraries were higher level, abstracted away a lot, letting you get to "hello world" very quickly. In ancient OpenGL 1.x immediate mode, "Hello triangle" is a handful of lines of code (minus the non-portable window management stuff). As we've moved to vertex buffers, moved away from the fixed function pipeline, less and less is abstracted for you and you have to deal with more and more granular constructs, more and more quirks, and more and more boilerplate. I haven't even started coding with Vulkan, but from what I know about it, any application I'd want to move over to it is probably going to quadruple in terms of line count, complexity, and "things to deal with".
On a more or less modern GPU you won't find those registers. The GPU reads a stream of commands instead of polling its registers. And the most primitive command is DRAW, which means "run the current vertex shader for the given index range, collect its output, rasterize the primitives made from the output and run the current pixel shader on the resulting pixels". So in order to draw anything it's required to have a command stream set up, two shaders set up and necessary data bound to both shaders. Naturally, there is no place for glVertex() in this setup.
There are high level libraries, which abstract the hardware but it's not the purpose of the OpenGL. It's only "abstracting" hardware in the sense that it works same on different models of GPU but it tries to expose as much of the hardware as possible in a model-independent manner.
It is a lot to learn at once, but no more than web dev where you are learning html and databases and HTTP and god knows what else at once. And like you said once you grok it, it's pretty easy.
Graphics programming terrified me for the longest time because people acted like it was dark magic and hard. Figuring this stuff out slowly but surely it shocks me how little there is to it and I wish someone would have told me sooner.
I've gotten rendering a cube down to an art (gotta start somewhere). You're talking ~100 lines of JS and double that of C++ but no more than that in any decent language. And that's a cube (or multiple) and a basic camera.
A library like regl shows how simple this stuff can get without lots of boilerplate.
https://github.com/regl-project/regl/blob/gh-pages/example/b...
Boom a triangle in 40 lines, but the underlying process of vertices, buffers, shaders, etc is apparent. Cool stuff.
Those asking about Vulkan - Vulkan only increases the boilerplate by design.
Now that I am becoming old, I have started to realize that anyone who creates a feeling of dark magic around almost any programming subject probably doesn't understand the subject very well themselves and has some sort of self-confidence issue that makes them portray whatever it is that way to people who know less than they do.
This is much alleviated when you use a wrapper around OpenGL (e.g. Rust's `gfx`), but most people start with raw OpenGL.
Ironically there's a bunch more debuggers that work with OpenGL as opposed to Vulkan(or know how to debug you're way out of a blank screen by tweaking shaders/api calls to eliminate things).
Very easy to identify things like misaligned data, etc, that would be impossible to find otherwise.
(This feature broke on my desktop, though, and I still can't fix it for the life of me, but I'm past that point in my project development anyway.)
Once you understand that stuff, everything becomes much more straightforward. You still have a lot of "paperwork" to do because you have to set up the whole pipeline to render anything, but at least then you can see the whole picture.
That was my biggest problem when I did some OpenGL work a few years ago. It takes a while to understand what OpenGL really is and what it isn't.
A lot of images need preparation and classification, there is a whole world of fun algorithms for that. If you can handle raw images in code then you can get that automated. This opens up a whole new world of possibilities that would be tedious to do in Photoshop style tools. Again with SVG in code editors you can write code that makes images in a way that is not so easy with Illustrator. Because you are not constrained to the same possibilities as the classically trained art guy you can see ways of achieving results that are creative solutions as seen by the rest of the team. Code therefore can make you apparently creative. Along the way you can work with the best creative talent with you becoming an asset rather than a cost on the company ledger.
Given the narrow definition of graphics programming I would urge others to have a go at 3D in the real world. For instance, 3D for TV might not be what it was but it is relatively easy to be the 3D guy for a client base of a few really good producers, to have great fun doing 3D graphics. Entry into that world is considered hard but isn't if you do have the technical chops. In this scenario you are gold dust, not one of many but if you go reasonable on the salary request the job is yours.
At the bottom of the 3D food chain are morally dubious things like fracking, the opportunity for someone interested in real world application of cool graphics is a good one with a low bar. There are many other industries where being able to step out the safe world of tech to embrace a non scientific discipline is the way ahead in graphics, to cut your own career.
If there is a personal hell for me somewhere, this is the definition of it. Perhaps as a compiler target, it could be usable. But on its own... not a chance.
In that vein, OpenGL or DirectX is probably a better starting option specifically because it abstracts more of the sausage-making in a way that is conceptually cleaner, especially for beginners. It won't take long to get to a point where you'll appreciate Vulkan, but it's a shallower learning curve that is still conceptually sound.
E.g. x86 assembly language is low-level, but I can start learning it quite easily. Writing "Hello world!" (either in BIOS or with system calls) is easy. Simple calculations and loops are easy. Assembly is approachable.
On the other hand, Vulkan is as explicit and intimidating as you could imagine in the worst case, and then some. Two pages of code for drawing a triangle!
There are also good high-level APIs, like three.js.
What is missing is some middle ground between them.
* Open a hardware context in the window manager.
* Describe to the window manager / operating system how to configure that context. (SDL/GLFW helps with this)
* Collect the functions and features (provided by different vendors based on OS, and hence this step is required) you require from that context. (GLEW helps with this)
* Write and load shaders. Including setting up that toolchain, writing boiler plate, choosing a language to use, choosing an IDE, etc.
* Describe a 3d object. Whether by loading, or directly in memory. Decide now about future steps.
* Load that object onto the hardware. Assuming there is enough room, etc.
* Describe to the hardware how that data is organized (in a different language of attributes, since the hardware cares about different things) that agrees with the shader.
* Describe how you would like to draw things if they aren't the "sane" defaults (often from 1990).
* Clear the screen (including describing how to clear the screen the way you want).
* Choose a shader (and describe how it should be used).
* Draw the object, based on how you configured everything earlier and the shader is written.
* Inform the window manager you are done with the hardware context and ask it to display the screen.
* Repeat the last 4 steps 60 times a second.
And that is for a zero features system. No dynamic movement, no lighting, no textures. Just a triangle or a cube sitting there. If you want easy ramp up, use a library that makes decisions about all of those things for you. In light of your assembly analogy, it's like you are trying to write an assembly program that prints hello world to a printer over the network, using it's assembly language so that it can optimize how many times a second it can print hello world locally. It's hard, because the goal is not "hello world" it's some complex structure that allows us to display "hello world" in a complex way (3d) in real time.
The difference is OpenGL when used incorrectly, does nothing. Vulkan comes back with an error and says "you forgot step 7a of describing how the memory you gave me was organized".
For printers, there is PostScript. Why there isn't a similar standardized 3D rendering protocol?
Additionally, what you have described is still 10-15 lines of code. Vulkan is still more demanding.
It's a bit hard to equate to assembly because the bulk of the complexity isn't so much "general purpose code" as "data processing pipeline." Additionally, graphics still has a very real "out of memory problem" that you can safely ignore for the most part in modern OSes and normal programming. The moment you hit "virtual graphics memory" your framerate tanks.
If you're looking for sane defaults, the answer is to use things like Unity, Unreal, etc. You can actually get quite far with their defaults and very little code. That's why I usually tell people to skip OpenGL, DirectX, or Vulkan and just go use an engine.
3d also just has an inherent complexity that at every stage you're dealing with the limitations of processing 3d data using 2d buffers. That's why any fancy effect requires you to manipulate primitives (buffers and shades).
Good abstraction layers are totally possible, like it was done for networking, 2D graphics, and other hard problems. However, with 3D, it is more of a cultural issue — the APIs were designed by hardware / driver people, who eat these kinds of complexity for breakfast. The rest of us have to suffer and adapt.
Additionally, Vulkan is increasingly pushed as a replacement for OpenGL, and this makes me cringe. It's like somebody took the worst of WinAPI and combined it with joys of developing device drivers...
There is. It's called OpenGL ;)
That said, I share your sentiment. Tossing aside the drivers and the general optimizations they offer seems to me to be akin to tossing aside your compiler because you can write more efficient assembly by hand. I think the ability to write such low-level code will always have its place, but I can't help but feel like we (as an industry) are going in the wrong direction with this.
In trying to be optimistic about Vulkan and the like, it may be an opportunity for developers to experiment with different higher-level APIs until we manage to find abstractions that are as good as the ones we have for networking (and hopefully better, even).
> it's like I am writing driver-level code, which is not my idea of fun.
Luckily, that's somebody's idea of fun :)
(In a twist of fate, the SuperBible example code only supports OpenGL 4.0+ and my primary computer is stuck at 3.3)
Most people don't care about tiled vs immediate mode GPUs when they're starting. They just want to get pixels on the screen and there's a ton of resources out there on OpenGL ES. Once you've got your feet under you(seriously, you need to understand matrices, texture formats, shaders, and a host of other things) then look at Vulkan.
From a product point of view, you don't need to sacrifice perf for the ability to run on more hardware types.
Precisely because Vulkan drivers are much thinner and do no error checking, it becomes very likely that you will end up writing code that works on one system but fails on the next.
Granted, OpenGL has some of the same issues. It's incredibly frustrating when vendor B accepts some GLSL that is forbidden by the spec, and then some application fails on the more conformant driver of vendor A because the developer failed to test properly.
The thing is, Vulkan multiplies these issues significantly. You still have a compiler in the driver, so you still get the same potential for shader incompatibilities. But you also get to manage barriers and texture transitions yourself, and boy is it likely that you'll get something wrong there. And since memory models are different across vendors (and even different between different generations of hardware from the same vendor), it is extremely likely that you end up with code that works on card A but fails subtly on card B.
There are tools that try to catch these kinds of errors, but they're far from perfect.
On the issue of memory management, I am unaware of any OpenGL issues here that make applications unreliable. Sure, drivers may not always manage memory optimally, but having an application that runs a bit less than optimally is very different from an application that renders garbage because you failed to allocate your textures with sufficient alignment -- which is something you can do in Vulkan.
So really, at the end of the day, Vulkan is about giving graphics developers foot-guns in order to reduce CPU overhead. That was explicitly its mission statement from the beginning!
I suspect it's similar to prefetching and non-temporal moves on the CPU in that way. If you're targeting a specific machine and spend a lot of time on optimizations, then using those advanced features can give you a performance boost. But then a new machine comes along, and your fancy tweaks turn out to have made thing worse -- this happened with some games when Ryzen came out, and I suspect it's likely to happen with Vulkan as well.
The main benefit of Vulkan is that it cuts CPU overhead because the driver no longer has to do state tracking. The state tracking is done (almost) entirely by the game engine, which knows precisely what is needed and can therefore do the job more efficiently. This is totally unrelated to memory management.
Though keep in mind I'm speaking from a Linux perspective. On Windows, video memory is managed somewhat differently at least in WDDM2. Still, I'd be surprised if the story is much different. Vulkan doesn't expose the details of video memory swapping.
Start from learnopengl.com and then you can try Superbible
However, even though its unpopular for many reasons, Metal is a very good compromise as a starter API. It's design is very close to Vulkan/DX12, but its complexity is much lower. It can even be said to be less complex than OGL in the way that memcpy is less complex than glSubBufferData.
No, I'm bit familiar with OpenGL, again not at the level of graphics-pros, more like overall knowledge, and my DX knowledge goes pretty much to DX8 or DX9, but instead of learning DX11 or 12, I think Vulkan might be a better choice.
I've returned my company laptop week ago, as I left it, and now I'm stuck with my chromebook + crouton, looks like Vulkan is good there (almost).
I certainly don't want to get stuck in Metal.
I wrote up a recommended path from nothing -> OpenGL -> Vulkan here https://www.reddit.com/r/GraphicsProgramming/comments/6hcri4...
I honestly haven't worked with Vulkan yet, but I hang out with pros from different companies who do and they complain about Vulkan a lot. A whole lot. Part of it is that it is very difficult even for pros to avoid undefined behavior and timing issues. Another is that the only semi-solid shading language front-end for SPIR-V is glsl. Lots of people prefer to work in hlsl and really would like to see Metal's shading language everywhere.
This article is literally advocating the opposite. 'So how did I finally break through that wall of understanding? ... "But Vulkan is the hardz" I hear you saying. "Shouldn't I start with something easy like OpenGL?" Emphatically I say NO.'