Hacker News Reader

After working with OpenGL for many years and then moving to higher-level systems more recently (e.g. three.js), I found a useful way of viewing the 'essence' of the API: moving data around. It's not so much a '3D graphics API' as it is a wrapper for interacting with a a GPU.

(A notable exception is shader code, which is largely doing custom geometric or pixel transformations. But the rest of it is pretty much an API for moving blocks of data in and out of the GPU.)

I think this is important to know, because the common perception is that OpenGL, as an API for creating 3D graphics, would give you primitives for talking about 3D objects/spaces—which for the most part it doesn't, aside from essentially utility classes (e.g. vectors/matrices for working with 3D spaces). In consideration of that, it's a pretty awful experience if you're interested in describing 3D scenes, as opposed to developing new rendering techniques or optimizing old ones or whatever.

So instead, it's an API well-suited to building actual 3D graphics APIs on top—scene graphs, game engines, etc. My understanding is that Vulkan/Metal take this to a further extreme, and I see it commonly mentioned in that context—but it's true of OpenGL too!

It should be more common to warn beginners: learn OpenGL so that you can understand the underpinnings of some other system you will eventually use to describe 3D graphics—but this isn't the endpoint, don't build all your games or whatever directly on OpenGL. You will have so much more fun with a good scene graph.

> I found a useful way of viewing the ‘essence’ of the API: moving data around. It’s not so much a ‘3D graphics API’ as it is a wrapper for interacting with a GPU.

It is very true that most of the WebGL functions just move data in & out of the GPU, but to me this feels similar to saying that WebGL is a 2D API. The thing is, all APIs essentially move data around, if we break it down. It feels maybe too reductionist. But it depends on your goal, right?

For a student who wants to know how WebGL works, it is helpful to break down WebGL into components: move data into GPU, rasterize into buffer, view or optionally copy data out. Where’s the 3d? You get support for 3d data & operations, but (unlike OpenGL 1) you now have to design your 3d->2d projection yourself.

For everyone else, for people who want to know what WebGL is for or why it was built, and for people to want to know how to make 3d graphics, it’s not that helpful to say that WebGL is 2d or that it moves data around. It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser.

In any case, it’s problematic to insist that either interpretation is the complete picture, or that one of them is wrong, agreed?

> So instead, it’s an API well-suited to building actual 3D graphics APIs on top—scene graphs, game engines, etc.

Or, OpenGL/Vulkan/Metal/WebGL are actual APIs for 3D that are good for building scene graph APIs and game engine APIs on top of? These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed.

> The thing is, all APIs essentially move data around

Technically true, but misleading to bring up. OpenGL is different because the primitives in it are explicitly about collections of data of specific types. Whereas an API that was 'about' 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time.

> It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser.

I disagree. I think for almost everyone's needs, the API they need is something more like three.js. If you are writing a new scene graph or whatever, then WebGL is what you're looking for.

More importantly though, it's about managing expectations of new users: yes, you can use OpenGL/WebGL to create moving 3D pictures—but it's going to be a linguistically awkward, and yet (potentially) performant, way of doing it.

> These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed.

To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs. I'm saying they are essentially hardware access layers with some special utilities for doing graphics stuff. They need to exist, and they should not be doing irrelevant things like providing scene graphs.

> OpenGL is different because the primitives in it are explicitly about collections of data of specific types.

Different than what? Three.js? OpenCV? Firebase? Instagram? All of these can be described as being explicitly about collections of data of specific types. This isn’t differentiating, you’re still describing all APIs.

> Whereas an API that was ‘about’ 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time.

That is what WebGL provides. You’re trying to split a hair that doesn’t exist. I realize you’re trying to talk about the difference between immediate mode and retained state graphics APIs. Both are “about” 3D, and retained mode is not more 3D or more “about” 3D than immediate mode.

I’d argue the other way around. Three.js is built on top of WebGL, it doesn’t do 3d graphics without WebGL, and the only thing it adds is retained mode state, i.e., a specific type of data. Three.js doesn’t do it’s own drawing, and it doesn’t add any 3D to WebGL that wasn’t already there, it only adds management of some host side memory so it can call WebGL on the same data repeatedly. By your own metric, three.js is less “about” 3d than WebGL.

Anyway, none of this really relates to the original point. Saying that WebGL is about data is neither supporting the argument that WebGL is about 2D graphics nor refuting the argument that WebGL is about 3D graphics.

> To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs.

Right. They shouldn’t because their purpose is 2D or 3D drawing in any form, not management of a node graph or scheduling of draw calls.

> All of these can be described as being explicitly about collections of data of specific types. This isn’t differentiating, you’re still describing all APIs.

Maybe my usage of 'explicit' was unclear. Here is an example of a primitive which is explicitly about storing/moving collections of data, independent of what that data is used for: glBufferData, glClearBufferfv, glGetBufferParameteriv, glMapBufferRange. Firebase and OpenCV I could maybe see having explicit data constructs; Instagram sounds hugely unlikely. And saying that three.js is explicitly about moving data around is ridiculous: it's core is about 3d graphics concepts, like coordinate systems, geometries, materials, and cameras. The only time where it talks about raw data in the same way are a small number of classes that need to be extremely efficient, so they compromise and break the abstraction to WebGL, making reference to its primitives.

> Three.js is built on top of WebGL, it doesn’t do 3d graphics without WebGL

Not at all true. It's architected to be independent of WebGL. When you use it you can construct a WebGLRenderer or a CanvasRenderer and get 3D graphics with either (obviously WebGLRenderer is going to be far more efficient since it uses the GPU, but it is in no way 'more 3d').

> Saying that WebGL is about data is neither supporting the argument that WebGL is about 2D graphics nor refuting the argument that WebGL is about 3D graphics.

I should have clarified the connection: I saw the argument that it's about 2D graphics is being one way of getting at a more significant point: it's not really about 3D graphics as much as people typically think. Which was my shared point.

> It's not so much a '3D graphics API' as it is a wrapper for interacting with a a GPU.

Mandatory link to Zed Shaw's blog post on the difference between an 'indirection layer' and an 'abstraction layer'.

Abstraction is used to reduce complexity. Indirection is used to reduce coupling or dependence.

* https://zedshaw.com/archive/indirection-is-not-abstraction/

Hmm, but I didn't say it was an indirection layer nor an abstraction layer...

If I'm understanding you and Zed correctly, you pretty much did, you just didn't use Zed's terminology.

As you say, the intent of modern graphics APIs like Vulkan isn't to spare the graphics programmer the hard work of writing low-level code. Instead the intent is to provide a minimal indirection layer which does just enough work to let graphics code work on any vendor's hardware, whilst preserving the graphics programmer's ability to do low-level cleverness.

So game-engines and scenegraph systems are in the business of abstraction, but graphics APIs are in the business of indirection.

Okay, I see your point. But indirection only enters the picture once you introduce the possibility of multiple graphics cards which are being communicated with the same API—and I agree, that is an example of indirection.

However, it's not the role I was attempting to highlight. I was considering the role of OpenGL/WebGL in relation to a single graphics card: if you consider that situation, the API is still necessary and what it's doing is neither (much) abstraction nor indirection. Instead, it's main task is essentially conversion: the graphics card needs data in one format, javascript programs assemble their data in another (I'm using data very loosely here; I mean to include even the program source, since it's in a 'format' we can just give directly to a GPU).

That said, even something like Vulkan does abstract a bit, and once you bring in the possibility of multiple cards, the value of its role as an indirection layer becomes clear.

> This is because OpenGL is at its essence a 2D API. The vertex shader accepts something as input and it produces 2D vertices in device coordinates (-1 to 1) as output.

And the same can be said for every other GPU API. That's what rasterization is. You take a bunch of input and put out a 2D texture. It's necessarily 2D because screens are 2D.

Luckily, first class support for matrices, 4D vectors, mipmaps, lighting (well, at least insofar as lighting is math), etc make it well suited for creating the illusion of 3D.

I get paid to work with OpenGL -- and, somewhat uniquely, 2D OpenGL. I'm confident in saying that while OpenGL might suffice for a 2D game engine and yes, might allow you to skip over a little bit of the math, it is terrible as a 2D UI engine.

GPU APIs are first and foremost for rendering 3D scenes. You can certainly use them for 2D but how well suited they are to that is dramatically dependent on what the use case is.

Do you think other gpu apis (vulkan, metal, direct3d) are better suited for 2d?

Direct3D alone not, but DirectX as a whole yes.

Same applies to Metal.

OpenGL suffers from "go fish libraries" mentality, where we only get a way to put pixels on the GPU and nothing else.

While Metal, DirectX, LibCGNM, NVN, GX, GX2 provide a complete framework around their APIs.

You get a way to do math, shader loading, dealing with fonts, textures, materials, scene descriptions, mesh loading, shader management.

If I had a penny for every OpenGL tutorial that teaches how to manually load, compile and link shaders, or which libraries I am supposed to hunt down, I would be enjoying some nice vacations.

Sadly OpenGL as 2D UI is Google's answer for anyone that wants to do 2D rendering from the NDK.

Which means everyone ends up writing their own little library on top of OpenGL, or increasing their APK size bringing in SDL, Cocos2D, ....

Or if 2D rendering performance isn't a constraint, calling via JNI the Canvas API, which happens to use Skia for its native methods, but isn't exposed as NDK API.

I think it can be a helpful way to think about it for some beginners, but yeah, it's not strictly true. I'd say "the goal of 3D APIs is to display 3D images on your 2D screen", and that means that, at the far end of the pipeline, after you've projected and flattened everything, you have 2D concepts: triangle rasterization and fragment shaders operate entirely in 2D, for instance. The only remaining "3D"-esque thing, which is perspective-correct attribute interpolation, happens in screen space.

That's cute but it's also about as relevant as saying cooking is just chemistry. You're not wrong but you're also ignoring the artistry, taste, tradition, image, class, and every other aspect of cooking.

If at the end of the day cooking is chemistry, well then serve me up some atoms.

All I'm saying is that it's a way of thinking about it. It clearly helped the author think about "rasterized 2D screen-space triangles" rather than "something something 3D". They clearly had some misconception and "WebGL is a 2D API" helped clear it up.

I'm not saying it's the one true way to think about 3D graphics APIs, or that it's wrong or right.

The point kinda is that Webgl is closer to the chemistry than the cooking: It's a fairly low level API. It doesn't offer you a scene graph or otherwise let your work with 3D objects. It doesn't care if the vertex shader input is 3d coordinates, or anything else. It doesn't handle the camera for you, and so on.

It offers you the essential elements you need to compose 3d abstractions (or you can use it for something else), or for a library to run on top of. Or you can choose to use it directly, but then you are in effect cooking with a chemistry set.

For a pretty amazing example of the 3D work you can do with WebGL, see N64 VR with Javascript https://medium.com/webmr/n64-vr-with-javascript-e188de42ced5

How does this connect to the parent comment??

If WebGL were a 2D API, it would have made it impossible to do this kind of work. Also VR in general.

It would always be possible to project a 3D world onto a 2D plane in any 2D graphics API, which is exactly what the projection matrix does in OpenGL.

The projection matrix transforms the 3D world into a 3D world in screen space. Two of the dimensions are the screen coordinates, and the third is depth into the screen. The depth is used for Z-buffering (hiding the stuff in back), and for fog and focus effects.

Depth in OpenGL is a separate 2D buffer. You can use it to approximate the effects of 3D space, but it’s still fundamentally a set of 2D operations.

Of course, it’s all fundamentally just bits on a heap, so past a certain point the argument becomes academic.

That, I sort-of agree with. Layering isn't unique to 3D, so it's debatable whether the incorporation of a depth buffer makes OpenGL a 3D API.

It's not layering. It's depth. The depth buffer will work for cases where two objects cross in Z. Unlike the painter's algorithm.

I would agree if it wasn't for the perspective corrected interpolation that happens in the w/z component for vertex properties[1].

You're not going to get that in a 2D rasterization API without doing a per-pixel calculation of each rasterized pixel(due to non-linearity) which would defeat the point of a 2D API.

[1] https://stackoverflow.com/questions/24441631/how-exactly-doe...

You get that in plenty of 2D environments though. If you do a perspective warp in Photoshop you can get it, and CSS transforms (for HTML content!) will do it if you rotate around the x or y axes.

That's because all of those are a perspective projection that flattens a 3 dimensional frustum into a 2 dimensional plane.

It's kind of like saying that your eyes can't see things in 3 dimensions because all of the light is focused only on the back plane of your retina.

That's the kind of argument I would simply discard, and it is only a matter of interpretation and can provide no benefit whatsoever..