Writing a bindless GPU abstraction layer | Hacker News Reader

Writing a bindless GPU abstraction layer

kevin-gibson.com

71 pointssurprisetalk12 comments

Relevant: https://news.ycombinator.com/item?id=46293062

Someone has started to implement Aaltonen’s “No Graphics API” idea in Odin: https://github.com/leotmp/no_gfx_api

Looking at the code examples (e.g. the cube[0]), I can’t get away from the idea that this looks very similar to standard OpenGL bindings, just with a different style of boilerplate.

At some point (for basic rendering examples at least), you’re always going to have to define a data layout of some sort, have that reproduced or reflected between the cpu/gpu environments, and iterate through each vertex. How exactly that turns into GLSL’s in/out variables feels to me like a relatively unimportant implementation detail that should be assessed on performance alone rather than usability[†]. I wonder if what we’re ultimately heading toward is better shader compilation, directly to SPIR-V, to support both bound and unbound use cases.

Sort of related, but Gabriel Dechichi’s engine [1] reflects shader vertex layouts back to C, so you get compile-time error handling if the struct layouts don’t match between cpu/shader.

[0] https://github.com/rkevingibson/loon_gpu/blob/main/examples/...

[1] https://m.youtube.com/watch?v=NTuLfB2ex5Q&ra=m

[†] Edit: Specifically talking about the final compilation unit here, not the shader language or library interface

> unimportant implementation detail that should be assessed on performance alone rather than usability

I disagree. Usability is important, or you end up with garbage like Vulkan. I'd rather have 99% of the performance for 100% usability, than 100% of the performance for 5% of the usability.

Unfortunately when it comes to graphics APIs, the choices are between bad and worse, which is why I switched to doing real time rendering in CUDA. Easier to write a fast software rasterizer in cuda, than to draw a single triangle in Vulkan

Sorry, I wasn't clear (and have edited the post), I absolutely do believe that usability is important, and to be honest I feel like it's the main problem with graphics APIs (at least for my skill level).

Where I'm at now is that Vulkan is great because it's removed so many of the hard-coded assumptions that were previously hidden in the driver layer, allowing much greater CPU and GPU utilisation and performance when you need it (if you can get past the ugly APIs).

Where the usability stuff comes in is that it allows for the creation of alternative, higher-level shader-reflection setups that work brilliantly for simple use cases, all in user-space, which can open up shader programming for creative coders and a broader spectrum of developers. The lower-level APIs are then still accessible to those who absolutely need those performance levers.

Generally I agree, but the bindless approach can give you both ergonomics and performance in some cases (not all). Certainly it can reduce CPU overhead - passing a single pointer to a shader is much faster than heavy bind group updates. The trade off is that you might have more pointer indirection on the GPU, which can introduce stalls.

Being able to use pointers and u64s for resources also makes having a shared header between shader and c++ source easier, though you still probably need some macro trickery to smooth over language differences. Metal is really good for this since the shading language is closest to C++.

> but the bindless approach can give you both ergonomics and performance in some cases (not all). Certainly it can reduce CPU overhead - passing a single pointer to a shader is much faster than heavy bind group updates. The trade off is that you might have more pointer indirection on the GPU, which can introduce stalls.

Yeah I'm totally with you there, it just feels like the interfaces aren't that pleasant at the moment (although I've not tried Metal)

Edit: didn't realise you were the project author, I think it's great by the way!

These attempts at better GPU abstraction layers (starting from the Sebastian Aaltonen article and continuing with the Loon GPU API in the linked article or the Odin-based no_gfx) all have the issue that they're building on top of the current jank APIs we have.

Basically, they're all still concerned with the realities of current day GPU hardware, which in turn means they too, much like DirectX and Vulkan, will eventually become "bad" abstractions.

IMO, if there is any mention of a "vertex" or "fragment" shader, it's still not there yet. End goal should be massively parallel CPU, no graphics-specific functionality beyond compiler intrinsics for whatever can't be done efficiently in software. Something like CUDA is the target to aim for.

I still have fever dreams about "near" and "far" pointers ...

I last wrote 3-D rendering code using hardware in 2005 or so. It was DirectX 11 or 12, I don't recall. I thought it was quite a mountain to climb. For my part, I grew up _implementing_ rendering pipelines, so I always had this sour taste of needing to learn someone elses (Microsoft, in this case) idea of how to render 3-D scenes -- using their concepts, even if they were industry standard (shaders etc). Meaning that somewhere inside their APIs are fundamental abstracts I _do_ find, well, accepted terms. After all, I am not arguing against using integration in maths as being useful. There's practicalities.

What am I getting at?

Well, if you take the "no graphics API" to its logical conclusion, then we should strive to remove everything that is "graphics" from it altogether. Reading the article linked and the article the former links in turn ("No Graphics API"), there's still an ungodly mix of graphics rendering _specifics_ and I think these too will start rotting "sooner rather than later".

Having grown up on matrix operations and optimisations that often venture into territory where they _lose_ resemblance to outright _graphics_ pipeline they originally modeled, I am of the firm opinion the logical conclusion I am advocating for, is a sensible one.

To explain with an analogy -- I think we should adopt the "exokernel" approach to GPU hardware. That means just exposes the hardware constructs without trying to fit these into any form of a common denominator (between AMD, NVidia and Intel), and have software specific to each glossing over the abstractions.

Before you say "well, that's exactly what DirectX, Metal and even Vulkan, did!", the key difference is not the encapsulation itself, but its form, the "how" of it.

An exokernel, for instance, does not expose or provide a memory manager (referring to use of MMUs), or give you a `malloc`. It simply exposes the paging mechanism, and you take it from there. If you need a high-level API and/or cannot be bothered to page things in your little command line application -- a fair concern -- you rely on middleware, aka "libOS" in the original, MIT's terminology, a library by any other name -- to provide `malloc` so you don't have to cry over the bare-bones exokernel.

The point I am making is that as the exokernel software (and libraries that would complement it) just exposes the GPU hardware, there is more flexibility to how to use it without "cramming round shapes into square holes" -- a bane of graphics programming _and_ especially optimisation there, that has plagued the entire field since 3dfx' days. Library authors can wrangle API design without touching the hardware or working _around_ it, while hardware vendors can tune subsequent chips to generic computing -- it's the way it's going anyway, except that they won't have to coordinate closely with software writers -- the dependency will go strictly one way -- from library to "GPU exokernel" authors to hardware manufacturers, and never the other way around (ideally). Every link in the chain has more flexibility then.

Practically, we're talking about a generic massively parallel processing unit with tons of addressable memory of different latency (system RAM, on-board RAM, on-chip RAM / L0-level stuff) exposed with a zero-cost abstraction layer known as an "exokernel". DirectX / Vulkan etc compatibility implemented strictly in software over the "exokernel". Those who want their retained mode or whatever else that doesn't require them to deal with cache thrashing issues, can use that, while those who like to holistically optimise and/or Unity / Unreal Engine developers proper, can use the exokernel by writing libraries for it, or outright affect the exokernel itself as new hardware is put on the market.

Sorry for the wall of text. This is from someone who started with implementing solid-painted triangle and polygon rasterisation, then Phong and Gourad shading using an Intel 386...

P.S. This isn't AI slop (although you're obviously free to feel it reads like one), I just like to use the em-dash.

You could try and do this today, to an extent. You could skip Vulkan and talk to the kernel driver in your app directly. vulkan-1.dll is regular user-mode code, there's nothing special about it. Nobody does this because it's a colossal amount of work for a small pay off. Your code becomes completely un-portable.

Exposing the hardware primitives directly is not an abstraction, so in reality all you end up having to do is implement your own Vulkan instead if you want to support more than 1 GPU. And your app won't work on new GPUs without shipping an update.

If you want a common API to target all GPUs you get Vulkan again. GPU hardware is too diverse to go much lower than Vulkan without just moving the hardware abstraction into your app instead.

I think you missed the main point of my argument -- you, the game developer, aren't programming the hardware directly or through the kernel -- you are using a common abstraction in a competing set of abstractions that don't require their vendors to beg for crumbs from the likes of NVidia because the dependency only goes the other way -- NVidia publishes an "exokernel driver" that merely exposes the capabilities of the hardware without assumptions about how to best use it. Fundamentally it's not even needed beyond a _specification_ since the hardware is fundamentally exposed through ports and addressable memory mapping(s), but since Nvidia does closed-source and closed-hardware the latter is not going to happen so kernel driver it is. You, the game developer, don't use it directly, you rely on middleware (like Vulkan) and/or middleware on top of that middleware and so on -- until you are left with an API you are comfortable using.

Indeed, Vulkan is the step in the right direction because it largely follows the path I was outlining, but it's still got ways to go because it assumes things and mixes in abstractions of its own, but more importantly because through its mere existence and the way how it is positioned it occupies space where nothing else has room to exist. Point being that Vulkan still requires you to play by someone else's rules of how they thought the API should work, which is detrimental in principle to "exokernels". So this is about exokernels, not about going full retard and programming a 100 billion transistor GPU on your own to draw pixels.

In case I wasn't able to elaborate: Vulkan looks like the part, it's certainly a sign of the people who started with the likes of DirectX have learned about the dead-end model they thought would work and are now finally seeing the "light", and I am arguing that whatever abstracts the GPU should just pretend it's a generic massively parallel computation unit with addressable memory of multiple kinds and places and so on -- without assuming it's for "graphics" (which it is less about today than ever before, what with AI etc).

Reply on news.ycombinator.com