jms55

Typical rasterized pipelines with deferred rendering:

1. For each light, setup a virtual camera at the light and 1a. For each mesh, rasterize depth to a texture for that light (the shadow map) 2. For each mesh, rasterize material properties and depth to a texture for the main camera 3. For each pixel on screen 3a. For each light in the pixel's bin: sample the shadow map several times to determine if the pixel is in the light's shadow, and if not, apply the lighting for that light

By contrast, megalights:

1. For each mesh, rasterize material properties and depth to a texture for the main camera 2. For each pixel, pick a random light influenced by how close it is, how bright it is, etc 3. For each pixel, raytrace to the light. If you hit another object before the light, then you're in shadow. If not, the light is visible and affecting the point, so write the light to the direct light texture. 4. For each pixel, denoise the direct light texture by essentially blending spatially (with nearby pixels) and temporally (with pixels from the previous frame's direct light texture). 5. For each pixel, sampling the material properties and the denoised direct lighting texture, apply the lighting to the pixel and write out the final result.

The rasterized method is:

* Expensive - having to culling + rasterize every mesh per light means you can only have a few shadow casting lights * Expensive a second time - Each pixel needs to loop over all the lights in its bin * Expensive yet again - It's a lot of memory usage to store all those shadow map textures * Expensive a fourth time - To get good results, you need multiple samples of the shadow map per pixel, which is a lot of texture reads. * Poor quality - Shadow maps are a discrete approximation of shadows. Even with multiple samples, you'll often have poor quality and even artifacts unless you have either artists or a very complicated system to tweak shadow biases and cascades.

Megalights (stochastic light sampling), by contrast:

* Have a higher base overhead - ray tracing is expensive! * Scale much better - With a fixed number of rays per pixel (usually 0.5, 1, or 2), the time and memory cost at a given resolution depends mostly on the BVH complexity used for accelerating raytraces (if the random sampling is done correctly, Nvidia research shows that this can be surprisingly slow if you're not careful). Forget having only a couple shadow casting lights, now you can have thousands! * Looks much better, with much less work - No more adjusting biases or cascades or shadow sampling patterns. There's no need since raytracing is a fully continuous representation of visibiliyy between two points, unlike shadow maps. You can also integrate proper transparencies, refractions, reflections, volumetrics, emissive meshes or textures, IES light profiles, etc with raytracing, unlike rasterization. * Has different failure modes - With rasterization, too many lights or too complicated lighting will kill performance, but not quality. With megalights, performance will be fine, but you'll end up with too much noise for the denoiser to handle, which will look bad.

You aren't missing anything. I tried to find the FDA's press release, including to what the article links to when they're supposedly summarizing what the "FDA alert" says. The linked website is just a general list of FDA alerts, and doesn't list kirkland butter at all.

The same link you posted (FDA event listing) is the only thing I can find directly from the FDA on it, and they don't say "throw out the butter". They just say they're issuing a recall due to mislabeled product, that's it.

Something else I haven't seen people mention in this thread, are kids.

I've had a severe peanut allergy since I was like 5 (don't know the exact age, but as long as I can feasibly remember). If at 7 years old I was at a friend's birthday party and they had cake or some candy or whatever, how did I know if I can eat it?

I was 7, I wasn't about to read an entire ingredients list and parse out what is or is not peanuts. I just checked the small list at the end for the standard "contains: peanuts" and that was it.

As an adult, if I'm over at a friend's house or a family dinner, I need to know about allergies in whatever they cooked. I don't ask them "does it contain peanuts or peanut oil or made in a plant that would have cross contamination" - they often won't know. But they can fetch out the boxes of whatever they cooked, and I can easily scan several different boxes in a few seconds and confirm if it's safe for me to eat or not.

I've never had a allergic reaction in the 20+ years I've been alive for, in large part thanks to regulations like these, and it makes me sad to see people condemn them over a small amount of mislabeled butter.

A new article I wrote following up on my last post[0], with all the changes coming to virtual geometry (a 3d renderer similar to Unreal Engine 5's Nanite) in the 0.15 release of the Bevy game engine.

[0]: https://jms55.github.io/posts/2024-06-09-virtual-geometry-be...

Yep, this is another great callout. Desktop GPUs are (in my experience) often heavily memory limited, and that's with their big high bandwidth memory chips. The latency is a problem, but latency hiding means overall throughput is good and it works out in practice.

iGPUs have less latency, but also much less bandwidth. So all those global memory fetches in modern GPU algorithms become much slower, when looking at a birds-eye level of overall throughput across the dispatch. It's why things like SSAO are way more expensive on iGPUs, despite needing to operate at a lower resolution.

The driver handles time slicing between processes, mapping virtual memory to real memory, etc.

You're right that this is an actual consideration for programs. I don't know about CUDA/GPUGPU stuff, but for games, you need to manage residency of your resources, essentially telling the driver which resources (buffers/textures) are critical and can't be unloaded to make space for other apps, and which aren't critical.

In my opinion, the biggest misconception around GPUs I see people have is that they don't realize it's an entirely separate device with it's own memory, compiler, scheduler, etc.

You don't call functions to tell the GPU what to do - you record commands to a buffer, that the GPU later executes at some indeterminate point. When you call dispatch/draw(), nothing actually happens yet.

Another kind of misconception: data transfer is a _really_ overlooked issue. People think "oh this is a parallel problem, I can have the GPU do it" and completely discount the cost to send the data to the GPU, and then get it back. If you want to write 20mb of data to a buffer, that's not just a memcpy, all that data has to go over the PCIe buss to the GPU (which again, is a completely separate device unless you're using an iGPU), and that's going to be expensive (in real time contexts). Similarly if you want to read a whole large buffer of results back from the GPU, that's going to take some time.

Hash tables on GPUs are cool. You can use them for meshless radiance caches, that automatically (for better or for worse) adapt to the surrounding geometry.

> performance is only good if you have a current-generation GPU that cost at least $1k

That's why I said games aren't currently designed with only pathtracing in mind, but in 3-5 years with faster hardware and better algorithms, we'll probably start to see it be more widespread. That's typically how graphics usually develop; something that's only for high end GPUs eventually becomes accessible to everyone. SSAO used to be considered extremely demanding, and now it's accessible to even the weakest phone GPU with good enough quality.

Again the fact that it's feasible at all, even if it requires a $1000 GPU, is amazing! 5 years ago real time path tracing would've been seen as impossible.

> The way it went for CP2077 makes your "70% usable" claim seem like quite an exaggeration

Based on the raw frame timing numbers and temporal stability, I don't think it is. RT GI is currently usually around ~4ms, which is at the upper edge of usable. However the temporal stability is usually the bigger issue - at current ray counts, with current algorithms, either noise or slow response times is an inevitable tradeoff. Hence, 70% usable. But with another few years of improvements, we'll probably get to the point where we can get it down to ~2.5ms with the current stability, or 4ms and much more stable. Which would be perfectly usable.

You're looking at years of careful artist and engineer work to get something that looks almost as good as pathtraced visuals. The fact that it's so good without any raytracing is a credit to the developers.

Replacing all that effort with raytracing and having one unified lighting system would be a _major_ time saver, and allows much more dynamic lighting than was previously possible. So yeah some current games don't look much better with RT, but the gameplay and art direction was designed without raytracing in mind in the first place, and had a _lot_ of work put into it to get those results.

Sure fully pathtraced graphics might not be 100% usable currently, but the fact that they're even 70% usable is amazing! And with another 3-5 years of algorithm development and hardware speedups, and developers and artists getting familiar with raytracing, we might start seeing games require raytracing.

Games typically take 4+ years to develop, so anything you're seeing coming out now was probably started when the best GPU you could buy for raytracing was an RTX 2080 TI.

Yeah I don't disagree with anything you said.

Vulkan/DX12 _are_ learnable by hobbyists. This was a pretty popular post here on HN 4 months ago of someone learning Vulkan and making an engine in it https://news.ycombinator.com/item?id=40595741. Universities usually teach theory, oftentimes in the form of a raytracer on the CPU, or like you said a simple OpenGL renderer using some prebuilt abstractions. I don't think it really makes sense for them to teach how to use Vulkan well or how to make a fast renderer, the details of that often change quickly year by year anyways.

> That's a pretty sad state of affairs given the "audience" is shrinking by the day. And then later those graphics programmers leave/get laid off by Unity/Epic/AAA Studio with a custom engine and they wonder why they can't find any DX12/Vulkan engineers to their satisfaction.

That's more a symptom of how garbage working in the game development industry is, and less about any underlying technology. There's a reason I work on a game engine for fun, as my hobby, and not professionally despite having the option to do so. Everyone I spoke to in the industry talks about how terrible the working conditions are.

A professional graphics developer I recently talked to summed it up well - everyone needs a game engine, but no one wants to pay people to make and maintain one.

> executing individual draw calls of meshes which map 1-to-1 to visible objects.

This has not been true since deferred shading became popular around 2008. Shadow maps were around much earlier than that even.

There's a reason the 1:1 draw:object API has fallen out of popularity - it doesn't scale well, be it CPU overhead, lighting, culling and geometry processing, etc.

That said, you of course still can do this if you want to. Draw calls and vertex buffers haven't gone away by any means.

> So now the idea that a dev can just bring their own shaders to plug into an existing pipeline kind of falls apart. You need a whole layer of infrastructure on top, be it node graphs, shader closures, etc. And dispatch glue to go along with it.

That's the job of rendering engines, not graphics APIs. If you want to work at that layer, then you use a rendering/game engine that provides the tooling for technical artists. If you _are_ the rendering/game engine, then you're thankful for the increased level of control modern graphics APIs provide you to be able to realize better looking, higher performing (more stuff is possible), and more flexible tools to provide your tech artists with.

> This is all true even with WebGPU where you don't have to deal with synchronization and mutexes. Just a shit show all around tbh. Rendering APIs have not kept up with rendering techniques. The driver devs just threw up their hands and said "look, it's a nightmare to keep up the facade of old-school GL, so why don't you do it instead".

Users of the drivers got fed up with them being buggy, slow, and limited. The industry's response was to move as much code as possible out of the driver and into user space, exposing more control and low-level details to userspace. That way, you would never be bottlenecked by the driver, be it performance or bugs. The industry has realized time and time again that hardware companies are often bad at software, and it would be better to let third parties handle that aspect.

The real failure of of the graphics industry imo was Vulkan 1.0 trying to cater to old mobile devices and modern desktop devices simultaneously, and much worse, never starting a large community project to communally develop a higher-level graphics API until WebGPU (which itself is underfunded). Even then its higher-level nature is largely a byproduct of wanting to enforce safety on untrusted webapps.

But yes, even WebGPU is still more complicated than OpenGL 2. If you find graphics APIs too much work, you're not their target audience and you should be using a higher level API.

What part do you dislike? If it's the complexity of newer APIs (Vulkan in 8 years old at this point, DirectX12 9 years), then you might like WebGPU or any of the other userspace graphics APIs such as blade or sdl3 that have been invented over the past few years.

As someone who starting using Rust from before it reached 1.0, it's insanely funny to me to see comments like this.

People said the exact same kind of thing about Rust at the time!

Of note is that tracy is aimed at games, where sampling is often too expensive and not fine-grained enough. Hence the manual instrumenting.

For the Bevy game engine, we automatically insert tracy spans for each ECS system. In practice, users can just compile with the tracy feature enabled, and get a rough but very usable overview of which part of their game is taking a long time on the CPU.

Just RenderDoc really. PIX is DirectX only, and everything else is vendor specific.

I don't really know why you wouldn't just use your vendor's tooling though. The only thing I use RenderDoc for is I find debugging with it a bit easier than NSight when I need to find a problem, but for anything performance related I use NSight (and would use PIX as well if Nvidia didn't gimp it).

Oh also Intel's IGA I've found to be very buggy as well, so I avoid that.

Wow, looking at the syllabus, I can't imagine learning all of this in a single semester at more than a very surface level (and the course notes definitely don't seem surface level).

Take just rendering (my area of expertise): There's game programming in general, OOP and ECS, input and update loops, setting up a window, common patterns, etc, necessary prerequisite stuff.

Then there's rendering APIs and the GPU, e.g. learning about vertex/fragment shader concepts and syntax, buffers and uploading data, binding resources, making pipelines and such, etc. Then there's how to make an actual rendering _engine_, e.g. abstractions for batching entities, generating draw lists, command buffer recording for various passes, etc. Then there's lighting - analytic direct lights, many many forms of baked or realtime indirect lighting, BRDFs and PBR shaders, the pain that is shadow mapping, etc. Then on top of all that there's actually optimizing everything both from a CPU and GPU (shader) perspective.

And that's _just_ rendering. Game engines are usually way more. Asset management, physics, UI, possibly scripting, possibly networking, animations, usually some sort of scene editor, etc. All of those with many many subfields and complexities.

This has actually gotten better recently. You can now implement custom error diagnostics.

E.g. Bevy implements a trait (QueryData) for tuples of up to 32 items (A,) + (A, B) + (A, B, C)...

If you went over that 32 items, you used to get a confusing error about not your type not implementing QueryData.

Now you get a nice error message explaining the common reasons why this your type does not satisfy the trait.

> But a high quality automatic LOD at build time may (read: almost certainly does) strike a much better balance for both current and near future hardware

You can't have a manual LOD for a cliff where half is near the player and should be high resolution, and half is further away and can be low resolution. Nanite's hierarchical LODs are a huge improvement for this.

You're also underestimating the amount of time artists have to spend making and tweaking LODs, and how big of an impact skipping that is.

It's been mentioned a couple of times in this thread, but Bevy also has an implementation of Nanite's ideas (sometimes called Virtual Geometry). I'm the author of that, happy to answer questions :)

As for this project, Scthe did a great job! I've been talking with them about several parts of the process, culminating in some improvements to Bevy's code based on their experience (https://github.com/bevyengine/bevy/pull/15023). Always happy to see more people working on this, Nanite has a ton of cool ideas.

Small correction: meshoptimizer only does the grouping triangles -> meshlets part, and the mesh simplification. Actually building the DAG, grouping clusters together, etc is handled by Bevy code (I'm the author, happy to answer questions).

That said I do know zeux was interested in experimenting with Nanite-like DAGs directly in meshoptimizer, so maybe a future version of the library will have an end-to-end API.

Not necessarily. Nanite compresses meshes (including in-memory) _very_ heavily, and _also_ streams in only the visible mesh data.

In general, I wouldn't think of Nanite as "one thing". It's a combination of many, many different techniques that add up into some really good technology.

Yeah. SDL went the path of "wrap native APIs". WebGPU went the path of "exactly what level of floating point precision can we guarantee across all APIs" along with "how do we prevent absolutely all invalid behavior at runtime, e.g. out of bounds accesses in shaders, non-dynamically uniform control flow at invalid times, indirect draws that bypass the given limits, preventing too-large shaders that would kill shader compilers, etc".

WebGPU spends a _lot_ of time investigating buggy driver behavior and trying to make things spec-conformant across a lot of disparate and frankly janky platforms. There's a big difference between writing an RHI, and writing a _spec_.

Yes but also no. WebGL lacks compute shaders and storage buffers, and so has a different path on WebGL than WebGPU. A lot of the code is shared, but a lot is also unique per platform.

---

This is also as good a place as any, so I'll just add that doing 1:1 graphics comparisons is really, _really_ hard. OS, GPU driver, API, rendering structure, GPU platform, etc all lead to vastly different performance outcomes.

One example is that something might run at e.g. 100 FPS with a few objects, but 10 FPS with more than a thousand objects. A different renderer might run at 70 FPS with a few objects, but also 60 FPS with a few thousand objects.

Or, it might run well on RDNA2/Turing+ GPUs, but terribly on GCN/Pascal or older GPUs.

Or, maybe wgpu has a bug with the swapchain presentation setup or barrier recording on Vulkan, and you'll get much different results than the DirectX12 backend on AMD GPUs until it's fixed, but Nvidia is fine because the drivers are more permissive about bugs.

I don't trust most verbal comparisons between renderers. The only real way is to see if an engine is able to meet your FPS and quality requirements on X platforms out of the box or with Y amount of effort, and if not, run it through a profiler and see where the bottleneck is.

- A Bevy rendering contributor

GPUs are setup as large amounts of SIMD blocks of threads with some shared units like registers, cache, ALU, etc for every few blocks of threads.

The typical way you schedule GPU work is to dispatch a very large amount of work all running the same program. So you'd spawn 10 million units of work, to run on 10 thousand thread blocks. Due to differing memory accesses per unit of work, each threadblock will complete their work at a different time. Whenever a threadblock is stuck waiting for memory accesses to complete, or has finished it's work, the GPU scheduler gives it a new unit of work to work on.

If you graph "occupancy" as the percentage of threadblocks busy doing work, then you'd see a spin-up period as threadblocks are filled with work, a steady period where all threadblocks are busy, and then a spin-down period as there's gradually less work available than the number of threadblocks.

If you wanted to run two programs (e.g., check meshes to determine what's visible and cull invisible meshes, then draw the remaining visible meshes), then you'd have a hard gap in between. Threadblocks would spin-up with culling work, work steadily, spin-down until 0 work is left, spin-up with mesh drawing work, work steadily, and then spin-down again. The spin-down period in between the two passes is bad for performance.

Rather than having the GPU go completely idle, wouldn't it be better if, as the GPU is running out of culling work to execute (available work is less than the number of threadblocks available to perform work), you could fill in the gaps by immediately starting the mesh drawing work for meshes that have already passed culling? That way there would be no idle time between passes. Workgraphs let you do this, by specifying execution not as monolithic passes, but as nodes that perform 1 unit of work and produce output for other nodes to consume.

Another benefit is memory allocation required. With the two distinct passes model, you need to allocate the worse-case amount of memory to hold the first pass output (input to the second pass). If you have 10,000 meshes, then you need to allocate space to be able to draw 10,000 meshes for the worst case that they're all visible - there is no runtime memory allocation on the GPU. With workgraphs, the GPU can allocate a reasonable estimate of how much memory it needs for node 1's output (input for node 2), and if the output buffer is full, the GPU can simply stop scheduling node 1 work, and start scheduling node 2 work to pop from the buffer and free up space.

As for whether you can do this with the existing GPU pass-based model, more or less yeah. You can build your own queue and use global atomics to control producer/consumer synchronization. It's called persistent threads. You might even do better than the GPU's built-in scheduling depending on the task, if you hyper-optimize and tune your code. However, it's a lot harder, and dependent on specific implicit behavior of the GPU's scheduler. If the GPU is not smart enough to sleep threadblocks waiting for the atomic "lock" and schedule threadblocks that are holding the lock, then you get a deadlock and your computer freezes until the GPU driver kills your program.

The big reason workgraphs are getting introduced is for Unreal Engine 5's Nanite renderer that came out a few years ago. It uses the persistent threads technique to traverse a BVH, doing cull checks on each node, with passing nodes getting their children pushed onto the queue. BVHs (trees) can be unbalanced, so Nanite uses the persistent threads technique to dynamically load-balance nodes amongst the available threadblocks. With workgroups, Nanite could instead define it as a graph, with culling nodes outputting work for more culling nodes, and have the driver handle the load balancing.

Something I want to point out is _why_ you would want a graph for GPU work.

The existing programming model is based on passes. You have one dispatch spin up thousands of workgroups, each of which do some computation. Then you issue a barrier, change your shader, and issue another dispatch doing more work, often reading the previous pass's result.

At first glance this seems fine, but it has some inefficiencies.

1. Often the size of the second pass depends on the output of the first. The first pass might determine that some work is unneeded, and therefore the second pass would have less to do. But from the CPU side, you need still need to allocate memory for the worst-case amount of output from the first pass. 2. Towards the end of the first pass, you'll have only a couple of workgroups left running. But you can't start running the second pass yet - you need to wait for _all_ workgroups in the first pass to finish, and then issue a barrier for memory synchronization.

Workgraphs let you fix these problems. You define two nodes A->B that would correspond to your old passes. Then, you only need to allocate a reasonable amount of scratch memory for the first pass's output - you don't need to allocate for the worst case. When A is running, B can immediately run and pickup the results from A via the scratch buffer as A's workgroups complete.

Workgraphs also give you a lot more flexibility with branching based on work type. A common technique is to split the screen into fixed-size tiles, and classify each tile as "cheap" or "expensive" to shade. Then you can run a cheap shading pass only on the cheap tiles, and an expensive shading pass only on the expensive tiles. The problem here is the same as we saw earlier - you need a full barrier and wait for the previous pass to complete before the next can be issued, even though they're independent. And that's with only 2 possible shaders. With more passes, or different length chains of work (e.g. this tile just needs some quick shadows, this other needs shadows and then 3 passes of denoising), it gets more and more inefficient. Workgraphs let you turn all that into nodes, and schedule them much more efficiently.

If you've never done graphics programming before, I wouldn't recommend starting with Vulkan. I would start with WebGPU via either wgpu (Rust + various bindings), dawn (C++), or the browser (JS). Or do raytracing on the CPU.

Once you've made a fairly capable engine, have rewritten it 1-3 times and are very comfortable with it, and are looking at profiles and going "hmm I wish I could do X to optimize this but WebGPU lacks support for it", then I would learn Vulkan.

Vulkan (and DirectX12) are targeted at existing engine developers who want to optimize every aspect of the rendering engine. It's not a very forgiving API, and learning the API (and how to use it optimally) is just one aspect of graphics programming. There's also linear algebra and coordinate spaces, PBR rendering theory, etc. It's a very involved field that's quite different from other programming fields.

RE: Vulkan requiring thousands of lines of setup, I think some of it is fair and some of it is not. Extension handling is definitely fairly cursed imo compared to how DirectX12 handles it. Memory allocation is rarely something that needs manually writing, using VMA is plenty good for 99% of use cases. On the other hand, device, queue, and swapchain selection and synchronization are all important. Sure there's a "simple" path where you just select the first device, the main queue, and use the default swapchain choice, but it can get complicated. Transfer and compute queues are important, and are a bit involved in requesting. Swapchain pacing is complex to get right, and you may want to support HDR/WCG displays. Vulkan gives you a lot of control, which is overwhelming at first, but useful for a lot of people.

It's better than OpenGL 3 in my opinion. It has a much saner API, is a lot more predictable, and API calls can be made from multiple CPU threads.

OpenGL 4 has some more advanced features like bindless that WebGPU lacks (and Vulkan has), but I wouldn't use OpenGL because of that.

My recommendation is start with WebGPU if you're new to graphics programming. Once you're at the end point where you have a bunch of working features and have optimized your engine as much as you can, and are starting to push against WebGPU limits, then I would think about switching to Vulkan 1.3 / DirectX 12.

I develop a lot of the more advanced features for Bevy, so I'm fully aware :)

The unfortunate reality is that wgpu does not have enough funding. Extremely valuable features like bindless, multi queue, and mesh shaders/raytracing are missing. CPU-overhead is fairly high. Things like debugging support are not great. Having to abstract over all of Vulkan/DX12/Metal brings difficulties.

I love wgpu, and don't begrudge the volunteer devs (they've done a lot of amazing work already), but I'm also frequently frustrated by its limitations.