My mesh shaders talk at XDC 2022
rg3.name
rg3.name
> The new combined shader will be named "Mesh" shader
It's only new for Vulkan. On Windows and XBox, mesh shaders were implemented couple years ago: https://devblogs.microsoft.com/directx/coming-to-directx-12-...
This new Vulkan extension is a direct copy-paste of the above, leveraging the same hardware.
However, the hardware support for mesh shaders is less than ideal. That thing needs feature level 12.2, so-called "DirectX 12 Ultimate". It only works on nVidia Turing (GeForce 20 series) or newer, and AMD RDNA2 (Radeon RX 6000 series, and current generation of game consoles).
That PS2 coprocessor is similar to a single ARMv7 core with NEON.
Mesh shaders don't run loops for elements of the meshlets, they do them in parallel, on different threads of the same thread group.
PC hardware vendors design their features in collaboration with Microsoft for a specific DirectX version, and then they land as yet another extension on Khronos APIs.
Vulkan didn't make this better, they have gotten so many extensions by now, that they finally adopted the version profile concept that is traditional in proprietary APIs.
Unreal Engine 5 ("nanite") shows that the high end game engines are already stepping around a lot of the fixed function pipeline. It's just going to get worse.
The problem is that the GPU compute systems throw away decades of CPU processor development. This is good in some ways. However, it's very bad in others.
GPUs don't yet have a well-defined memory model (to be fair, even CPUs are still a bit sketchy in places)--so you can't guarantee atomicity or progress or fairness. GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc.
The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want.
There is a great deal of space between 'fixed function pipeline' (which no one is using) and 'all compute all the time' (which pretty much no one is using for real-time graphics either).
> GPUs don't yet have a well-defined memory model
Nvidia has a memory model. (I have heard--from an nvidia employee--that amd plays a bit more fast and loose.)
https://www.youtube.com/watch?v=VogqOscJYvk
> GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc.
You do not need an MMU to partition or to protect. (Not to say that GPU security isn't a shitshow.)
> The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want.
https://s22.q4cdn.com/364334381/files/doc_financials/2023/Q2... look at where nvidia's revenue is coming from, and tell me again that they're not already there.
Assuming "fixed function pipeline" means the currently standard programmable pipeline that most people use (and not the extremely old fixed-function pipeline), is there really much of a difference from 100% compute pipelines?
If compute shaders had the ability to manually invoke the hardware rasterizer, what functional difference would there be? The only other major difference I can think of is how the non-compute pipelines can optimize keeping data in registers/cache between different shader steps that can't really be done with compute shaders.
That’s exactly the problem addressed by mesh shaders.
Each thread group of the mesh shader outputs a batch of triangles to be rasterized. These batches have a hard limit of 32kb of output data per batch. I think such a small size limit is there so the vertex + primitive data stays in the faster memory, inside the GPU chip.
If graphics drivers could just allow compute shaders to manually call the rasterizer and give functionality to control what data stays in memory in different parts of a pipeline/command-buffer, then you'd be able to implement both the older graphical pipeline and the newer task/mesh shader pipeline entirely in compute shaders.
I'm not entirely experienced in how graphics hardware works at a lower level though, so I'm sure I must be missing something important.
All modern GPUs (last 5-10 years at least) have MMUs.
You need that sort of MMU to have any sort of secure process isolation without doing expensive command buffer validation on the CPU.
Vulkan has a perfectly fine memory model, and it's non-optional in Vulkan 1.3. The rest of GPU land of course has some catching up to do. It guarantees atomicity but does not guarantee progress (there's active work on characterizing that). And fairness is probably not something you want, as it almost always comes at a performance cost.
Nanite IS stepping around the fixed function pipeline because the triangles they draw are on the order or less of the size of a pixel. They only allow the fixed function pipeline to render the triangles when they are big (ie. almost never).
Current GPU memory models don't give you enough to guarantee progress. Most models seem to do that well enough, but there are pathological cases that aren't that uncommon.
Both of these are backed up from Nanite presentations. So, if you think those are false, know that you are disagreeing with Unreal who has put it on the record.
GPUs don't have MMUs. They have memory interfaces, but they don't have the parts that allow you to protect memory, handle system vs user protections, and manage load. This is why things like Shadertoy can bring your computer to a grinding halt or even cause you to crash.
About the only thing "false" was the fact that the graphics cards don't have a defined memory model. As was pointed out, Vulkan 1.3 does define a memory model. However, the manufacturers have yet to catch up nor are they compliant with that memory model.
Perchance you would like to revise your statement?
> I really wish that everybody would finally admit that the "fixed function pipeline" is dead and shift the development work to making the use of the GPU compute units mainstream. > Unreal Engine 5 ("nanite") shows that the high end game engines are already stepping around a lot of the fixed function pipeline. It's just going to get worse.
Nanite does everything you stated here and thereafter in your reply, but Nanite does not speak for high-end game engines from other companies, which do use ffp units like the rasterizer. Yes, those guys are also moving quite a bit of stuff to compute as I already stated, but to say that the ffp is dead at present is overblown. Nanite does everything you mentioned and for the reasons you mentioned; what I pointed out was not true was the generalization of that to other modern and respectable engines.
> GPUs don't yet have a well-defined memory model (to be fair, even CPUs are still a bit sketchy in places)--so you can't guarantee atomicity or progress or fairness.
This we already cleared up.
> GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc.
GPUs do have MMUs, like others have stated, that handle things like virtual address translation. See, for example: https://learn.microsoft.com/en-us/windows-hardware/drivers/d... Yes, you are probably right that they don't have complete isolation -- has anybody developed side-channel attacks for this yet? -- but that doesn't mean they lack an MMU.
Also, the part about handling load is a bit nuanced. Drivers typically put timeouts on calls, for example, that'll kill the application when it induces a hang (or something long enough to be flagged as a hang.) I've run into this on mobile devices in particular. So yeah, Shadertoy does grind down low-end devices and the result is not exactly pretty, but that does not mean there are absolutely no load protections in place (there are).
> The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want.
Speculation.
Also the virtualization support advertised by NV and AMD uses this. Allowing for GPU partitioning, used eg in VDI setups.
Eg NVidia seems to even have multiple generations of HW modes for this, one timeslicing based one (but I guess still HW enforced memory protection) and one core-splitting based one called MIG.
And here's AMD in 2016: https://www.amd.com/system/files/documents/amd-mxgpu-white-p...
As usual with GPUs the problems seem to be more in the drivers and rest of sw stack, bugs, difficult programmability, proprietary nature, fragmentation, etc than strictly HW features.
MIG for NVIDIA seems confined to the data center chips: https://developer.nvidia.com/blog/dividing-nvidia-a30-gpus-a...
For AMD, from your reference, this is Firepro only? Any other VDI references seemed to be data center only as well.
The preformance advantages only come when you can take advantage of the flexibility to apply new optimisations. Such as sharing work across a mesh invocation, or avoiding launching mesh invocations from task/amplification shaders, or avoiding vertex fetch (procedural generation).