Mesh Shaders and Meshlet Culling in Metal 3 | Hacker News Reader

Mesh Shaders and Meshlet Culling in Metal 3

metalbyexample.com

48 pointsibobev24 comments

Whether this is a win is an difficult question. Rend3 recently put in meshlets, and performance got worse for my use case. It's a way to reduce draw work at some cost in GPU memory. But in a metaverse application, where you have user-created content, you have less instancing, and tend to run out of GPU memory before running out of rendering power. Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption.

There's the question of whether you want to do occlusion culling down at that level. Yes, it cuts GPU load way down when you're in a windowless room. Then go outside and the load goes way up. If you have windows in your buildings, the occlusion culling doesn't help as much. This trick improves average GPU load for a scene, but not worst-case GPU load, which is what matters for playability. When I tried this on an older GPU with slow compute shaders, the frame rate got worse by a factor of 3, and the GPU memory consumption increased.

If you want to do occlusion culling, there's an argument of doing it by pre-computing occluded zones for a level. Unreal Engine does that sort of thing. Analyze once at build time, not once per frame.

That's a problem with pushing work down to the GPU level. You have to do it on every frame, which may be a lose.

Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption.

Though on Apple’s ARM devices (and consoles) memory is shared between the CPU and GPU. I am unsure if making some asset available to the GPU to render (e.g. a texture) requires making an extra copy in ram just for the GPU, but if it is not required and you can just have one copy, then shouldn’t doing LODs GPU vs CPU require roughly the same amount of memory?

If you don't need a higher LOD for a texture yet, it doesn't need to be in memory at all. It can be out in a file or cache.

Big-world networked games have something that looks like a cache hierarchy. There's content in the GPU, content in main memory, decompressed content in a cache on SSD, content that hasn't been decompressed yet, and content that's still on a server somewhere. Much of this data wrangling is done at a slower time scale than the frame rate. So you don't want to push all that down to the GPU level.

The heavy machinery behind this is much of building a metaverse is really about. If you don't do all this, the result is low-rez, like Decentraland and Horizon. If you do it right, things look pretty good.

Imagine if Apple would just add Vulkan support to MacOS - I might actually be able to use my powerful graphics hardware for something

the modern graphics API landscape is a disaster. they killed opengl es, they basically killed opengl. Apple is trying to lock everyone in with metal, khronos group is trying to get everyone to move to vulkan, microsoft is doing the same with d3d12. The only people winning here are the major engines and AAA studios who have the scale to deal with all the platform complexity and benefit from it because it creates a moat from competitors. There's an argument for regulation here because the market is dysfunctional but I don't think we'll ever see the day where politicians understand complex technology issues enough to fix it.

The graphic API landscape is how it always has been, specially on game consoles, really OpenGL only had a shot thanks to id Software, and even Carmack changed his mind later.

> Speaking to bit-tech for a forthcoming Custom PC feature about the future of OpenGL in PC gaming, Carmack said 'I actually think that Direct3D is a rather better API today.' He also added that 'Microsoft had the courage to continue making significant incompatible changes to improve the API, while OpenGL has been held back by compatibility concerns. Direct3D handles multi-threading better, and newer versions manage state better.'

https://www.bit-tech.net/news/gaming/pc/carmack-directx-bett...

On the subject of this thread, it was also DirectX 12 Ultimate that introduced Mesh Shaders.

Meanwhile had it not been for AMD Mantle, Khronos would still be wondering how OpenGL vNext would be like, Vulkan is already reaching the same level of OpenGL extension spaghetti, and the latest extension acknowledges pipelines were a design error that failed to gain adoption.

"On the application side, many developers considering or implementing Vulkan and similar APIs found them unable to efficiently support important use cases which were easily supportable in earlier APIs. This has not been simply a matter of developers being stuck in an old way of thinking or unwilling to "rein in" an unnecessarily large number of state combinations, but a reflection of the reality that the natural design patterns of the most demanding class of applications which use graphics APIs — video games — are inherently and deeply dependent on the very "dynamism" that pipelines set out to constrain."

https://github.com/KhronosGroup/Vulkan-Docs/blob/main/propos...

Design by committee at its best.

> On the subject of this thread, it was also DirectX 12 Ultimate that introduced Mesh Shaders.

It was actually Nvidia who introduced mesh shaders by way of Vulkan and DX12 extensions. And then both Khronos and Microsoft went through concurrent processes to turn it into something that is somewhat suitable for cross vendor use.

Though how suitable remains open. The sweet spots for mesh sizes etc still seem different for different vendors.

I stand corrected, just checked and NVidia's blog post predates DX 12 Ultimate.

> The graphic API landscape is how it always has been, specially on game consoles

This is not what I want as a consumer though. Honestly I don't care if I have a MacBook, Windows PC or a Linux desktop - I just want to be able to play games.

With the current state, I have my MacBook Pro for work and I travel frequently. Before Apple Silicon, I could bootcamp Windows and get by with one computer. Now I actually travel with two computers.

I understand that design by committee can be challenging because of potential for spaghetti - but that doesn't make a fractured/siloed ecosystem better.

Ultimately, Apple could support Vulkan alongside Metal. It's not a binary choice.

You're in good position then, all game engines that matter support all major APIs.

If a game isn't available in a specific platform, the 3D APIs aren't the issue, rather additional QA costs, platform own quirks, addtional support costs, expected sales, see Android NDK games versus how little of them are available on GNU/LInux, yet Vulkan/OpenGL ES/OpenSL/C/C++ are available in both platforms.

It's good that OpenGL is dead since it became a mess of an API starting at around 2.0, and it only got worse afterwards.

The problem is that Vulkan is also already a mess of an API, that's why it is so important to have alternatives like D3D or Metal.

Also... the regulation argument is silly, competition is always good for everyone even in obscure niches like 3D APIs.

(FWIW at least Vulkan is starting to improve with radical recent extensions like VK_KHR_dynamic_rendering, VK_EXT_shader_object and VK_EXT_extended_dynamic_state, but especially the last two are almost a 180 degree reversal on the previous design philosophy - funny enough, those extension look quite similar to the OpenGL programming model, I'm not sure if the pendulum is swinging a bit too much into the opposite direction)

Imagine if you would use one of the middleware engines that support Metal, instead of complaining.

I use WGPU, which tries to abstract over Metal, Vulkan, and Direct-X, with reasonable success. But it adds additional concurrency restrictions, which the developers are struggling to remove. It's a huge headache building and maintaining compatibility layers.

Not all middleware is born equal.

Just like the extension spaghetti from OpenGL and Vulkan is an "headache building and maintaining compatibility layers" of its own doing.

Khronos losing steam on improving GLSL, rather backing on Microsoft and HLSL for future shading language improvements.

And then there are the Web versions that are also their own set of issues, and what better way to cater for adoption than introduce yet another shading language.

You can use Vulkan via MoltenVk, personally I take Metal over Vulkan any day though (the Metal API is a lot more ergonomic - dare I say 'enjoyable' - than Vulkan).

With a modern shading language. At Vulkanised they stated that eventually would rather adopt whatever Microsoft is doing with HLSL than trying to improve GLSL (no resources for that currently).

The one good thing coming out of Vulkan is the whole software ecosystem which evolved around SPIRV (so that details like shading languages don't matter all that much anymore).

You're probably right about the semantics of the API from the perspective of a developer - however from the perspective of a consumer, the best API in the world means nothing if I can't run any games on my machine.

That said, it's not a binary choice - MacOS could support both, they just choose not to.

A custom 3D API wrapper around Metal and Vulkan is quite trivial compared to the other challenges of porting a game to another platform though (and the main reason why "macOS got no games" isn't even technical, but it's usually just not worthwhile from a business perspective).

And MoltenVk definitely does a 'good enough job' if all you have is a Vulkan-based renderer (but most cross-platform games already have a wrapper over platform-specific APIs, including the 3D API).

...also, if Apple would provide a builtin Vulkan API, it would also (most likely) just be a wrapper over Metal, just as they did with OpenGL, so there wouldn't be much of an advantage over a 3rd party solution like MoltenVk.

I don't think a missing Vulkan driver is the big obstacle here, Unreal Engine supports Metal since some time - the Intel laptop graphics hardware was just too slow, thus games only got released on iPhones and iPads. Now that Apple Silicon macs have a reasonably fast GPU, maybe we'll finally see some releases?

Also, a future viable route could be via Asahi Linux and Wine/Proton, but thats a different can of worms.

I am not a graphics programmer, but this procedural geometry reminds me of "tessellation" from ten years ago.

Tessellation has been in metal since it’s release or close to I believe.

Tessellation debuted in Metal in 2016, so yeah, relatively shortly after the first release on iOS (2014) and macOS (2015).

There are definitely similarities if you zoom out far enough. The object shader (task shader in Vulkan) basically maps to the tess control shader / hull shader, and the mesh shader maps to the tess eval / domain shader.

One big difference is that tessellation is fixed function and doesn't allow you to control the topology / primitive connectivity within each patch.

Another big difference is that the mesh shader runs as a workgroup, which allows things like parallel geometry decompression where all threads of a mesh shader workgroup collaborate in the decompression.

Reply on news.ycombinator.com