A Taste of WebGPU in Firefox | Hacker News Reader

A Taste of WebGPU in Firefox

hacks.mozilla.org

226 pointsfeross77 comments

I love how much this is like Metal! I've been really getting into Metal lately and I find it so much easier to use than I did OpenGL.

The one glaring exception is binding. It's the thing I hate most about OpenGL. I don't really have to think about it in Metal at all. The fragment shader just takes a texture or buffer as a function argument and I can use it just like I'd use any other C++ object or struct in the shader. From the calling side, I just say, here's the texture that goes with that argument (by name in the code, not by some random ID assigned at runtime). No figuring out which texture unit to use, or which texture target. No cases of accidentally forgetting to enable a texture unit or bind a texture to it, or using the wrong texture target. All of that fiddling around makes OpenGL such a pain in the ass to use. But it seems like WebGPU kept some of that nastiness. Why?

The binding model of WebGPU is nothing like OpenGL. I don't know where you get that idea, especially considering a dedicated section in the article describing this.

Metal binding model works well because each buffer can contain references to other resources (`MTLArgumentBuffer`). So you can change whole packs of resources at once with this. But argument buffers are not implementable on other APIs. We chose Vulkan's descriptor set-like model as the least common denominator.

In modern OpenGL with DSA[1] (Direct State Access), you barely have to bind anything at all.

1: https://www.khronos.org/opengl/wiki/Direct_State_Access

Will this API be opt-in, or can we expect another security disaster like with WebGL?

https://news.ycombinator.com/item?id=16457791 , https://www.contextis.com/en/blog/webgl-a-new-dimension-for-... , https://www.contextis.com/en/blog/webgl-more-webgl-security-...

Edit I should have given a quote from one of the articles covering the issue, so here's one:

> anyone running Firefox 4 with WebGL support is vulnerable to having malicious web pages capture screenshots of any window on their system

It's up to the browser to decide if it's enabled by default at some point in the future. There is nothing in a Web API saying that it has to be forcefully enabled.

I do share the concern that security of talking to the GPUs is hard. We'll do our best. I think it's important for the Web platform to reach this capability.

Interestingly, the implementations are more diverse, because there isn't a single Angle-like library that everyone is going to use. This may mean that it's easier to find a vulnerability in a concrete browser, but harder to make an exploit that works portably.

I feel like this should require the user's explicit permission, just like audio, camera, location or Flash and Java Applets at the end of their life.

WebGPU is a great innovation, but we have both privacy and security concerns here. It should be available where and when the user wants it, not silently in the background.

Why don't you say the same about CPU or RAM? In the end it's just resources that are abstracted away to the user, used with the purpose of performing tasks in the most efficient way possible.

Because some resources have mature security controls around them, and some don't.

Why do you care if someone can see your bank account number? In the end they're just numbers.

> Because some resources have mature security controls around them

So we thought, then ROWHAMMER arrived..

What you really want is a security first web browser, not something that must prioritize feature completeness to achieve commercial success. I would love a spin-off of the Tor browser sans Tor.

Whether or not the API is opt-in is orthogonal to its security hardening - this is a false dichotomy.

But that's not what parent was saying, I think. He was saying that WebGL not being opt-in was a security disaster; while saying nothing about the security hardening of the WebGL module itself.

Even when a module is well tested, I'd like non-essential modules to be opt-in too.

I’d like Javascript to be opt-in. Turning it on by default was a mistake.

You are free to disable it as a very small minority.

The web is not a static document viewing anymore. Now I agree, that sites should have a static fallback, as I also do not want to run a full webapp, when all I want is a static article for example. But this is because of a different problem - the financing of most of the web, through advertisment and not the fault of javascript.

Much like frames, iframes and window.open are really useful features... advertisers and abusers made it bad. I remember the first time I saw a popup in the 90's, my first thought was, "oh this is going to get bad." And I would never buy a product from X10.

I'm not suggesting any false dichotomy, I'm trying to reason about the likely security consequences given what we saw with WebGL in Firefox: serious security flaws in its implementation, with the features enabled in the browser by default. See my links above, one of which summarises with anyone running Firefox 4 with WebGL support is vulnerable to having malicious web pages capture screenshots of any window on their system.

My thoughts are along these lines:

    ┌────────────────────┬──────────────────────────┬──────────────────────────────┐
    │                    │  Secure implementation   │  Insecure implementation     │
    ├────────────────────┼──────────────────────────┼──────────────────────────────┤
    │ Opt-in             │  Minimal security impact │  Significant security impact │
    │ Enabled by default │  Minimal security impact │  Security disaster           │
    └────────────────────┴──────────────────────────┴──────────────────────────────┘

First time I've seen a decent-looking ASCII table in an HN comment. Well-done!

Thanks, it's technically not an ASCII table, it's from the Unicode (single line) style of https://ozh.github.io/ascii-tables/

Running firefox 4 (or any other comparably old browser), this would be the least of your security problems, it's not even RCE. The stream of discovered vulnerabilities is constant.

What's your point? Do you really think I was suggesting that people are still running Firefox 4?

When WebGL was first implemented in Firefox - long ago - it shipped with serious security flaws. This struck many people as sadly predictable, as graphics APIs like OpenGL are not intended to be accessible from untrusted code. WebGPU seems awfully similar to WebGL in this respect: it's trying to wrap a low-level graphics API in a secure sandboxed API. This isn't the equivalent of a routine addition like a new feature in CSS.

The point was that seen from the future, old browser versions are always full of holes in all kinds of features, more in new features than old ones, we should fix the systemic problems that make browsers so ridden with vulnerabilities.

But yes I am guilty of some whataboutism here, WebGL did have some unique problems and WebGPU devs hopefully learn from them.

Fingerprinting is one security concern.

Security implications are always good to point out, though this is not unique to webGPU at all. Some fingerprinting techniques don't even require enabling javascript.

Crypto mining is another

Browsers should be run in a VM, via a fake GPU which is mapped to a window ( possibly root window ) of a main desktop. Giving code fetched from the internet in real time full access to hardware, especially complex hardware, is lunacy. I'm shocked that neither Google nor Mozilla still have internalized this lesson.

The browser should simply disable WebGPU by default. It's already intended to be sandboxed, but given what happened with WebGL, I'm not convinced we can trust it. There's nothing to gain by faking a GPU.

Edit For clarity, I'm saying you may as well just disable WebGPU. A CPU-based simulated GPU would indeed improve security, but its performance would be so atrocious that there would be no point having it compared to the existing drawing APIs.

There is - it is a GPU with all the bells and whistles but even if the browser's security context is broken, the browser cannot access anything on a real desktop.

It's not clear what you have in mind here, but I think you're referring to using a CPU-based virtual GPU to power the browser's WebGPU engine, to avoid GPU-related security issues.

A CPU-based fake GPU is guaranteed to have performance so poor that it isn't worth doing. This is the reason modern graphics APIs don't make provisions for CPU-based implementations.

If we aren't confident in the security of the WebGPU implementation (and we're agreed that we aren't), the solution is simply to disable support for WebGPU in the browser. The browser can still offer facilities like the Canvas API.

> It's not clear what you have in mind here, but I think you're referring to using a CPU-based virtual GPU to power the browser's WebGPU engine, to avoid GPU-related security issues.

That was my mistake - i was not clear. I'm talking about virtualized GPUs that can be passed to a VM running a minimum system needed to boot a browser.

Ok, so you're suggesting always running the browser in a VM, and using the GPU from within the VM (via the host OS of course).

I believe significant work has been done on the browser/graphics sandboxing problem. Whether browsers use techniques comparable to virtualisation, I don't know. I'm afraid I don't know much about how that's done, or how effectively virtualisation is able to contain the operations of VMs, or how high the performance penalty is.

Going with guesswork though, it seems to me virtualisation doesn't offer particularly solid guarantees here. If the host OS has a buggy graphics driver, it would presumably be possible for it to leak data back to the VM. Any facilities the GPU can offer to help contain semi-trusted graphics contexts, should presumably be available for use directly by the browsers, without running a guest OS in virtualisation.

The reason why I want to see a virtualized web browser is because it is becoming an insanely complex piece of software with an insanely large attack surface.

I do not want my browser to be able to talk to Bluetooth devices that are connected to my desktop. I do not want want my browser to be able to access random files on my file system simply because it is accessible to me. I do not want a web browser have ability to have GPU access outside the specific virtualized section of it that can only access the part of the screen and off-screen buffers associated with the window of a browser.

I started localizing different web browsers into dedicated VMs that have just a minimal operating system and a web browser. It works well enough for my use case ( i can playback youtube video almost like I could do it on my 2015 laptop ) but it is definitely hacky.

> The reason why I want to see a virtualized web browser is because it is becoming an insanely complex piece of software with an insanely large attack surface.

This is true, and it's the reason Chromium goes to great lengths to sandbox itself to contain the damage regarding JavaScript engine exploits. Annoyingly Firefox is lagging behind in that regard.

> I do not want a web browser have ability to have GPU access outside the specific virtualized section of it that can only access the part of the screen and off-screen buffers associated with the window of a browser.

Virtualisation will improve your general security, but it's isn't a perfect abstraction especially when it comes to GPUs, and it's not a rock-solid guarantee of the isolation assurances you want. [0] I believe cloud-computing providers never split a GPU between customers, for instance. Disabling sharing of the GPU with the VM would further improve your security at the price of performance.

[0] https://www.hpcwire.com/2018/05/31/gpus-excellent-performanc...

> Annoyingly Firefox is lagging behind in that regard.

This has not been true since quite a while. Firefox has employed sandboxing even before the multi-process work (which culminated in the Quantum branches of Fx releases that added more and more sandboxing with each release). Before that, Moz went a different way than OS level sandboxing by principal containerization (I forgot the correct term, sorry), which worked in terms of separation of execution contexts (of Web JS and other parts like the styling system, plus the browser internals). Elements of that implementation have been removed by now (iirc) since the multi-process split required different communication paths anyway (which also enabled per-origin/-tab/-window OS-Level sandboxing), so that code was no longer needed.

Thanks for the reply, you know much more about it than I do.

What could an attacker do if they were able to trick the JIT into emitting evil native code?

I would say there is no simple answer to your question. However, I am not able to give you an authoritative answer to your question anyway. If you are interested, you can start here[1] or try and ping the moz-security people

[1] https://wiki.mozilla.org/Security/Sandbox

Thanks

> There's nothing to gain by faking a GPU.

Moving applications that can do anything into sandboxed pages in a browser that can't do arbitrary things to the user's system is a win. The more capable the browser is, the more things that once would have been downloaded applications can become safely sandboxed pages.

As I said, it's already intended to be sandboxed for security. As I've clarified in an edit, my point was that CPU-based virtual GPUs perform so poorly that they aren't worth implementing.

Could just have an interface to request access for use, such as with camera and sound/microphone.

Browser should not be an arbiter of it wanting to access camera/microphone/bluetooth/gpu/etc. Browser should only be able to ask host and it should be hosts' job to allow/deny access based on a policy, asking a user, time of day, etc.

If they did that the feature would be DOA. Some of us want it.

Obviously you know better than Microsoft, Google, Apple, Intel, Mozilla and anyone else working on WebGPU.

Interestingly, this is not far from how things currently work with webgl - the sandbox, separate renderer process & browser internal safe GL implementation. No doubt webgpu will use similar things. It'd of course be a nonstarter to give full access to hardware or anything of the sort.

I agree that it would be nice to see stronger hypervisor based sandboxes.

The web is an ever growing rube goldberg machine of security vunlerabilities. The web is impossible to secure and the web will never be secure. Only browse sites that you trust, with JavaScript disabled and adblocking enabled, and hope that they haven't been compromised. This is the best you can do in the face of the reckless behavior of browser vendors.

A lot of other web APIs and features had rce vulnerable implementations in stable browser releases when they were new, some worse than others. Eg webaudio, SVG, etc. Not an excuse of course but tells of the cross cutting systemic problems.

This is really old information

I hope it will bring real performance gains besides just easier API that is abstracted away in most apps.

I have 2 projects using Three.js and pure WebGL: https://bad.city (multiplayer browser game engine + editor) and http://fonted.io (shader toy + fonts)

Right. I think, if by 1.0 we aren't able to show significant performance gains in a wide sense (latency, frame stability, and the actual frame time), it will be a failure.

I'd love to give bad.city a go but the need to sign up to play is a little offputting. Do you mind if I ask what the reason is, please?

I know some people coded up raytracers with glsl texture shaders, but rayracers are massive parrallel with no shared state needed algorithms (the entire scene is tiny in those examples). Would this new API enable shared state (to some extent)? For example, someone may want to solve numerically the diff equation of the liquid flow. This usually involves dealing with big lattices where parts of the lattices can be updated in parallel, but on the edges, there need to be some data sharing involved. Same question, but for multiplying big matrices, as many of those numerical methods allow to reduce the task to multiplying huge matrices.

Yes, compute shaders will enable slightly more ability to do computations that are less parallel than just fragment shaders can handle.

We are exposing writable storage buffers and textures in all shader stages (there is a caveat discussion [1] on the topic though), which means you can write data out directly. Sharing data between shader threads can be efficiently in the compute shaders (via the shared group memory), which are included in the core of WebGPU.

P.S. vange-rs [2] is written on wgpu-rs and uses ray-tracing in fragment shaders for the terrain. I hope to run it in the browser one day!

  [1] https://github.com/gpuweb/gpuweb/issues/639
  [2] https://github.com/kvark/vange-rs

> writable storage buffers and and textures in all shader stages

Beware the behaviour of writes will not be consistent between vendors or shader stages, and may be surprising.

Yes, we are fully aware of that. There must be a line drawn somewhere between work that we can make portable, and the work that is platform-dependent. Enforcing any ordering on storage writes is completely unfeasible.

Shameless plug of my own WebGL based ray/path tracer: https://github.com/apbodnar/FSPT Compute shaders will allow for some performance wins over my current fragment shader version by being able to do a ray binning/sorting pass to shoot rays coherently allowing for better cache behavior and shared shader state.

Are those pics in the readme the actual output? If so, this is freaking astonishing cinematic-level quality.

This is not working for me on Safari even with turning on the experimental feature, when will this land on regular Safari ? Excited to see WebGPU possibilities.

Safari implements a large surface of the API ([1]), but they don't accept SPIR-V shaders. When WGSL design is complete, and we have it supported in the browsers, it will be time for truly cross-browser applications.

[1] https://webkit.org/blog/9528/webgpu-and-wsl-in-safari/

Now the cryptominer JS scripts can REALLY get some performance while you are on their site!

And of course its another way to do unique profiling of users.

Would this enable bitcoin mining using gpu in a browser?

It's already possible, see https://gpu.rocks/.

Would likely just make it faster as you wouldn't have to read the results out of a canvas, although I haven't checked what the pure data API for WebGPU looks like.

After a bit more searching: https://bitcointalk.org/index.php?topic=27056.0

Can I debug shader in webgpu like JavaScript in devtools?

WOW! Finally WebGPU. Web Games will become a thing now!

I really dislike when these articles say "works on Linux" (Reminds me of people that say "works on PC" when it actually only runs on a single OS.

Some Linux setups run Xorg, others run Wayland (with the world moving towards the latter). Yet, many times these new features (especially things related to rendering!) only work on older Xorg setups, but not on Wayland.

For anyone Linux users out there: these updates only work if you're still running Xorg.

I don't know why it wouldn't work on Wayland. We are currently using "software" presentation, which is a CPU roundtrip, as described in the article. This means we don't depend much on the actual Linux compositor. As long as WebRender can run on it, and you have Vulkan support, WebGPU will be available. If you are having issues, please file a bug!

Setting `gfx.webrender.all = true` makes sites like google maps render as just plain black (eg: no image). I'll try to debug in further detail and open a bug when I have clear reproduction details.

That's worrying! Mozilla gfx team would be interested in fixing that, as we are trying to roll WebRender on more platforms.

So this WebGPU stuff is pretty much a Javascript binding to Vulkan. The design is almost 1:1 identical. Good luck securing that! I find bugs in the standard Vulkan validation layer pretty regularly after it has been in development for a couple of years. Sure, some of the stuff is pretty hard to check like device memory addresses. But just this week I had the validation layer blow up spectacularly because I simply dared to give it an unrealistically high index for a vertex binding and the layer attempted to resize its internal state tracking array for vertex bindings to accomodate that.

Vulkan has another fun aspect to it: the application doesn't have full control over the layer loading mechanisms in the mandatory driver wrapper. A user's system can be configured to load arbitrary validation layers in various ways and the application cannot prevent any of that. It can only ask for more layers to be loaded. This is great for development, but it also means that browsers can't generally assume that the Vulkan API does what they think it does. There might be a little extra in there.

Is this unique to Vulkan or would the same hold true for DX12 or Metal? If it’s unique, perhaps WebGPU could steer Vulkan closer to the others in this regard. I don’t think WebGPU is meant to be a Vulkan binding exactly but pushing secure and safe low level GPU computing across platforms. If Vulkan isn’t good enough implementation layer it will just need to evolve

Vulkan itself is what it is now: an abstraction of (reasonably) modern GPUs on a very low level. It is certainly very useful in that it allows applications to use GPUs in ways that are very efficient and simply impossible with older APIs, especially OpenGL. The flipside is that this flexibility and the intentional thinness of the abstraction makes it hard and actually undesirable to check the validity of the API usage in actual production use. That's why the debug layer is an optional layer and not part if the Vulkan loader.

I need to check out a few things, but if WebGPU prevents access to a few hairy bits of Vulkan like explicit synchronization and memory management (the Vulkan app has to provide that), it might be securable. There are still some benefits over OpenGL left like the entire pipeline state being expressed in pipeline objects.

I don't know about DX12 or Metal in practice, so I will not comment on that.

This isn't JS to Vulkan. You are talking about the Obsidian proposal [1] that was not accepted or implemented.

WebGPU is higher level. High enough to hide the most nasty aspects of Vulkan, yet low enough to still extract the performance benefits of a proper API.

Implementing validation and proper input sanitation is undoubtedly a big challenge. Each browser engine(!) vendor is going to be tackling that separately. We are ready to learn on Vulkan mistakes and drive this story of validation and CTS to be better. One of the things that make a difference in comparison to Vulkan is that validation is not an afterthought, not a side developer assistance tooling. Validation is in the core of WebGPU, it's mandatory and always enabled, so chances are it will be more polished.

[1] https://github.com/kvark/WebGLNext-Proposals/blob/obsidian/O...

WebGPU is extremely different from Vulkan. They have similar concepts because they're both from the same family of "explicit APIs" (that also contains Metal and D3D12).

Vulkan is hard to validate because it was designed assuming it trusts the developers to follow the valid usage rules. For example it is the developer's responsibility to only destroy GPU resources after they are done being used. Validation layers have to track for each submission the link from vkCmdBindDescriptorSets to descriptor sets to texture views to textures just to validate there is no use after frees, and that's just one of the many many valid usage rules.

In contrast a guiding principle in the design of WebGPU is that it should be fully validated at a reasonable cost. Keeping the same example, calling texture.destroy() in WebGPU will make the browser destroy the texture after all previous execution completes, and prevents any further usage of that texture. The Web Platform Tests (WPT) is a test suite that ensures browsers are interoperable. WebGPU's tests in WPT will contain test cases for all validation rules and corner cases, including overflows like the one you mentioned (and check it produces an error that is handled gracefully).

From the programmer's POV the API model is really more like Metal with a few bits and pieces "inspired by" Vulkan, but that doesn't even matter much for security considerations.

I don't think any of the existing implementations just "passes through" any "request" to the underlying 3D APIs (which is not just Vulkan btw) without taking it apart, looking at each piece to make sure it checks out, and then reassembling it. The security implications shouldn't be much different than WebGL (for better or worse of course).

The philosophy of the modern 3D-APIs like Metal, Vulkan and D3D12 of doing all costly configuration operations upfront instead of during rendering should help a lot to improve rendering performance compared to WebGL, where those security validations literally happen at random times during rendering.

I'm not following your final point. What does it matter if Vulkan gives you more runtime checks than you were expecting?

The loader is a backdoor to load arbitrary code as long as it is disguised as a Vulkan layer (a specialized shared library). It is intended for debugging, but a layer can hook the API to do anything it wants.

Give me one example of a graphics API today on desktop where this isn't possible. Unless you're a console graphics engineer, you're shit out of luck if you really think you can stop client-side injection. Short of a kernel-mode injection like the crap Valorant is pulling, layer hooks and trampolines are literally how those APIs ship (D3D, OpenGL, Vulkan, etc).

Stop propagating misinformation. Personally, I wish WebGPU had gone a slightly different way (thanks Apple :/) but it's definitely no way 1:1 identical with Vulkan.

Also your last point makes no sense. That's been the state of desktop graphics programming for years and years. You really don't control much as IHVs are always free to intercept calls and control things at the driver level.

Reply on news.ycombinator.com