WebGPU Fundamentals | Hacker News Reader

WebGPU Fundamentals

webgpufundamentals.org

429 pointsitsuka199 comments

It's a pity that tfjs never truly developed any decent ops. E.g. you need lgamma to implement the cap for zero-inflated poisson regression and tfjs simply doesn't have that: https://github.com/tensorflow/tfjs/issues/2011 Those guys simply dropped the ball on the floor, and the further framework development fizzled out. I tried to find if WebGPU could be a replacement, but it's very hard to find a good description of the API surface. MDN also doesn't seem to help much – https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API I really hope that one day I'll find something similar to pytorch, only in typescript. Tell me I'll live long enough to enjoy that day.

WebGPU does not play in the same league as tfjs, it is a successor to WebGL, the technology tfjs is implemented with. There is already a WebGPU backend for tfjs: https://www.npmjs.com/package/@tensorflow/tfjs-backend-webgp...

There are also several ONNX runtimes using WebGPU in case you want to run models written with PyTorch.

Exactly, the lack of the deep learning frameworks to train decent models on the client side forces us to use pytorch on the server, leading to centralized data collection, privacy concerns, copyright and data ownership issues.

Imagine the world where you could trivially train a model entirely on the client side, without uploading all that data to the cloud. Then we can settle on federated learning or simply use ensembles of models trained on different clients, all without sharing data with the server.

BTW, I did have some experience with ONNX, also ran into problems with some ops (like nn.SELU not working correctly in the browser - https://pytorch.org/docs/stable/generated/torch.nn.SELU.html...).

> Imagine the world where you could trivially train a model entirely on the client side, without uploading all that data to the cloud.

You just rediscovered "Federated Learning", but FL also allows you to benefit from the training being executed in other edge nodes, all without disclosing any data.

Instead of sending the data to the training server, they send the model to the users to train on a few batches of user data, then average the models from everyone.

It’s Hadoop all over again. Send the code to the data. To be clear, I think that’s an excellent idea and good to see it coming backs again.

This looks heavily geared towards graphics. I’m interested in the compute capabilities of WebGPU to build an in-browser simulator (e.g. physics/quantum). Anyone know of any good resources for that? (Samples, best practices, etc.)

This is usually done with shaders and a circle of buffers which maintain state.

Total graphics / shaders / GPU noob here. Does that mean you'll essentially get free visualisations (albeit non-sensical ones) as a byproduct of your computations?

I think it depends, but given an arbitrary compute pipeline, you should be able to write the results (or intermediary results) to the screen with minimal effort.

No, it's an @compute shader rather than a combination of @vertex and @fragment shaders (which would do graphics) in the case of WebGPU.

Surely you could visualize it but not as a side effect.

If you wanted to do compute in a shader before WebGPU with WebGL instead, then I think the answer is kind of yes. It wasn't "for free" without any code but it was required to do. But now WebGPU supports compute shaders properly so you don't have to do compute in a shader that produces textures.

A texture in WebGL is just a memory buffer that can be accessed by shaders. It doesn't end up on screen automatically.

Fragment shaders can only go so far. Can you do in webgpu something like this?

outputTexture2d[inputTexture2d[inputPosition]]++

In other words, if you have a large texture with (x,y) coordinates of points, can you draw another texture that shows a density cloud of these points? In webgl2 it becomes a phd level problem.

One needs to do multiple passes, not sure if that is to be considered a PhD level problem.

I think Steven Wittens’s work at acko.net and https://usegpu.live/ is your best bet here. I’ve been working for a couple of years on building a computer algebra / physics system in Clojure and integrating it with Steven’s previous work on MathBox, with some in-progress demos like this:

https://emmy-viewers.mentat.org/dev/examples/simulation/quar...

https://emmy-viewers.mentat.org/dev/examples/simulation/toro...

Use.GPU looks like it will be close to a drop-in replacement for MathBox, running on WebGPU. Maybe someday I’ll actually be able to build my general relativistic ray tracer, with explorable symbolic physics all the day down…

The WebGL equivalent sites were a delight to go through; So well structured and thoughtfully displayed. It's so great to find a resource you can put some trust in amongst all the outdated and fragmented code out there.

Looking forward to ploughing through this when I have the free time to.

Question for discussion: when have we reached the point where the Browser Feature-creep is too much and we say "okay, a browser probably doesn't need this"? Because after seeing WebUSB and WebGPU I think my personal limit has been reached.

I don't see a problem you don't have to use it

the more important question is whether the browser's surface area for bugs and perhaps security vulnerabilities are worth the marginal increase in features that most people might not use!

You can find a security focused browser that don't implement these.

Please, let me know which one!

Firefox doesn't by default (and when that eventually changes, disabling it is an about:config tweak away).

It's not like native OS features don't have security vulnerabilities. At least all the major browser have at some major portion of their implementations that are open source projects. So if you're stuck working on Windows or MacOS, at least you can know that Web browsers aren't limited to one vendor finding and fixing those defects.

The browser is basically an OS ontop of an OS ontop of. .. turtles all the way down

The browser is basically an OS ontop of an OS ontop of. .. turtles all the way down

The OS runs on hardware, what do you think is below the OS you're running?

The OS runs on hardware, what do you think is below the OS you're running?

Even excluding VMs, there are a lot of OSes running in a modern computer. Some chips have their own. BIOSes and Secure Enclave, networking chips, and probably a dozen more.

Try to understand the context of more than one comment at a time if you can. This person said "it's turtles all the way down" because there are two layers, the browser and the OS, which is ridiculous.

Try to understand the context of more than one comment at a time if you can.

Try not to be a condescending dickweed, if you can.

Intel has its ME which last I heard runs Minix, AMD has the PSP, though I don't know what kind of software that runs.

WebPSU - this will allow your browser to directly access your computer's power supply. simply plug your appliances into your browser and they can use the same power that your computer has access to.

Why does it bother you that browsers have many features? At this point browsers are essentially a cross-platform VM and it really seems like this trend will only accentuate in the future.

At some point we're going to have to drop the pretense of running an operating system underneath because the browser is the operating system.

Blink is my operating system and Linux is its bootloader.

How’s chromeos treating you?

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=chromium

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=chrome

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=firefox

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=webkit

Those are the main reasons. And seeing how browsers are essentially designed to run other peoples code with minimal to no vetting processes, I consider this a serious issue.

So how is this any different if we don’t have a browser? People still want to visit “websites” which in your world is even worse. Because now every website is a distinct application (yikes!). The problem is that people are willing to run arbitrary code in the first place.

How many times have _you_ as a developer, run arbitrary scripts from the internet? Blindly accepts packages that you have not vetted?

Too many to count.

Remember Flash? ;)

I would be very different if browsers were tools to view documents though. I am not saying that browsers don't have a purpose. I just question that browsers should answer the question about what their scope is with a loud "yes!"

Thanks for sharing, that's interesting angle that I did not consider. So essentially the argument goes that more "surface area" == more vulnerabilities? While that makes perfect sense, don't you feel like the pressure for more feature comes from somewhere? By analogy, dynamic languages like JS/Python make the job of people writing the compiler much, much harder. But, on the whole and despite all their flaws, it is true that they increased productivity by a lot.

https://www.yubico.com/support/issue-rating-system/security-...

Every new feature is more space for security issues to exist.

because I want limited scope tools. if I want something on the GPU, I can just write a native tool. I dont need the browser to do anything beyond web browsing.

Whatever features browsers have, it does not prevent you from writing native tools.

But browsers don't limit themselves to serving your needs only, and they are not in the business of promoting your approach to computing among their users. Why would they?

But it does prevent the browser from excelling at their original purpose: To browse the web.

One example of many: Backspace no longer taking your to the previous page because that conflicts with usage in web apps.

Yes, the browsers evolved very much beyond this original purpose.

Again, it did not happen without any reason, it happened for a reason. The purposes the modern browsers serve apparently are more important to their users than that original purpose!

Indeed, I believe there are obscure browsers that don't support any of this new stuff, and are only good for rendering plain old HTML. They remain obscure for the obvious reason: they cannot be used for what most people use the browsers most of the time.

well you should have no problem implementing your own static web browser then, why complain about the binaries others are running?

> Because after seeing WebUSB and WebGPU I think my personal limit has been reached.

Sure, but where do you draw the line really? For me, having WebUSB and WebMIDI for example is useful, I want to be able to interact with synths over MIDI in the browser, or be able to access other accessories. I also love the idea of GPU access, so my personal limit has not been reached.

Multiply this by every vendor, developer and user of every browser who contributes to the specifications, and you end up with probably 1000 different directions everyone wants to move in. How do you decide which is the right direction?

Right now, the choice you have is which browser you use. If you don't want WebUSB or WebGPU or anything else "new and fancy", choose a browser that doesn't implement those things.

Browsers are just “standardised” OS at this point. Really, what is the difference? It seems to me that people basically wants a 1 to 1 mapping between every OS feature to browsers. I wouldn’t be surprised if this standard will fall apart in the next decade once Chrome runs everything. That is to say Chrome is the standard. I am already seeing websites drop support for Firefox and won’t even load using it.

> I am already seeing websites drop support for Firefox

Examples?

The Honda Financial website is Chrome-only.

https://honda.americanhondafinance.com

Firefox 112 doesn't seem to have any problems rendering it. What do you mean by "Chrome-only"?

I mean it doesn’t do any QA on them. My banking website won’t load on Firefox for example.

What's your bank considering we still haven't been given an example? The Honda one works fine in Firefox.

It's sad that there are specific browser-oriented websites (and development processes, obviously) instead of the standards-oriented ones.

(Sure, it's Chrome-oriented ones. We've seen similar previously with IE, by the way.)

We have standards for the web. Real ones: the docs, which are discussed and approved in the industry. We have them for a long time!

So if some browser does not comply to the standards, it's really not the best strategy to adapt a site to the browser instead of the standards.

We are in the situation when (effectively) one company (Google/Alphabet) can lead anything to the whole market, step by step (even when changes contradict the web standards that are in place). The market is not the browsers market, of coyrse, but the internet ads through browsers control, which brings the most money to Google. By projecting its power to each and any aspect of it, Google ensures the uninterrupted market control for years ahead. So Google will continue to do. In the long run, we need to rely on standards instead of specific browsers. Otherwise it's just the monopoly of Google and web tech "market" is just their own backyard. That will bite us all hard.

Having a standard is not even possible technically when you have 1 player that is too good. Due to Hyrum’s Law, any small divergence from the spec will be observed and relied on. Why would you work against the spec which is nebulous, when you could be testing against 99% of what your user use?

> ...when you have 1 player that is too good.

It's not too good. It's just wealthiest. Because it holds ads market monopoly. Because it happens to be the popular search engine at the same time.

But it's not the best. Firefox is on par (I know they get some (most?) payments from Alphabet). And people were using Firefox/Netscape browser long before Google existed.

I've found enough Firefox-only bugs when doing things through spanish government sites that I started using Chrome for them preemptively.

I'm not sure if these are examples of dropped support, but I run into issues on websites that prevent me from doing something I really need to do: - I could not unsubscribe from amazon prime yesterday using firefox. The page where you select the option was not rendering correctly. It was white for half the page vertically and the link/button I need to press was absent. - about 6 months ago I could not sign into apple id on apples site on firefox. (or something like this, I forget exactly what I was trying to do). - about 6 months ago I could not sign into nintendo's site to cancel a subscription.

So it's not super frequent, but every few months there are important things I can't do in firefox.

In my experience, problems like that are almost always a matter of cookie/cache sticking around when it shouldn't or plugin interference. The only sites I ever have have blocking trouble with in FF are shitty web interfaces for local device configuration, old automatically generated webpages like from MS Access or some other super old enterprise abomination. I worked on a team of web developers that generally developed using FF and then tested heavily in chrome-- everything from simple pages augmented with JS to complex SPAs-- and the differences were pretty minimal.

Yeah that could possibly be it. If I run into again I could try clearing caches. And also wanted to mention that since firefox is what I use daily, of course I will mainly see issues there.

If I used chrome daily perhaps I would see the opposite (broken on chrome, works on firefox).

I get the same feel from DuckDuckGo. I use it, till it doesn't work, then switch to google when it doesn't. Of course google would perform better, as I only use it for the cases where DDG fails.

Firefox has a better plugin ecosystem, and it's plugins that cause a lot of site issues.

It's a very clear trade-off in the hands of the user, which is correct.

Yeah, with one exception it has always been dark reader that caused a page to render wrong in Firefox. The exception was some misconfigured oauth stuff that didn't work.

Snapchat web

Even if Chrome were the standard, there would still be variety due to plugins. A lot of sites have glitches in Chrome too, if you install the wrong plugins or turn stuff like third-party cookies off.

I'm not sure I trust people who say something doesn't work in Firefox, unless they tried it in a fresh profile with default settings.

I wonder if you could just duct tape a kernel to a modern browser and boot right into it. Ok, well, I know you can do that, but I wonder if you could get people to use it.

You'd need some tinkering to open a new browser window per monitor, couple of routes on 127.0.0.1 for config.

That's basically ChromeOS, right? Some people seem to use it.

The difference is that it is not a "standardized" OS interface. Rather it's a complex layer on top of operating systems offering dumbed down and less performant versions of what is available all of which is interfaced by a dynamic and not so performant language.

If it was something like a modern POSIX I don't think you'd have much complaining. But it's a lot heavier than that.

In the context of this thread it is enough to compare Web 3D capabilities and tooling, and how lacking they are when compared against with the native conterparts.

I want to be able to interact with synths over MIDI in the browser

Help me understand why this is. Is it because there aren't native programs for the platform you're using? Is it to allow plug-ins or other abilities that wouldn't otherwise be available? Is it so that you can sync up with other musicians and play together in a way that wouldn't be possible without a browser?

choose a browser that doesn't implement those things.

The way feature creep have been going lately, in about six months that will mean Lynx.

Because it's really cool to try out experimental instruments other poeple have made by simply opening a webpage, without having to execute untrusted native code that could have bad consequences.

AKA laziness. yes, lets please bloat of the scope of browsers until they are no longer recognizable, so that you dont have to install some software.

You oversaw the untrusted part, this is the only reason I prefer web over native really. If there was a way to run native apps with that level of isolation, I would prefer native.

Snap, Flatpack, iOS sandbox, Android sandbox, UWP/Windows sandbox,...

Unfortunately, none of those are cross-platform... Closest we get to something similar to the web is either the JVM or APE (Actually Portable Executable) but then those are generally not as isolated as the alternatives you mentioned, sadly.

Or CLR, or now the fashionable WebAssembly.

Plus, plenty of languages have cross-platform runtimes and libraries, so not a big issue, not everything needs to be JavaScript.

Instead, you're executing untrusted browser code, which is hardly better.

I'm interested if this is true. It seems like the js sandbox is pretty well implemented now in most cases. Webgl had some problems at first leaking host data, but the willingness to sacrifice performance for security seems pretty ingrained in these consortiums?

I don't understand how you can possibly say this with a straight face.

> Help me understand why this is. Is it because there aren't native programs for the platform you're using?

Easier to create, easier to share and run. Mainly easier to create because the ones I'm sharing small MIDI sequencer experiments with are also web developers, so we just send each other links where we can run stuff direction from, and we can help each other out as we all use the same technology. Really easy to understand what the other is doing too, as you just open up the source of the page and that's it.

I've played around with other languages/runtimes for doing the same thing, but nothing is as fast to implement as with JavaScript, probably mostly due to familiarity.

Tangentially in the WebMIDI topic, I wonder how DAWs will react with things going to browser and generative audio scaling quickly.

I'm not going to argue against these features, but I find that last argument you raise to be a rather dull and useless one. Yeah, I'll just go ahead and pick from one of the hundreds of competing implementations to find one that doesn't. Oh, wait, they don't exist.

It's like saying "if you don't like the laws(/taxes/whatever) where you live, go somewhere else" and acting as if I can just hop on over to Mars. I can't.

That is exactly what they are saying. Having control over your relative experience is the only thing people want.

As we all spend most of our time inside of browser interfaces its not a surprise we see app, features and interfaces move to that process portal to. (Online IDEs, Cloud, Photo editing, etc.) Sure you can use your solution but in general, as always, its whats most efficient for the masses.

Couldnt there be plugins for these things?

I tend to feel that way mainly towards new features that are very similar to ones that already exist. eg. WebGPU being a more advanced form of WebGL. Yes they work differently, but they generally solve the same problem.

Eventually I'd like to see WebGL deprecated, and perhaps an extension released to reimplement it. Or it could be adapted to translate calls to WebGPU, rather than having a completely separate implementation.

Is that realistic? No, of course not. Browser vendors take backwards compatibility pretty seriously, and WebGL is used on a ton of websites. Still, that'd be my preferred solution for avoiding said feature creep.

That's actually not unreasonable, as WebGL under the hood is already implemented through Vulkan, Metal, and DX12.

Free yourself from the notion that a web browser purely browses The Web.

It's nowadays also a cross-platform insta-deployment GUI application runtime environment with mostly bad native OS integration and performance characteristics (both ~improving, there's even native filesystem access now).

In any case it's Good Enough (C), so it sticks.

there's native filesystem access?

edit: wow, I had no idea and I've been a web dev for over a decade lol

> Free yourself from the notion that a web browser purely browses The Web.

Free yourself from the notion that a web browser should do anything more than browsing the web. native applications have existed for decades, no reason to bloat the scope of a browser.

One big reason is that installing the software is one click (on the link) and it works every time. With native software, on phones you have to open the App Store, search for an app and usually wait to download hundreds of megabytes before anything happens. On personal computers it's much worse.

Hmm, RGamma describes reality: it is a fact that web browsers don't purely browse The Web, has been for decades now, and there is zero indication this is going to change.

You seem to demand that people ignore this reality, because you don't like it. This is not helpful.

I don't like this reality either, but despite that I find your statement also factually incorrect: "native applications have existed for decades, no reason to bloat the scope of a browser".

Of course there are reasons, otherwise people/companies won't do it, and users won't use bloated browser that give them no benefits.

I agree that downsides outweigh the upsides, but the upsides are obvious and immediate while the downsides are long-term and mostly subtle...

> Of course there are reasons, otherwise people/companies won't do it, and users won't use bloated browser that give them no benefits.

The reason is of course control. Things running in the browser gives Google control. That's why they push for everything running there. Want to compete with Google? Tough luck, no tracking for you while they give themselves IDs built into the browser. Do something Google doesn't like? Maybe your site isn't "safe" enough according to Google and won't be shown to users. Invested in your webapp but want to integrate something novel? Guess who gets to decide if you can?

I wish it wasn't true either.

Seeing a semi-technical acquaintance use in-browser software for daily productive work hurts my programmer's soul. It's alright for some light things, like vacation planning, but remote desktop or an IDE...

Then there's that other scourge of Electron-like apps with gigantic memory budgets and all. I have resigned to throwing more hardware at it when I can't guarantee a lighter replacement (i.e. in a work setting):

All my productive systems have 32GB RAM and 8+ threads now and with browser, Teams, Outlook, IDE and dev containers the air's getting thin again :/

I think I would rather have a bloated (but reasonably fast and featureful) browser, than clog my computer with native apps for every app out there. Most of the apps doesnt need much power anyway.

I really dont want to see a repeat of phone apps for every company I interact with.

Most "apps" including "webapps" shouldn't be interactive programs to begin with but a web of documents you can explore using a tool made to your specifications and noone elses.

The issue of the Web's lack of distinction between document viewing features and being an application platform (with access to system resources and private data) is decades old. Unfortunately, the question has been academic for decades as well: before the advent of HTML5, proprietary browser plugins gave Web sites the same abilities (ActiveX and Flash being notable, as were their security vulnerabilities). Flash was ubiquitous circa 1998-2008; your typical user would always have it installed. Nor did the typical (non-techie) user know that the Web was not originally intended to be an application platform, or the implications of it being used as such.

The more of this features in the browsers - the sooner the stranglehold of the app stores could be broken

Or you will just find yourself in the domain of a new lord.

I used webUSB to make an NFC login system and it was awesome, what’s the problem?

Who's the royal we here? It's great that you have personal limits. But why should "we" care about those?

Seems like a perfectly reasonable new feature to me. I'm actually looking forward to people using this to build some cool stuff. Why should that not be a thing?

WebUSB was actually useful to me not long ago. I installed Home Assistant and its ESPHome[1] companion, and ESPHome had support for using WebUSB to upload the firmware to a blank ESP8266 I had. I didn't have to install or set up anything on my Windows machine, and the whole process was very quick and convenient.

It seems clear to me that the main issue here is that a lot of programs aren't closely tied to the hardware or the OS. Building them to be so is an annoyance for both users and developers. As Scott McNealy said, you don't need to know how to operate a nuclear power plant to turn on the lights, and most users just want to get stuff done.

That we have three major operating systems still going relatively strong is evidence that there are differences that matters to people. But for a lot of applications those differences don't. The WebXYZ stuff thus increases usability a lot by simultaneously solving the cross-platform compatibility issues and distribution.

In my ESPHome example above literally opened the website, plugged in the USB cable to my device, clicked a button on the web page and the job was done.

So while I don't really like that the same web browser is used to surf other web sites, given that these WebXYZ components present additional security risks and fingerprinting opportunities, I can't deny that they're useful.

[1]: https://esphome.io/guides/getting_started_hassio.html

I had a similar pleasant experience with a Wooting 60HE keyboard I purchased recently. Updated my keyboard’s firmware and settings right in Chrome! So much nicer than downloading icky gaming peripheral software. I’ve been told a lot of that stuff is borderline spyware.

Excited to try it out with some ESPHome projects too now.

I think of the browser as another kind of OS, and as Microsoft keeps making their OS worse the browser is a nice reprieve. The more that goes into the browser the more I can ditch Microsoft and hopefully be rid of it all together.

Though ironically maybe given that the browser can do so much is the reason that Microsoft has to focus on ‘value add’ crap instead of just a rock solid OS. Which sets up a self reinforcing cycle. Microsoft should recognize that this was intentional move by Google and not take the bait.

WebGPU in browser isn't really interesting to me. But I do like WebGPU in native applications, as we are yet again in scenario where Windows prefers DX, OSX their own Metal and Vulkan is yet again mostly linux only player, and popular WebGPU implementations have backends to all of these (+ more).

Yep. I just read through the entirety of this guide specifically to better understand Mach, which uses Google's WebGPU implementation for everything graphics-related.

Windows supports Vulkan just fine. People use DX because of Xbox and tooling - the existence of yet another API won't change that. And neither will WebGL be better on macOS than MolenVK.

Yes until it doesn't or vulkan implementation is considered as second class citizen. Been there seen that. The other thing is that webgpu is slightly higher-level so you do not have to go into extremes with memory handling and synchronization as in vulkan. So unless you absolutely need to squeeze everything out, webgpu gives nice balance.

I think one of the main drivers of new browser features is ChromeOS. Manufacturers are selling a lot of Chromebooks, especially to schools, and because ChromeOS is primarily a conduit for running Chrome, the best way to add features to Chromebooks is by adding to the browser. From Google's point of view, anything ChromeOS needs is a candidate for a new browser feature.

In the age of app stores and notarized binaries, anything that makes the browser non-trivially more competitive with native apps is defensible.

And you think your webapp is safe from those kinds of requirements? Not when one company runs the show.

I think WebCPU is where it's at. First class virtualization in the browser so you don't have to emulate :)

Why not? People are already running VM's and other OS's in the browser.

https://github.com/atrosinenko/qemujs

https://jamesfriend.com.au/pce-js/ibmpc-games/

https://leaningtech.com/webvm-server-less-x86-virtual-machin...

and so on...

Why exactly would programs running in web browsers not want the ability to access the most powerful piece of hardware in a users machine? WGPU unlocks an enormous amount of space for userland programs to explore.

I agree there have been some questionable 'advancements' in the web spec recently (web workers as a solution to multithreading, for example, were hilariously inadequate for quite a while), but as far as I can tell WGPU is a solid effort to unlock browsers as an actually interactive platform, instead of the fairly static image/text display devices they are today.

Maybe our visions of what browsers could or 'should' be are different

At least for me a browser should be an interactive document viewer. Not CAD, not file conversion, not writing OS images to an SD card, not generating GCode. Ingesting HTML, CSS, and JS and turning it into an interactive view of that XML document nothing more, nothing less.

But the web is where the stuff I want is. Just about everything else gets in the way.

It will stop once it's easier, more reliable and more secure to run someone's code outside your browser.

How do I render text with GPU? Do I render text on texture with CPU and then send that texture to GPU? Do I vectorize it and draw millions of tiny triangles to approximate those vectors? It's interesting for me to explore WebGPU as a canvas for app development, but it's not obvious to me how to draw text and text is everywhere.

Some people have been using signed distance fields (sdf) to define fonts on the GPU.

Note that this approach is useless on its own if your app needs to render user-created text because nowadays everybody expects emojis to work and look roughly the same as on Apple’s platforms, which means detailed multi-colored layered vector shapes that SDF font renderers can’t handle.

The solution I've used for this, which was a bit tedious but not that hard to implement, was sprites for the icons/emojis and SDF for everything else. I'm sure there would be other solutions too, like layering. So it's not useless, you just need to get creative to overcome the limitations. Like everything else in 3D graphics.

Yes, I agree — I did write “useless on its own” to highlight that you need to extend the SDF rendering model with a different one to handle actual user-generated text.

Use.GPU handles this just fine: it uses a subpixel SDF mask with a plain color image. You get crisp edges, just the interior is a bit blurry.

If you have a couple of different sizes of texts you can render out each character into a sprite map and render a quad per character which is probably the fastest and works for most 2D UIs. You can prerender ahead of time or have some runtime code that renders the characters with a given font using the 2D canvas api. You will need also need to store the characters sizes somewhere unless it’s a monospace font

If you need to scale the text arbitrarily or render it in 3D space you can look at multi channel signed distance fields (MSDF) which also renders out each character but encodes some extra data which makes them scalable. It has pretty good results

The easiest is just to use a html canvas element and draw text to it and then upload that as a texture map. This can handle kerning, ligatures, emojis, different fonts and languages. Writing your own font layout tool is horribly difficult and only do it if you must.

One of the most exciting features of WebGPU, especially over WebGL, is the ability to run compute shaders. That opens up an entire class of 2D vector graphics rendering techniques using compute shaders, including Vello. It's still in early stages, but the current code does include text rendering. There's a very rough demo at https://levien.com/vello-demo (the demo doesn't do variable fonts but the code can), and we'll be polishing that in time for Chrome M113 going stable.

Thanks to Google blocking WebGL compute shaders adoption.

We could have had them three years ago.

Interesting. Do you have any more context around this? It sounds uncharacteristic of Google to do so given their usual stance on browsers being all things to all people.

Yes,

https://registry.khronos.org/webgl/specs/latest/2.0-compute/

https://github.com/9ballsyndrome/WebGL_Compute_shader

And then Google decides not implementating it after all, because of WebGPU.

https://github.com/9ballsyndrome/WebGL_Compute_shader/issues...

There's a lot of literature about font rendering predating compute shaders too, compute shaders are not a functional prerequisite by any means.

Here's eg one case: https://github.com/astiopin/webgl_fonts

Yeah, that as well.

WebGL 2.0 is kind of PlayStation 3 like in capabilities, that we seldom see that in action besides shadertoy, Google Maps/Earth, and 360° views on ecommerce sites is another matter.

Yes, I also don't understand why google abandon webgl compute shader, just because webgpu will have same, so they force developer using newer api.

Is it (will it be) possible to use Vello from C++ or is this firmly Rust only?

There are no active plans to do this, but it's all open source and there are tools such as cbindgen, so if there is sufficient interest there's a good chance it will happen.

> Do I vectorize it and draw millions of tiny triangles to approximate those vectors

That is becoming feasible, using something called mesh shaders. The millions of tiny triangles will only be created on the fly inside the GPU, you will just send the vector geometry.

This is true, but there is a lot more to the story. For one, WebGPU does not (yet) support mesh shaders, though it may later as an extension. For two, consider a glyph such as "o" that has two contours. Real triangulation generates a mesh that only generates triangles between the outer and inner contours, and mesh shaders aren't good at that. There are techniques compatible with mesh shaders (cover and stencil) that draw twice, incrementing and decrementing a winding number stored in the stencil buffer (see contrast renderer[1] for a clean modern implementation), but it does require nontrivial tracking on the CPU side, and can result in lots of draw calls to switch between the cover and stencil stages unless sophisticated batching is done. (The outer contour is drawn as a disc with winding number +1, and the inner contour is drawn as a smaller disc with winding number -1, so the inner part ends up as 0)

Compute shaders avoid all these problems and work on WebGPU 1.0 today.

[1]: https://github.com/Lichtso/contrast_renderer

> Real triangulation generates a mesh that only generates triangles between the outer and inner contours, and mesh shaders aren't good at that.

I'm a bit confused. Can't you send the shape of O as a low res rough band (a closed wide loop) and enhance it in the mesh shader? This is how previous generation tessellation shaders and sub-division surfaces worked.

You might be able to do something along those lines, but I know of no font / vector 2D renderer that does so. There is a lot of literature[1] on this, so I suspect if doing things similar to subdivision surfaces were viable for font rendering, it would have been explored by now.

[1]: https://2d.graphics/book/gpu_rendering.html

It depends on the quality and performance requirements, but you are basically on your own (e.g. the browser's integrated text rendering won't be of much help unfortunately - at least if performance matters). You need to bring your own font data and 'pre-rasterize' glyphs either on the CPU or GPU into a font cache texture, or render directly on the GPU using an approach like Slug or Pathfinder.

Of course there's also always the option to overlay DOM elements over the WebGPU canvas if you just want to render some 2D UI on top of the 3D rendering.

You could convert the paths in a font file into a series of triangles using some vertices as a control points for quadratic bezier curves. Then some code for laying out the glyphs in a line. Much of this could be done on a shader I think.

Most web games use the dom as a layer over top of the 3D canvas and then use 3D to 2D projections to align the elements. This allows you to use regular HTML and CSS for a lot of text elements.

Also, you can render and rasterize fonts + dynamic text to a texture and UV map it to a quad in 3D space if you need to. This is pretty inexpensive and easy to do as well.

Lastly, the most difficult but performant option is using a shader to compute the font rendering. This is basically moving the font raster from CPU to GPU and there are various shader examples for this in GLSL (popular shader format used by WebGL/OpenGL) that you can use.

From personal experience, if you're trying to make a game that cares about its frame rate, do not ever use the DOM for anything, ever. It routinely takes several frames to layout a single absolutely positioned element.

I recommend the second option. Rasterize your font with stb_truetype to a bitmap, and draw quads yourself

I'm curious, isn't something like a HUD a perfect thing to render in a DOM overlay?

It shouldn't matter that much if it takes a few frames to layout and/or update your ammo count or whatever.

It wouldn't matter if the DOM layout engine ran on a separate thread, but it runs on the main thread, so it blocks your games update loop for a few frames every time you shoot, or whatever.

Now, that's not to say it has to be that way; real engines have a separate render thread that would sort of solve that problem. And they run the game update code in parallel, and all kinds of other stuff. But, by the time you have all that stuff, you can probably rasterize a font and draw quads ;)

But don't take my word for it. Maybe it's better now. Make a thing that moves some divs around in a requestAnimationFrame callback and take a look at the profile. I bet 2% of the time the layout engine takes 30ms+ to do basically nothing

This doesn't work for webxr.

This is great! I have been meaning to get more into WebGPU!

One question: Does anyone know more details about the Firefox WebGPU story? Even on this site's homepage, if you access it from Chromium with WebGPU enabled, you get nice floating rotating triangles, if you access it from Firefox stable, you get, understandably, static triangles. But if you access from Firefox nightly, with WebGPU enabled, you get a dark background and the error:

    TypeError: navigator.gpu.getPreferredCanvasFormat is not a function

Firefox doesn't implement that method yet: https://developer.mozilla.org/en-US/docs/Web/API/GPU/getPref...

You can probably use "bgra8unorm" where getPreferredCanvasFormat isn't available, that seems to be the preference for at least Blink and WebKit.

Chrome doesn't support rgba8unorm on Mac, probably because Apple seems to have chosen bgra8unorm as their format for their Metal API for some reason, so you'll have to use BRG rather than RGB if you want to support Apple hardware at the moment.

Regarding all the arguments about ever-increasing browser complexity and the resulting inefficiency:

Why is nobody asking the reverse question: How to make operating systems more like browsers??

From the end user's PoV (consider someone who isn't computer savvy) what are the actual objective differences in usability, between the OS and browser?

• OS: Apps have their own windows and menu bars.

• Browser: Multiple apps run in a single window, and have no menu bar.

• OS: Need to download and "install" apps before you can use them.

• Browser: Just type the app's "name".

• OS: Need to make sure apps are compatible with your OS and hardware. Need to keep apps updated. Need to delete apps when out of local storage space.

• Browser: Nope.

• Browser: Lets you share links with people to show them exactly what you're seeing in an app (YouTube, HN, Reddit)

• OS: Nope.

SO. Instead of constantly reinventing the wheels that have been perfected by operating systems for centuries, and then machete'ing through the resulting struggles of power and politics between the OS and browser,

WHY NOT REMOVE THOSE HURDLES FROM THE OS?

one can hope that with web assembly we can get closer to that dream, and eventually just remove the whole web browser.

.jar files exist and so far they haven't replaced any browsers as far as I can tell.

All of this is perfectly possible. End users don't seem to want to use these features for native apps for some reason, though, and developers seem to have all switched to the web instead of building native experiences.

It makes sense: native apps need to be developed for Windows, macOS, Android, iOS, and maybe Linux. Web apps need to be developed once. Android apps will work on Android, ChromeOS, Linux (after some setup), and Windows, but I doubt Apple is going to add an Android runtime to their platform any time soon.

App stores are on every major platform and custom URIs for sharing app state and views are implemented on phones already. Most people seem to run their desktop applications either full screen or nearly full screen. Pin the Windows task bar to the top of the screen and you've basically got yourself a tab bar. I don't know about macOS, but Android and Windows will start clearing caches and temporary files automatically under storage pressure. Sandboxing applications is available on all platforms as well.

I don't want every website to become a native application, though. I don't trust most websites enough to download native code from them. Let's not go back to the ActiveX/Java applet days where you needed to download some scary code for every website you wanted to visit.

I don't see why I would benefit from WebGPU support, I think I'll leave it disabled by default. It's probably one of those technologies that'll be useful for platforms with overbearing control (i.e. how game streaming on iOS is made possible). Maybe some small itch.io games will make use of it so I don't need to download Some Guy(TM)'s native executable every time I want to try a small experimental game.

They already did, it is called app stores.

The better question is do we even need so many apps to begin with? Do we need YouTube, Dailymotion, TikTok, Vimeo all to be their own app? Or should they all just use the something like NNTP but for video sites?

I have been trying to add WebGPU support to https://www.fciv.net - however the Three.js 3D engine I'm using isn't ready for WebGPU yet, and no current browsers support WebGPU yet.

Chrome supports webgpu on desktop.

Right now it's in beta (Chromium 113), stable support is planned for 26th of April

The Three.js people have been working on WebGPU support for several years now. I'm not how far they've got but there were already some cool demos a year or more ago.

works for me

https://threejs.org/examples/?q=webgpu

To see the spreading support for webgpu check out https://web3dsurvey.com

Unfortunately no source published as far as I can see. Would be handy to have available + issue tracker.

Since you seem to be the one responsible, I'll write my feedback here instead of a issue tracker:

It shouldn't recommend hot-linking the script, but rather to include the script into your project (or at least include a Subresource Integrity attribute so it doesn't change unexpectedly). Also feels like it's unnecessarily minified as it's so small, hardly makes a difference in payload size, but makes it a lot harder to review.

I purposely didn’t mangle the collection script. You can read it easily with all original variable names. Just format it which many browsers do automatically these days. I did this so you can verify its behavior.

I can add a npm package as well so people can embed the collection script as well.

I will also add an integrity check as well.

Highly unavailable still, but it seems to be non-experimental in chrome 113-115, as of writing this

https://caniuse.com/?search=webgpu

Support should land in stable in 11 days

I haven't kept up with WebGPU. As a more casual web user (and not a web developer) usually these features introduce more annoying webpages, as well as some new way to be fingerprinted as you move across the internet. Can anyone comment on this? Are there privacy mitigations that users should be enforcing to ensure their privacy is kept intact?

You can disable JavaScript or WebGPU in browser prefs. (Once it ships...)

Can this somehow be leveraged to crowd-train LLMs or maybe do something like SETI@home within the browser?

From what I understand not at this time but potentially in the future with different algorithms. DL training requires long sequential computations that can't be done in parallel the way folding at home could.

No, distributed training is one of these holy grails nobody has quite figured out yet. You can split training across multiple GPUs in the same computer, or multiple computers in close proximity connected by infiniband, but that's about as much latency as we can handle right now. We need some breakthrough to make it possible for internet-connected computers to usefully collaborate.

If it did work we wouldn't be waiting around for WebGPU, it would already exist as a desktop program.

Pardon my naivety. The only reason individuals would willingly participate at a large scale, would be for an open source AI project. Does that mean once the problem is solved, everybody will likely be able to run their own state of the art AI, or are the real-time calculations too intensive for a single server?

As a reference point, the large version of facebook's LLaMA was originally designed to be run on 4 server-grade GPUs, but after it was leaked people managed to make it run on a normal computer with just a CPU and 40GB RAM or so, though it's somewhat slow when running that way. GPT 3 is about 3 times larger than that. But at least for LLMs we have mostly explored how to either make really big models with the best capabilities (OpenAI etc) or models that are faster to train for comparable capabilities (facebook etc). There's comparative little work done so far on models that are able to run with the least amount of resources possible, though there are indications that you can invest more resources in training to get models that are much easier to run (training a small model for really long to get the same capabilities as a larger model). That might be a great route for an open source project, spending great combined effort to make something everyone can more easily run at home.

Of course that's mostly LLMs, for image generation you can easily run Stable Diffusion on any somewhat decent GPU, and being able to collaboratively train better models might be a huge boon there too.

What are the main obstacles they need to be overcome to make this a reality?

I wish they had kept the c/c++ syntax style for WGSL, the rust syntax is just awful and alienating.

I much prefer the new syntax. The C based shader languages always end up in some kind of uncanny valley where it looks like C and C++ but behaves completely differently.

It's not exactly Rust either, the @annotations aren't written like that in Rust for example.

The let-syntax is closer to Javascript with type annotations, so that makes more sense for web programming than doing C style variable definitions. Using let and var instead of let and cons is a bit of a weird choice, though.

Other than the variable definition syntax, this syntax may as well have been C++-based without all the verbose names that C++ likes to add to its namespaces. Shades will usually mostly be math code, though, so it shouldn't even matter all that much in practice.

I feel like they made a lot of weird choices like let being the new const, while still having constants, that won’t be confusing at all. While disappointing I will try and port over some of my webgl stuff and see how I feel about it.

Not if you're a rust programmer!

Thankfully I am not and never will be.

Not sure how it remotely looks “awful and alienating”. It looks like basically every recent language to me (swift/kotlin/rust/typescript). I feel like you just saw something that has the same numeric type names as Rust and jumped to conclusions.

> swift/kotlin/rust/typescrip

I don't use any of these and because it just<does>

One of my favorite fundamentals I'm hoping for with WebGPU is for it to finally work on Linux/chromium.

Ah, this brings back memories of being blocked by Jonathan Blow on Twitter for poking fun at him for being really mean to the people who worked on WebGPU. https://i.imgur.com/ZDgXBXH.png

What is the “pollution” by WebGPU that he was referring to?

You’d have to ask him (I don’t recommend it) but I think he means the API is badly designed so it makes computers in worse general.

It is sad that WebGPU has pretty poor support under Linux apparently. Even more so, that it builds on Vulkan which isn't fully supported on older architectures (like Haswell.)

Compute on WebGPU is gonna be interesting- "crowd sourced" model training.

Great resource, but this website really doesn't want you going back to previous page.

It appears that the issue only affects pages containing a code editor (and Firefox-only).

I did a quick check and found that the editor is based on gfxfundamentals/live-editor, and it is doing stuff I'm not familiar with, such as injecting a blob for each editor and using some URL params hacks. I'm not sure what the issue is, but you can take a look at editor.js if you want to dig deeper.

What's certain is that you'll need to hit the back button 1 + n times (n being the number of editors on the page).

[edit] I have opened a GitHub issue for this

This appears to be a bug in Firefox

https://bugzilla.mozilla.org/show_bug.cgi?id=1828286

Trying to find a workaround

> It is expected you already know JavaScript before you read this article. Concepts like mapping arrays, destructuring assignment, spreading values, async/await, es6 modules, and more will be used extensively.

So in other words, it’s just like Vulkan where the beginner tutorials have you set up your own allocators, deal with fencing, and use a swap chain just to put a single triangle on the screen? What the actual fuck? This doesn’t sound at all like “fundamentals” and makes me not want to ever use it if the very basics require all of the above. I think your average programmer, even one who does 3D graphics, is going to look at this and just be turned off.

Most of those concepts are foundational to web programming, yes.

I also believe that for most people who "just want a triangle on the screen" they'll be utilizing other higher level tools to do so, and those tools will then do the "dirty" work of using WebGPU. Think, Figma making it easy for people to design but behind the scenes it's incredibly complex using C++ (might be outdated on this) compiled to WebAssembly.

Same will apply here, most people who want lower level access to the GPU aren't really doing so just because of simple triangles or basic graphics work. There are already engines and frameworks for them.

This is a beginner tutorial for WebGPU, not JavaScript. It's entirely reasonable to assume knowledge of language features.

WebGPU is not meant to be a high-level, beginner-accessible graphics API. It's meant to be a low-level GPU API that expert users can build nice tooling atop. It's an embodiment of the extensible web manifesto's principle that browsers ought to expose lower level hooks to enable API experimentation in user space, like we had with Promises starting out as a constellation of competing approaches in libraries.

Reply on news.ycombinator.com