This is pretty cool - great to see what a small codebase can do with raw WebGL. Have you looked at WebGPU at all?
295 karma · joined December 13, 2023
For the rive-renderer / 2D integration, it is going to be a much longer path to get working in a browser together with wander.
No, the goal is not to create a C++ API to give you GPU functions.
The C++ API for wander is used to embed the WebAssembly module of graphics code into the application. The API footprint is very small - load a file, pass parameters to it, iterate through the tree it produces.
This could be viewed as logically equivalent to programmatically loading a flash/swf file. Or similar to what Rive has built with a .riv, although this is static content, not code.
> 1) Why use WASM?
You're loading arbitrary, third-party code into an app - that is the renderlet. The benefit is to have a sandboxed environment to run code to put data on the GPU.
2) Why use renderlet instead of webGPU, which is already a high level cross platform abstraction including shader definitions?
WebGPU is a low-level API. If you are a graphics programmer, and want to build an app around WebGPU, go for it! A renderlet is more of a graphics plugin system than an entire first-party app.
> The renderlet compiler is currently in closed preview - please contact us for more information.
This is the system to build the renderlet. This is not writing raw C++ code to talk to WebGPU, this can be higher-level functions (build a grid, perform a geometric extrusion, generate a gradient) - you can see in the video it is a yaml specification. The compiler generate the necessary commands, vertex buffers, textures, etc, and soon, shaders to do this, and builds a Wasm module out of it.
> Is this like a 'alternative to webGPU' with a different API / easy mode tooling?
I certainly wouldn't describe it as an alternative to WebGPU, but easy(er) tooling to build graphics, yes.
> What is the use-case for 'I've compiled a part of my application only into a cross platform binary renderlet and I can now run that cross platform ... after I've compiled the rest of my application into a platform specific binary for the platform I'm running it on?'
Let's take an example - Temporal Anti-Aliasing. There are libraries that exist to implement this, or you can implement it through raw code. This requires changes structural changes to your pipeline - to your render targets, additional outputs to your shaders, running additional shaders, etc. Wouldn't it be nice to easy connect a module to your graphics pipeline that contains the code for this, and the shader stages, and works across platforms and graphics APIs, with data-driven configuration? That is the vision.
> ... rest of your application into WASM/platform native code... is that not strange? It seems strange to me
There is not really such a thing as a standalone Wasm application. It has seen great success as a data-driven plugin model. In a browser, it is hosted with / interacts with JavaScript. Even built for pure WASI, as a standalone app where everything is compiled into a single module, there is stil a runtime/host environment.
Does that help clarify?
I already see web even taking over in things like embedded UIs where native toolkits like QT historically were popular.
Text / fonts is very much on the roadmap! For input and audio I would have to think through the scope.
On the "backend", we will switch fully to wgpu as we retool around wasi-webgpu. I explicitly don't want to rebuild a project like wgpu, and everybody should commit upstream to that - we will likely have stuff to upstream as well.
The state of shared memory for Wasm is not great, although raw SharedArrayBuffers work ok in a browser for running multiple guests. Getting multi-memory properly working through llvm is likely a better solution.
We've got a bundle size issue as well even with -O3. I thought it was due to the amount of templated glm simd code we run, but now am convinced its deeper than that into Emscripten. Haven't been able to look into deeply yet.
The renderlet is a bundle of WebAssembly code that handles data flow for graphics objects. Input is just function parameters, output writes serialized data to a specific place in Wasm linear memory. With the Wasm Component Model, in the future can use much more complex types as input and output.
LoadFromFile() - Instantiates the Wasm module
Render() - runs the code in the module, wander uploads the output data to the GPU
Functions on the render tree - do things with the uploaded GPU data - like bind a texture to a slot, or ID3D11DeviceContext::Draw, for example.
There's some nuance about shading. In the current version, the host app is still responsible for attaching a shader, so should be no issue using the data in a deferred shading pipeline. In the future, the renderlet needs to be able to attach its own shaders, in which case it would have to be configured to use a host app's deferred shading pipeline. I think it is possible, but complicated, to build an API for this, where the host and then the renderlet are both involved in a lighting pass.
Of course, if all shading is handled within the renderlet, it entirely the concept of deferred shading, and this becomes an easier problem to solve.
On average, running the Wasm guest code is about 80% of the speed of a native build I use. That is both dependent on what is running in Wasm and not a very scientific measurement - wander needs better benchmarks. We think that performance profile is sufficient for anything that needs a GPU except the highest-performance 3D games.
I agree that the feature set around WebGPU is constrained and becoming outdated tech compared to native platforms. It shouldn't have taken this long just to get compute shaders into a browser, but here we are. The lack of programmable mesh pipelines is a barrier for a lot of games, and I know that's just the beginning.
For memory, architecturally, that's why I'm treating wander as a tree of nodes, each containing Wasm functions - everything gets its own stack, and there is a strategy to manage Store sizes in wasmtime. Deleting that is the only way to free memory vs a singular application compiled to Wasm with one stack/heap/etc. More of a data-driven visualization framework than a full engine like Bevy, which I still think is one of the most elegant ways to build browser based games and 3d environments.
I couldn't agree more. My goal is not to simply build "a better game engine", but to make this kind of low-level tech accessible at a higher level and with much better dev tools to a broader class of developers and applications
> Don't underestimate what an individual or small teams can produce if they are operating on a solid platform
This gets into my motivations for building a company - larger companies have the resources to build moats, but often can't quickly realign themselves to go after novel technical opportunities. It's not either / or - both models exist for very valid reasons.
To me, this reads like the intersection of "Web Components as Wasm" and "The Browser as an OS" - almost something analogous to WASI as browser APIs that are delivered via Wasm ABI instead of JS/WebIDL. It's an interesting take, and as long as it can operate alongside existing code, I'm all for that.
There are strong parallels to what we're building - small modules of Wasm graphics code that can interoperate across a common interface.
Check the repo for the GPU integration - it's like a super trimmed down version of wgpu, where graphics data is copied out of Wasm linear memory and a host specific API (WebGPU/OpenGL/DirectX) takes care of the upload to the GPU. There is a wasi-webgpu WebAssembly L1 proposal that I am involved with in the works, driven by Mendy Berger, and at some point all of this will be tooled on top of that with wgpu as a backend.
For renderlet the company, the goal is to build developer tools that make it easy to build renderlets and these kinds of applications without having to write raw Wasm code. The meta-compiler in the video is the first step in that direction! The runtime itself will always be open-source.
Worth noting that their original GPU backend was Skia, and now they are retooling around Flutter GPU (Impeller)[0], which is kind of designed similarly as an abstract rendering interface over platform-specific GPU APIs.
I was sad when UE4 sunset HTML5 support, and glad to see a spiritual successor! There are a lot of parallels to other large in-browser apps in terms of load time for games - not just for the content but the size of game code itself. Are you able to use streaming compilation or some sort of plugin model?
The host app owns the event loop. I don't foresee that changing even once we re-architect around WebGPU (allowing the Wasm guest to control shaders), as the host app is responsible for "driving" the render tree, including passing in state (like a timer used for animations). The host app owns the window pointer, as renderlets are always designed to be hosted in an environment (either an app or a browser). Open to feedback on this, though.
FFI is coming with the C API soon!
I don't know much about audio but I see a ton of parallels - well-defined data flow across a set of components running real-time, arbitrary code. Simulations also come to mind.
From an Adobe perspective - it doesn't. If you go to photoshop.adobe.com in Safari, you will see the answer. Things can work in a single-threaded build, but that is not production code.
I can't speak for the Safari team, but I do see this getting traction soon with the current priorities for Wasm. Seems like now the most common answer is just to use Chrome.