Use.GPU Goes Trad
acko.net
acko.net
I recently played around with using https://reactpixi.org/ to build a simple ant farm simulation (https://meomix.github.io/antfarm/) and was left disappointed regarding rendering performance, but not surprised. It's a real challenge to get strong numbers when stacking a declarative wrapper on an imperative base. I found myself fighting React's reconciler for performance almost immediately.
I was talking to my coworkers about the issue and one of them suggested trying to use react-three-fiber and just force it to render 2D, but, as the author notes, the problem feels intractable with competing layers of abstraction.
I'm really excited to learn about this library. I feel that I was between a rock and a place with my web-first, declarative graphics tooling. I was pushing myself to learn Rust, to use Bevvy, to have a well-supported, declarative framework, but I felt I would prototype quicker if I stuck with JavaScript. I considered A-Frame, but it's really not about 2D rendering at all with its VR-first approach.
There's really quite a desert of modern, active, declarative, web-first graphics rendering frameworks. Stoked for UseGPU to hit 1.0!
In Fiber to animate is to use useFrame, which happens to be outside of React and bears no extra cost or overhead, nor does it conflict with reactive prop updates. useFrame allows any individual component to tie its mutations into the render loop.
The reactivity debate is next to irrelevant in WebGL because that is a loop driven system, it is not event-driven like the DOM. If you doubt that observe Svelte/Threlte, which can update the view directly, yet relies on useFrame modeled after Fibers. That is because you need deltas (to be refresh-rate independent), flags (needsUpdate), imperative functions (updateProjectionMatrix etc), it is never just a prop update.
That said, i don't know much about React-Pixi/Konva and so on, and if these do not have a loop outlet then yes i agree. But the whole premise of that article just falls flat.
Since WebGPU is still quite experimental and not usable without install canary browser builds, I'm really hoping that Live can be used separately. Like the author, I'm pretty sick of doing the data retained/declarative dance with libraries like pixijs many times now.
At least from this article, it seems that the author is using their @use-gpu/state[2] package independently. Would love to know if anyone might have any more details or experience?
I’ve hit this with SwiftUI. Isn’t this a general problem of the entire “reactive” style of programming? If you have anything that requires a computation step after model changes, you need to gate when you take into account a set of model changes before you kick off your calculations or you end up with this problem of constant updates that kill performance. I don’t do a lot of front end work, but that’s been my experience with this method of programming in general.
Others may be more concerned with the applicability of the pattern itself.
The gap can be bridged for granular reactive systems as it can be for React, provided the reactive system is well designed.
In theory, these state updates shouldn't be a problem as your rendering logic and your state logic should be separate, at most rendering one or two frames wrong if the state updates are inconsistent. Video games have done this for over a decade and every "modern" GUI framework uses a GPU surface to paint its own controls rather than use existing system windowing, so you may as well take a page out of games' book.
Sadly, I don't think many frameworks do this stuff right and a lot of time is spend doing unnecessary state recalculations. It's certainly easier to write bug free GUI code through reactive programming, as state conflict bugs now get gracefully handled by the frameworks, but I think because these issues are now no longer clearly visible they also don't get recognised as bugs anymore.
I think this is just one of the many ways development has shifted towards developer comfort over user experience; changing a number in a region in memory and redrawing a string doesn't need to go through 50 chained method calls to update application state, but they make life for the devs easier so they're often tolerated. You get more features in return, but I'm not happy with the price we paid for those (mostly useless) features.
a) If you use memoization properly, and don't put all your state at the tree root, it doesn't.
b) In my experience, devs dramatically overestimate how useful such a hand-optimized fast path is, because in a real application, users expect the UI to always be consistent. It's not just about responding to a string change but also potentially the reason you're displaying a string in the first place.
And FWIW, Live is actually a lot leaner than React.
You never need to preemptively invalidate a cache, you just reevaluate in tree order.
And you can scroll up to see it in full 3d!
Nowadays people complain they can't see the WebGL on their phone.
I guess that's progress?
> When I tell people this, they often say "well, it might work well for your domain, but it couldn't possibly work for mine." And then I show them how to do it.
Signals-based reactivity is garbage collection on steroids. It does solve cache invalidation, and extends it to effects as well. It really should be a language feature at this point.
I wish the model could be extended to distributed systems, but adding network unreliability to the mix is non-trivial.
>Three.js might seem like a great choice for the job: it has a 3D scene, editing controls and so on. But, my scene is not the source of truth, it's the output of a process. The actual source of truth being live-edited is another tree that sits before it. So I need to solve a two-way synchronization problem between both. This requires careful reasoning about state changes.
Related, I don't feel comfortable giving up control of when a real DOM element is replace by a new instance. For some things, like canvas, file inputs, audio tags, this replacement needs to be carefully controlled or prevented. This seems to violate the react philosophy.
It's fairly straightforward. React uses a module-scoped variable, assigns to it when it starts rendering a component, and nulls it out when it's done rendering the component. Any other function in the hooks implementation file can then update that current "Fiber" object when the hook gets called by the component.
Shawn Swyx Wang has an excellent talk called "Getting Closure with React Hooks" where he builds a miniature version of this in about 20 minutes:
You can see the actual implementation in this file - I've highlighted that "current Fiber" variable:
https://github.com/facebook/react/blob/v18.0.0/packages/reac...
As for the "when a real DOM element is replaced" bit, _you_ have control over that. React will keep existing DOM nodes in place as long as you continue to tell it "render a node of this type in this spot in the tree". It only removes that DOM node when your render logic stops telling React it should exist (ie, a component is unmounted, or the component's render output changes and no longer includes that `<div>` or `<canvas>` or whatever type).
See my post "Guide to React Rendering Behavior" for some more details on this:
https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-...
What exactly is a "spot in the tree"? If I need to keep an element, must I ensure that all prior elements, in DOM order, are kept in place? What if I want the stable element to be preceded by a conditionally rendered element? Is that supported by react?
"UI is a function of state" sounds kind of cool, but in my mind "function of state" means that state is an input to the function, not acquired through back channels.
There's three major aspects that control what happens to the DOM nodes that React created for `<Child>`:
- If `<Parent>` stops rendering `<Child>`, React will completely unmount that `<Child>` instance, including removing all DOM nodes that were created for `<Child>`
- If `<Child>`'s rendering logic noticeably changes the HTML elements it returned, like replacing a `<p>` with a `<div>` at that same spot in its tree of returned elements, React will remove the DOM nodes that are no longer needed.
- If you identified specific child elements using the `key` prop, and you change the `key` you applied, React will take that as a signal to unmount the previous instance of that child and re-create it
So, loosely put: React will keep the existing DOM nodes as long as you're still telling it you want elements of that type, at that depth and path in the tree of elements you return. So, sure, if you had something like `return <div>{condition ? <A /> : null}<B /></div>`, then React will keep that `<B>` instance alive, because you're continuing to ask for it in the same location.
I'm not sure what you're trying to say with the "back channels" bit. Are you referring to how React manages hooks?
React has _always_ kept the real props and state values stored in its internal data structures. In that sense, both class components and function components are just facades over how React actually stores things. In fact, even with class components, React would assign `componentInstance.props = theFiberObject.props` at the last second before rendering it. The fact that hooks tie into React's internal Fiber objects is just a different syntax for the behavior React always had.
Per your other comment, it sounds like you are confused about how React rendering works. I'd _really_ encourage you to read my post that I linked above, which explains the general rules and behaviors. The behavior has also been described in the "Reconciliation" page in the React docs:
function fn(arg) {
let foo = useFoo();
// do something with arg and foo
}
Every React component, whether it's written as a class component or a function component, has props and optional state. In class components, those are accessed as `this.props` and `this.state`, which may seem "familiar" for folks who are used to OOP usage. But, as I note above, the fact that you can even access them as `this.something` is only because React itself has done the work to track those values in its internal `Fiber` structures, and assign them to the component fields when it's ready to render the component.
For function components, it's the same thing, with slightly different syntax. Function components always get `props` as the one argument to the function. State, in the form of the `useState` and `useReducer` hooks, is accessed on-demand, but it's still coming from React's source of truth: the `Fiber` object that represents React's tracking of this component instance.
I'll agree that it's not an _obvious_ argument to the component. But then again, neither is React Context, which is another source of input to components that isn't directly passed in as props either. That's my point - it's really all about React storing the data for a component internally, and then making that accessible to the component as needed while rendering.
The first caller gets the first state, the second caller gets the second state, etc.
If you have a normal chunk of static jsx, child "i" maps to slot "i". If you wish to omit a child dynamically, you put a null in its place.
If you have an array of children being built dynamically, every child needs an explicit key. This avoids state being shifted up and down needlessly when you add/remove.
You will never get another component's state back because changing the type forces a remount.
Unlike React, Live has the ability to preserve children even if the parent type changes (a morph), but this is opt-in only and will still discard the state of the parent.
In lists, identity can instead be tracked by a user-provided key: https://reactjs.org/docs/lists-and-keys.html
A simple example might be a tree structure. Think of folders and files on your computer. So first I go implemented a file system and I have folders that are a collection of files and folders. This data structure is hierarchical.
Now I want to make a React based UI where the user can create and delete folders and files.
5 folders deep the user want's to add a file. According to all the react docs I'm not allowed to just add a file to that folder as that would be mutating data. Instead I need to make a new folder, copy all the old entries to it, put the new file in. But, since it's 5 level deep subfolder, by the same rule, I need to do the same with its parent (make a new parent, copy all the old entries except the old folder, put in the new folder that's holding the new file). Oh but we're 5 levels deep so repeat all the way up the tree.
React people will often say "flatten your data" but that's the entire point, I shouldn't have to change my model to satisfy the UI library! I've already got working code without a UI. I just want to add a UI, not re-write all the other non-UI code.
I'm sure I just don't get it. Maybe a react expert will tell me how I use react without having to change my model.
setState(data.map { d-> d.tree.map { d2 -> ... } })
data[5][6][2][4].addChild(file)
setState(data)