Towards Principled Reactive UI
raphlinus.github.io
raphlinus.github.io
1. Differential dataflow. Like Adapton and Incremental (which Raph cites), I have found DD to be hugely helpful for understanding which computations can be efficiently incrementalized, and how (including in Rust).
2. Operational transforms. Often times, I am working on UI as part of a larger distributed/networked system. In these cases (especially when I already want the UI to display other people and their actions), the division of responsibility between the UI, the network code, and the various internal models has always felt like a missed opportunity for better and more principled integration / compositionality.
3. Build systems. The problem of figuring out how to bring a UI efficiently up to date in response to some parts of the underlying state changing can also be seen as a build system, or cache coherency problem. From that standpoint, where do the paradigms of the Build Systems a la Carte paper fit into Raph’s pictured theory of reactive UI?
https://github.com/TimelyDataflow/differential-dataflow
https://github.com/mstone/focus/blob/master/docs/ot-theory.a...
Certainly I've spent a lot of time thinking about collaborative and distributed systems, and there is no doubt scope for cross-fertilization of ideas. A huge inspiration (cited in the post) is Martin Kleppmann's automerge, which uses a proxy object to capture an explicit diff from a computation which is expressed using ordinary mutable logic.
Regarding the build system work, I will say this: the build task is inherently graph-like, and a lot of the interesting problems involve choices of sort order to utilize parallelism best; I'm fortunate to have a 16 core machine, and it makes me sad to wait on a single-threaded link step. I think by comparison the reactive UI task is mostly dealing with tree structure, and that's a simplification that could be useful to exploit. It's easier to describe mutation of a tree than general mutation of a graph.
But certainly there are common themes in all these incremental systems, and I continue to be drawn to figuring out the fundamental principles rather than trying to build systems ad-hoc.
Next, regarding your other thoughts, two notes, one on performance and one on “common themes”.
Re performance: while in my initial comment I linked directly to McSherry & friends’ work on differential dataflow mostly because I value it as a principled and effective framework for incrementalizing many kinds of computation I care about, I would be remiss if I failed to also highlight here their complementary “COST” project of using this framework to demonstrate how carefully engineered (single-threaded!) implementations of graph computations in Rust can often considerably outperform more complex but more wasteful pre-existing approaches (at least in the database world).
(Whether the same flavor of result will also appear over time in the build systems or UI worlds seems like an interesting question for future work!)
Finally, re “common themes in incremental systems”: yes! - this and your point about “fundamental principles” are exactly what I was hoping to suggest by linking to the build systems + OT materials!
Thanks!
Thank you for drawing attention to this. I spent a good chunk of last weekend pondering how accessibility might be added to IMGUI toolkits, and I do think it would be difficult. In particular, as you said:
> I believe a proper approach to this problem involves stable identity of widgets, about which much more below.
Agreed. In platform accessibility APIs such as UI Automation, each node has an identity ("runtime ID" in UIA) which is expected to be stable. One of the IMGUI toolkits I looked at was Nuklear, and I didn't come up with a way to derive stable identities for its widgets. On the other hand, Gio [1] (an immediate-mode toolkit for Go) looks more tractable, because the application holds a struct for each widget's state. Still, in an accessibility API like UIA, even simple static text nodes are supposed to have stable identity; I don't know how that would be solved with something like Gio.
[1]: https://gioui.org/
You can wrap tools like Dear ImGui and Nuklear into a reactive framework that handles state management and IDs and such and presents an API with a declarative interface a la React, but at that point you're pretty much building your own UI engine and just passing rendering on to those tools, which is not simple to architect.
First, most of the immediate mode UI's tend to be more "vector based" so the UI actually actually has the possibility of scaling properly unlike--well--basically everything else (giving me 1.0, 1.5 and 2x (effectively reducing my 4K monitor back to 1920x1080) does not count as "scaling"). That's actually a really important "accessibility" feature that STILL doesn't work properly for almost everything.
Second, for accessibility features like tabbing and screen reading, every user and every programmer bears that overhead (which the author points out may have significant architectural implications) for what is a user base of zero for the vast majority of programs.
Finally, who died and made "tabbing navigation" a pronouncement from God? Why isn't "spatial navigation" a better choice, for example? Why isn't there something better than that?
In addition, why is it the job of the GUI toolkit to do accessibility? Yes, I understand why "accessibility" wants to attach to the GUI toolkits as the programmers will do the math and never implement it otherwise. However, that doesn't mean the GUI toolkits should necessarily accept that task without at least thinking about the implications of doing so.
• All UIs are based around positioning and drawing things. There is no difference whatsoever between any paradigm in that regard: any UI can be scaled properly to whatever level you desire, it just needs to actually do it.
• Navigating through controls is far more common than you seem to imagine. On desktop platforms, a significant fraction of users (though still certainly a minority) will become extremely frustrated if your app doesn’t conform to platform norms; and it’s not just power users: in line-of-business apps, all the best interfaces use Tab for rapid navigation between fields. Mice are really slow as field navigation devices. On mobile platforms, this is more tightly integrated with the keyboard, so that in forms, the “Enter” key gets changed to “Next”. Again, you’ll frustrate a lot of people if you flout platform norms.
• Exposing an accessibility tree for things like screen readers, now that part is more commonly associated with overhead, especially on the web; I’ve seen one web app not do ARIA stuff by default, but have a checkbox in its settings to enable the accessibility tech, with the warning that it’ll make it slower. Can’t remember what app it was. The ideal might be to not build that until something requests it, but the web platform at least doesn’t provide the means of doing that at this time. I’m not familiar with the underlying protocols and whether native apps can work this way.
• Linear field navigation matches how people think most readily and how almost all apps work, and is a long-standing convention across all platforms, whether done with a Tab key or by other means. It’s what everyone is used to, which means you’d better have a really good reason if you decide to disregard it, and provide an obvious alternative. The most common alternative to linear field navigation is 2D spatial navigation with arrow keys, used in things like spreadsheets (augmenting Tab) or games (typically supplanting Tab). Gaming platforms tend to replace Tab navigation with this 2D navigation with arrow keys or a gamepad. But for most apps, 2D field navigation doesn’t tend to be as convenient as 1D.
• I’m baffled about who you think should provide for accessibility if not the GUI library. If you are saying “let the end developer do it”, I respond: are you serious? That would have the end developer duplicate basically everything from the GUI library, so people would abstract that into an accessibility library that wraps the GUI library clumsily, then hey, let’s merge the two so it’s not a pain, and now we’re back where we started. If that’s not what you’re saying, then I’m baffled, because those are the only two options I can see—unless you would have the screen reader do OCR on what’s on-screen and throw AI at it to guess how the app will work!
Native accessibility APIs do allow this. Chrome, for example, doesn't build accessibility trees unless it detects that an AT is actually consuming them. But you're right that the web platform itself doesn't allow this. The concern is that websites could then discriminate against people with disabilities, or offer misguided "alternative" versions. Some would probably also argue that if you're working with the grain of the web platform and not against it (i.e. semantic HTML and not too much JS), you get accessibility at no additional cost. I'm more pragmatic than that.
> unless you would have the screen reader do OCR on what’s on-screen and throw AI at it to guess how the app will work!
This may be the only way that the long tail of applications using custom toolkits will ever be accessible, especially considering the responses I get on threads like this one. And the VoiceOver screen reader in iOS 14 is actually doing this. It kind of sucks though that we have to burn battery power to reconstruct the UI semantics that are already there inside the application.
What about a PCB layout program? What about Blender and animation? What about video editing? What about a 3D modeler?
If you are using an immediate-mode UI, you probably aren't making a text-based CRUD system--especially as immediate-mode UI's tend to be remarkably bad at text rendering.
A UI toolkit doesn't magically make every application accessible. If the aforementioned applications are to be accessible, that application has to change in major ways.
Not everything is text.
Forcing everything through that narrow lens is an impediment.
Any fancy components will still need to be able to notify any accessibility tech what they are.
I contemplated saying more about other types of programs like those you mention, but decided that they’re really no different, beyond Tab being far less useful and often justifiably repurposed. The “canvas” parts may easily be just a black box that isn’t exposed to accessibility tech, but all of the rest of the UI around it (menus, configuration panels, property sheets, &c.) will still need to be exposed properly.
Try saying that to a blind person who lost their job because an essential application was inaccessible. It matters to them.
This strikes me as wrong and exaggerated.
This plays out in other areas. UC Berkeley was told that it needed to make public videos "accessible". That was going to be expensive, so they decided to simply remove the videos. This, of course, benefited no one because now nobody else could make the videos accessible either.
https://www.insidehighered.com/news/2017/03/06/u-california-...
Accessibility isn't free and I wish people would quit acting like it is. We, as a society, choose to impose that burden because, on the balance, everyone needs accessibility to some degree as they age or if they get injured.
However, if nobody is willing to pay for it, don't be surprised if some people decide, like Berkeley, that exiting the arena is the better choice.
At which point, everybody loses.
I would go as far as putting immediate mode UIs and reactive UIs into the same "mental model bucket". Both only describe the desired UI state (e.g. no separate "UI creation" and "UI updating" phases), and let an intermediate layer figure out the minimal required state changes to "realize" this desired UI state (whether this is exactly how the UI system works under the surface is mostly irrelevant to the API user).
The only (visible) difference between reactive and immediate-mode UIs seems to be that reactive UIs seem to prefer nested data structures, while immediate mode UIs prefer nested code blocks to describe the desired UI state.
Thanks for that info. I hadn't looked closely at Dear ImGui yet.
[1]: https://salsa-rs.github.io/salsa/ [2]: https://rust-analyzer.github.io/
This is particularly of interest to me because I work day in and day out with what is to my knowledge the only application of the incremental computation approach to browser UI,[3] and colleagues and I have talked about what it would take to reimplement in Rust—what the tradeoffs would be, where the problems would emerge, etc. I suspect part of the issue is that the constraints of using the DOM informs a lot of the design considerations in the space, whereas implementing from scratch gives very different tradeoffs?
[3]: https://v5.chriskrycho.com/journal/autotracking-elegant-dx-v...
Regarding DOM, yes. I think it's held up pretty well considering how long the ideas have been around. It's slow, it's clunky, but it gets the job done. But I also think now is a good time to consider better ways to implement its core functionality, which is tree mutation.
I'm aware this is significantly because they "cheat", but I feel like you're already looking at borrowing from other peoples' trickery so maybe it'll still be of interest.
Is this really true? My intuition is that reactive UIs are fantastic for widgets and "wraps a database" type apps, but fall flat when it comes to deeply interactive software where performance and responsiveness is paramount — i.e. where there's a tight feedback loop between user input and rendering, and where the model state tends to be quite large and granular. (DAWs, drawing/illustration software, movie editors, mapping applications, text editors, etc.) I also get the sense that most revolutionary or industry-changing software falls into this category. So how can the reactive approach be anything other than a small tool in the app developer's toolbox?
Could somebody dissuade me from this notion? In my admittedly limited experience with the reactive paradigm, I feel like I run into a wall as soon as I've implemented all the basic widgets and need to actually work on the main content view of my app. It doesn't feel like it could ever be a good fit for software that pushes boundaries.
(Perhaps this is an entirely separate matter from "object-oriented UI" vs. "reactive UI", though. Or maybe I'm "using it wrong", in the sense that reactive UIs are meant to be mostly about widgets and not complex interactive views. But then I often hear people talk about reactive UI as the future of app development, so I'm not sure.)
DOM is not slow, in principle.
What makes updates slow is the fact that each mutating DOM function must left the DOM in consistent state - rendering trees invalidated, etc. This creates significant overhead on massive updates. But:
element.innerHTML = "new content";
I have tests of different methods of DOM updates in Sciter (https://sciter.com) see: https://github.com/c-smile/sciter-sdk/blob/master/samples/te...
The test shows that these two methods:
element.html = "new content"; and
element.content( vector-of-vDOM-nodes );
What you've done is a pretty small change to the DOM and generally compatible with it. What I've done in Crochet is a more radical rework, but certainly with some of the same general principles (my `Mutation` datatype is also a transaction). I should also point out that my tree mutation is a bit more limited, as it can't do general movement of subtrees, only sibling movement within the same parent (and, in the current prototype, no swapping, only order-preserving, which includes insert, delete, and update).
If some compatibility with browser DOM is a goal, then your approach is appealing. I'm not pursuing it in my prototype because I'm also holding overall system simplicity as a whole. But in any case, I agree that people should be aware of what Sciter does; it's an impressive system.
document.mutate(function(mutator) {
mutator.changeAttributes(el,{...});
mutator.appendNodes(el,...);
...
});
I don't understand the main issue brought up with observables. What exactly is that "context" you're talking about, and how is it bad?
Observables, at least the way used in my https://github.com/raquo/Laminar/ UI library, define the dataflow graph of your application, letting you channel data into other data and/or into incremental DOM updates without using virtual DOM.
You don't need any additional "context" to know what change happens in response to what, defining the flow of data with observables and then binding some of them to DOM parts that you want them to affect or to source data from is literally how you write your application logic. You will need to express that application logic somehow, and doing that using observables is quite straightforward.
I'm not familiar with SwiftUI, but Svelte is not really any more about observables than React is. Disregarding how it's implemented under the hood, Svelte's API is made to be superficially similar to React but without virtual DOM, and that comes at a very high cost of designing their own language and requiring their own compiler. So you end up with a callback-driven API that to the user feels a lot like React's state + props + render method, and it's nothing like working with plain observables.
I can remember a long time ago reading about an Observer pattern for a UI. Views would subscribe to a model. When the model changed, the views were notified. The Views then queried the model and determined how to update. It was suggested that for more efficient updates the change notification could carry some information(context) to indicate what had changed. The article didn’t discuss this information optimization further.
But what does the author think of using observables then I wonder? I mean the streaming kind, like ReactiveX (but not ReactiveX specifically). They're first class representations of event streams and state – exactly the domain of a UI library – and working with observables has been very pleasant in my experience.
I don't have very strong feelings either way about stream-based computation. I am skeptical that using them as a primary primitive for building UI is going to work out well. Things are nice in the static case, but it seems to break down a bit when there's dynamic reconfiguration. Evan Czaplicki has given good talks on his evolution away from purist FRP. It's also interesting to compare the original Elm thesis[1] with the farewell[2].
That said, I think it's a super-interesting experiment to try to integrate these ideas with the Crochet architecture and see how it turns out.
[1]: https://people.seas.harvard.edu/~chong/pubs/pldi13-elm.pdf
Look at this pinnacle of elm architecture for example: https://elm-lang.org/examples/time
All this boilerplate just to display current time in current timezone. The same can be done with three very obvious lines of code in Laminar for example, without any regard for abstract purities, but with the same concrete outcome – a well bahaved h1 element that displays time. Scales to larger components and applications very well too.
Cycle.js has the same problem as old signal-based Elm by the way, and for the same reasons: obsession with purity, and ignoring that observables are a poor match for virtual DOM. You can have one without the other, and have a much simpler system. If you already use observables, virtual DOM adds nothing of value, only a way to satisfy the irrational demand for purity.
In many such languages support "setter" and use such to naively implement observer patterns, you can see a lot of repeated updates for the same object through the call chains. Because often than not, you are not update one property for one state object, you need to update a set of properties across some number of state objects, and simplistic "setter" based observation cannot differentiate these and coalescing properly. People can workaround this and implement "backpressure" etc, but it just to add more complexity on a broken promises.
What you really need is an observer implementation with transactional guarantees. Thus, the state change cannot be observed until an update transaction is done. This will naturally reduce number of unnecessary updates, and doesn't really need "setter support" from the language.
You should join the xi channel on zulip https://xi.zulipchat.com/, there's a dedicated Crotchet subchannel.
How do things change if we were to use a list instead? What becomes easier/harder?
Layout calculations might become slightly more expensive, though it seems that that may be acceptable since most time is spent rendering; and perhaps there are other ways to accelerate these calculations if needed.
The fact that most UIs are happy to provide a single linear tab order (and not something more involved that allows users to navigate the tree structure) makes me think that there might be something there.
Is a text box with a reply button a widget? What about just a text box? What about just a button?
Assuming all three, it sounds like you have one widget composed of two other widgets, which naturally ends up creating tree structures.
It is, in some ways, but this is not the best model to reason about code, IMHO.
Sound theoretical underpinnings are usually necessary for a coherent system that allows for good abstractions.
Note that in code while I don't say the word "tree" while writing it. I create a tree with my braces (C) or indentation levels (python) and file structure. The language designers created a syntax using a tree. And so on.
I guess what I'm saying is that while the tutorial should not say tree, and I should not be thinking about tree nodes and edges while writing it, I should be writing a tree (and in code I am).
I am familiar with the structure, but I've never found it practical as an abstract tool.
Any hierarchy (group boxes holding widgets) is purely visual, and the rectangle doesn't "own" its child widgets in a programmatic sense.
Text widgets have a fixed size. If your font gets wider (from DPI scaling or translations), text can become cut off or overflow onto the next line and disappear.
You can say that all nodes on this leaf of the tree are contained within that node... It's an elegant way to describe a UI in my opinion.
Eg: You want to move a button and you need to move its text and background shape. How do you keep track that these two elements are grouped together?
"The idea of the app logic producing an explicit tree mutation while holding an immutable reference to the old state of the tree feels very functional, though the app state itself is mutable. Even so, I haven’t seen this particular pattern in the functional programming UI literature (hopefully readers can fill me in). "
IIUC, this sounds a lot like what MWeststrate does with Mobx-State-Tree^1
For example, comparing a virtual DOM model with manual UI changes: Deliberate UI changes, if done well (The article mentions they're often not) are faster... at the expense of losing the elegant model of declarative UI. VDOMs conduct many more operations than are required, but allow you to describe how the UI should behave elegantly.
As the article mentions, Svelte is a cool approach that attempts to reconcile this. Personally, I wrote a React/Elm-like WASM framework in Rust. After stepping back, I'd only use something like that for complex, non-performance-critical UIs. In embedded or simple websites, that sort of overhead isn't appropriate, even though it leads to nice code.
I think we'll be able to dodge this compromise soon. Maybe with something like Svelte, or maybe with a different approach.
- functional core is responsible for computing the new desired state in response to events
- driver/plugin is responsible for computing the diff between the current and the desired state
- imperative shell is responsible for applying the diff to the process... which may be partially or wholly implemented as another component
We've all seen "smarter" approaches failing more than a few times, obviously on the simplicity side but also, sadly, on the performance side, even with all the caching and retained structures.
This is the strange beauty of brute force algorithms, they may look dumb and ugly for our refined human minds, but they tend to map very well to the capacities of computers.
It is often faster to not bother and blindly render something instead of running complex logic to decide if you really need to render or not.
And if you're not satisfied with the performances of a "dumb" imgui, you can always add some smarter caching without touching the API.
Altough I bet this approach would only get you marginal benefits.
I am not sure I understand this question.
I rest my case.
You have to set a compilation flag to enable the keyboard/gamepad navigation, if you missed it from the source code.
Even with integration into system accessibility APIs, Narrator (and other screen readers) performs a lot of heuristics to improve the user experience. These heuristics depend on a separate tree data structure [2] and how it changes over time, so in order to implement even the most basic accessibility, IMGUI would have to retain that information instead of throwing it away every frame. Component identity is crucial to how most (all?) screen readers work today and it's a happy accident that it also allows an efficient hybrid retained-immediate mode architecture.
I don't know if it has been fixed or not, but a year or two ago you could completely nuke the Windows screen reader just by assigning React components randomly generated keys so that the DOM nodes would be destroyed and recreated every redraw.
[1] https://dequeuniversity.com/screenreaders/narrator-keyboard-...
[2] https://docs.microsoft.com/en-us/windows/win32/winauto/inspe...
If the ability to add metadata to any widget is not sufficient then I don't know.
What about the “zoom under my cursor” type things? That seems much harder (though not impossible) if your interface to clients is “just draw directly”. Though thinking about it, it’s actually probably what macOS does: as a post process, take your frame buffer and then magnify the stuff in a region of interest and re-render that on top.
You’re winning me over :).
Edit: I assume your metadata solution is “if someone asks, the text under this region of interest says < string >”.
A structure is built every frame with all the data to render later, as a batch.
And of course if you need a magnify lense the obvious way would be to render it as a post process.
It felt almost the old days where your UI is derived strictly from business logic with fixed set of widgets is better for accessibility than our new crop of apps where you have more gestures and shortcuts. It is no wonder that imgui shines if you need more gestures and eye-candy animations.
IMO, the bigger problem is the one that Raph pointed out: widgets (including simple ones like static text) need to have stable identities throughout their lifetimes, even as their state, contents, and locations change. It seems to me that IMGUIs, or at least some of them, make this particularly difficult.
As for the stable identities, yes, it is important. But I also don't see much of a problem with explicit Key: the UI components ultimately will have map back into a data object, which should have enough contextual information to provide the said Key.
I am running a prototype immediate mode gui that I built for my yet unreleased engine and the frame time is staying below 500ns for a fullscreen app with many widgets and layers.
The whole thing is built on C + Vulkan but is not entirely optimized yet.
And this is running on a decent but not extremely fast computer. (An Intel NUC from 2018)
There are many ways to improve performances of idle/almost idle ui.
Caching can be useful for large blocks of text as it tend to be the slowest thing to render.
But it is preferable to incrementally optimize/add complexity and check assumptions and not make large architectural decisions ahead of time.
One way to reason about it is to think in terms of pure data transformations.
If you have a static UI, you can skip rendering frames until "something" happens. That's going to burn no more power than a retained-mode UI.
Sure, the immediate mode UI may redraw 100,000 triangles vs the retained-mode 100 triangles. However, that's just not a bottleneck on practically every device nowadays. And I doubt that the difference is that stark as rendering scaled text properly is still just egregiously expensive in terms of computation.
However, everybody nowadays has to "ANIMATE! <jazzhands>" so you're rendering 60Hz anyway irrespective of type of UI.
And if animation is required by application (film editor, digital audio workstation, etc.) it's not clear that you really can do anything other than immediate mode. Do note how many of those kinds of programs wrote their own GUI over the years--that says something.
One thing that people don't mention is that immediate-mode UI's tend not to suffer from as many synchronization issues and retained glitches as retained-mode UI's do. Even if you don't get the update correct in an immediate-mode UI, the next frame the error goes away. In a "retained-mode" UI your widget needs to "refresh", which may not happen except in very rare circumstances. I have had to grab the corner and adjust a window size a couple of pixels innumerable times in order to clear UI glitches over the years.
I should probably comment about threading and the "UI Master Thread", but I don't think I've seen a good example of an immediate-mode UI running multi-threaded, yet. You probably need Vulkan to even have a hope of pulling that off (OpenGL is notoriously multi-thread hostile).
I’m a big fan of immediate mode myself, because it’s kind of amazing how much you can actually push through via brute force.
However, wouldn’t you agree that Unity, Open Scene Graph, etc. are successful counter arguments?
I would say it depends on your audience. The bottom layer of the stack might be a very simple retained mode, or fully brute force immediate, but it’s not clear to me that a “UI framework” for application developers should be immediate mode.
An UI, from an abstract point of view, is nothing but a big structure.
When you're writing an application or a game, you very often already have an existing structure containing everything you need.
Take a simple example : a leaderboard.
Using anything but an immediate-mode API will force you to create a second, complex structure to hold this leaderboard, and a lot of glue code to copy data to and from this structure.
I think that argument is “people can’t usefully agree on common abstractions, why bother forcing a translation”. And so for users of scene graphs, it’s basically “well, we can all agree there’s a basic graph and bounding boxes are a thing” so you’re not giving up much.
But for UI, is the argument that it’s often more like drawing/painting, so you spend more cycles (in both mental and processing senses) on trying to express the thing you already knew you wanted to achieve?
I've seen that made many times.
Its design is fundamentally much simpler than that of the immediate mode UI libraries. There is no need for implicit state. There is no need for layout to be a core feature of the library; layout can be 'just another widget'. This opens up the possibility for different types of layouts: constraint-based, grid-based, etc.
It's even possible to implement an imgui on top of this fairly easily; but not the other way around, because imgui has implicit global state to manage.
Imgui is not slow, especially with GPUs. But it definitely loses on the simplicity front.
These frameworks, contrivances, and the resulting principles will spiral on, never solving the real issue: DOCUMENT Object Models and HYPER TEXT MARKUP Language are for representing linked documents, not stateful object based applications. It's right there, in the names.
In the article he mentions how a lot of the web frameworks don't have to worry about handling tabindex because the browser does it for them.
I have also yet to find another non-web UI framework that handles styling nearly as well as CSS. Between flexbox, grid, and absolute positioning, you have so much power for creating layouts. Throw in some drop shadows and borders to help highlight the important UI pieces and bam.
I think the common critique of React, Vue, etc on HN is fair when talking about people building simple websites with these highly sophisticated and massively overkill tools. But when it comes to applications I have yet to find something nearly as flexible and good looking with so little work. And they do it by using HTML and CSS which as it turns out are pretty good abstractions to build on. Or at least they are from my perspective as someone who's been using a variety of these frameworks for the past 5 years or so, on quite a few medium sized projects.
* encapsulation - what includes what / belongs to what
* visual - what overlays/obscures what
* event-passing - if X gets clicked, or otherwise interacted with, who else would get the event passed to/notified
* ??? can't remember this one.. but i remember it took many conversations to find out we need it. Probably was dependency-linkage (in aspects other than above 3, e.g. some model-logic dependencies)
This aspect-distinguishing may or may not be usable for you. For example, a usual "scrollbar" is composed of multiple smaller elements, which are tied together in all above aspects. But it does not have to be like that, e.g. the position-mover ("thumb") and the lower/higher buttons need not be on same visual place, e.g. a circle volume knob and separate quieter + louder knobs elsewhere (and that may be very far). One may argue it's not a scrollbar anymore.. expect it does the same thing, just visually differently.
It would be interesting if one can really separate these aspects in the code, somehow.
That said, how would your toolkit handle this: a small "innocent" interaction that causes lots of small changes all over the tree. Think taxforms: a form, mixture of subforms and fields, many levels deep, and i need to highlight the user-changed fields (or subforms containing changed fields), all the way up. What i ended up recently, in React (i.e. in DOM-recipe-tree), is, anytime the user presses some keystroke in some text field down in the depths, the whole thing needs be repainted. A few hundred fields probably. Which.. isn't that fast anymore, when depth goes beyond 3-4. (From user perspective there are 2-3 times less levels than programatical). Probably i've missed some "i have not changed, don't repaint me" step somewhere.
But I am glad to see native developers finally get a half-baked, sparsely documented reactive UI framework (still a year later!), and like a decade late. Apple is over, and it’s hilarious!
Of course, you can still write your low level bitblt functions in Rust if you need to (though these functions are probably best left to a GPU). But an entire application and its UI, then you're probably pushing it too far.
I also think that the success of the combination of Qt/C++ is mostly a result of the evolution of programming environments. C++ was massively popular, so that could explain it.
This blogpost doesn’t show any UIs built with this rust paradigm, and the github homepage only displays a few screenshots of a calculator app.
Technically, it appears interesting. Practically, it seems very far off from the tools I use to build usable (or hopefully aesthetically pleasing) UIs.