Towards a unified theory of reactive UI
raphlinus.github.io
raphlinus.github.io
According to the second half of the post (the one starting at “Case studies: Ok, since the above was so abstract…”), it is relevant to Imgui, Flutter, Jetpack Compose, React, “DOM”, etc, but even someone who uses or knows a lot about any one of those, unless they got to that part, may not become aware that this post is relevant to them or contribute to the conversation.
(This comment has the same problem.)
This is a space I'm very interested in. Having a high-quality, reactive UI toolkit for developing native desktop apps might make it easier for web developers accustomed to these reactive frameworks (e.g.: React) to develop native apps (without having to resort to hacks like Electron).
Linux in particular could really use more developers developing native desktop apps and I feel the current UI toolkits (QT/QML, GTK) and in particular their language choice (C++, but I'm aware there are bindings for other languages) are not appealing enough to the casual web developer. It could be argued that Rust might not be the answer for these developers anyway, but one can still hope.
Both Qt and Gtk have Python bindings, which would be far more in the realms of a web developer (dynamic typing, no compilation, garbage collection...).
Why is Rust a great option for building native applications? Rust is a great option for building applications that would traditionally be in C/C++. These languages are traditionally chosen to have lower overhead than other options. You could build a new Qt or GTK in Rust, and put similar Python bindings over it, just as you described to go after those developers, if that’s desirable.
In addition to that, many casual web developers are recognizing the benefits of a compiler with a type checked language, it’s why Typescript is growing so quickly.
So you want to choose Rust when you want a lighter weight application, that has lower memory overhead, generally faster runtimes, and a long term plan around maintenance (the type system and various safety guarantees). Or more succinctly, because you’re someone who enjoys writing programs in Rust.
What is missing is big-ticket suites like Adobe's software or video editors etc. In other words, huge software packages that I would very much like to prefer to be developed by proper skilled developers who can manage to write modern C++ (is it so hard? We're not talk assembly here...), not by "casual web developers" who are used to writing subpar code because the web and the browsers let them get away with it.
what a strange thing to say
I fail to see how a systems programming language notorious for it's novel/different take on memory management will cater to a community known by their extensive use of high-level languages that are very permissive and abstract away memory management.
Well, unless we're talking about buzzword-oriented programming.
Basically, widgets (in this context) are the immutable parts (e.g. "config"), "elements" would keep the updates, and relationships (parent/child), and renderObjects - layout, drawing, etc.
I think the model you are describing, has parallels with Yaron Minksy's Data Driven UIs, incrementally
https://www.youtube.com/watch?v=R3xX37RGJKE
Perhaps if you could review the above (if you have not done already), additional ideas/data points might help with your formalization effort.
For example there is a paper by Martin Odersky himself on incremental computations on lists (which should be very familiar to you due to your work on Xi's ropes): https://link.springer.com/chapter/10.1007/978-3-642-39038-8_...
http://archagon.net/blog/2018/03/24/data-laced-with-history/
I hope all new UI systems aim for SwiftUI's syntax. Not to copy Apple, but because you can't really get more succinct that this:
ZStack() {
Rectangle()
.foregroundColor(.unicornPuke)
VStack() {
Text("Hello")
Text("World")
}
}
It would just be a model for describing the UI, that a hypothetical engine backend would take to spit out OS-native widgets. For another OS, you'd plug in a different engine.
Having a universally agreed-upon syntax would provide the benefit of "learn once, write anywhere", greatly reducing the friction in building cross platform apps with native APIs.
[0] https://news.ycombinator.com/item?id=21495338 - Rust 2020: GUI and Community
Not in a million years would I have guessed JS devs would praise XML for readability.
And yeah, I totally hate XML, would still never use it for data or config, but for some reason JSX just kinda feels good.
I also do not see the relevance of this argument for for the reactive UI topic.
"[...] duplication between the initial construction of the UI and updates. [...] to add a slider showing the value, the onclick handler also needs to be amended to add an update for that".
[1] https://en.wikipedia.org/wiki/Persistent_data_structure#Fat_...
UI necessarily includes GPU because that's on the path to the hardware screen.
I hear what you are saying, but I think the situation is a lot more complicated than that. Certainly, these days, if you're not using the GPU then you have failed; with high dpi and high frame rate displays, combined with the end of Moore's Law, CPU rendering is no longer viable. So the question is how you use the GPU. I agree with you that power is the primary metric of how we should measure this (it certainly correlates strongly to memory bandwidth, and is the metric users will actually care about; if there were a magical way to achieve high memory bandwidth at low power, I don't see any problem with that).
One way certainly is to use the compositor, and this is very seductive. The major platforms all expose an interface to the compositor, and it's generally pretty well optimized for both performance and power usage. Since animation is part of the API, it's possible to do quite a bit (including, for example, cursor blink in an editor) without even waking up the app process. Scrolling is another classic example where a compositor can do very well.
However, the compositor comes with downsides. For one, it forces the UI into patterns that fit into the compositor's data model. This is one reason the aesthetic of mobile UI design is so focused on sliding alpha-blended panes of mostly static content.
But even from a pure performance standpoint, heavy reliance on the compositor has risks. It's tempting to think of the composition stage as "free" because it has to blit the intermediate surfaces to the final composited desktop, but this is not strictly true. On mobile devices, and in the process of coming to desktops (there's some implementation in Windows 10 for Kaby Lake and higher integrated graphics) is the use of hardware overlays to replace actually blitting the active window to an in-memory surface. When the overlay is available, it's both a lower latency path and also saves the GPU memory bandwidth of that blit. And generally the heuristic is that the application window is a single surface, in other words that it does not rely on the compositor.
In order to justify doing the final scene assembly under GPU control in the app, that has to be as efficient or more so than the compositor. I have some evidence that, for 2D scenes typical of UI, this is possible. Regions that are flat color, for example (which occupy a nontrivial fraction of total area) can be rendered with no memory traffic at all, save the final output stage. And most of the other elements can be computed cheaply in a compute kernel on the GPU.
The tradeoffs are complicated, and depend a lot on the details of the application and the platform it's running on. In the 2010s, a strong case can be made that a compositor-centric approach is ideal. But in the 2020s, I think, it's increasingly not. The compositor is evil because it saps latency, and because its seductive promise of power and performance gains holds us back from the future where we can use the GPU more directly to achieve the application's rendering goals.
GPU rendering and compositing amount to the exact same thing most of the time. A "region of flat color", at least in principle, is just a 1x1 pixel texture that's "mapped" onto some sort of GPU-implemented surface that in turn is rendered onto the screen.
Hardware overlays merely accelerate the final rendering step; one can implement the exact same process either in hardware, or as a software-based "blitting" step.
Of course the compositor is using the GPU. The difference is entirely in how the capabilities of the GPU are exposed (or not) to the application. My thesis is that doing 2D rendering in a compute kernel is ideal for 2D workloads, because it lets the application express its scene graph in the most natural way, then computes it efficiently (in particular, avoiding global GPU memory traffic for intermediate textures) using the GPU's compute resources.
Of course you could in theory have a compositor API that lets you do region-tracking for flat color areas, and this would save some power, but no system I know works that way.