So you want to write a GUI framework (2021)
cmyr.net
cmyr.net
- Rendering and dealing with text in all of its extreme permutation. Styling. Localization. Character sets. Etc.
- Modern accessibility standards. There are countless annoying yet de facto UX patterns that are just nightmarish to check off the list from scratch. Inputs/forms/etc. Even worse than accessibility, because you can decide to forgo that completely if you really wanted. The sheer volume of UX use cases and edge cases involved in even seemingly basic input components is also an exercise in rote dread. You've got input validation, error states, just so many ergonomics to deal with.
- Fluid layouts and scaffolding. This is where modern HTML/CSS finally starts to shine because making flexible layouts is actually quite painless. But it's a motherfucker to roll your own approach.
EDIT: I see that there are people down voting several comments in this thread in support of accessibility. I wonder what disagreement that you have? Surely it can't be "not a large enough market". There are more legally blind people in the world than Firefox users, yet web developers will happily test compatibility with that dying browser and eschew screen reader testing.
We asked our disability services office to review the webmail's accessibility. The manager of that office was a blind woman, who was both a staff member and a student, so she had her own personal student account. She told us she couldn't use Sun's webmail, it wouldn't work at all with her screen reader. We complained to Sun, their response was "we know, wait for the next major version".
What did we do? We offered her a shell account on a Solaris box with Pine, and told her that we'd offer the same to any vision-impaired students or staff members who requested it. And she was happy with that solution.
IANAL a lawyer, but from my understanding of disability discrimination law (which is different in different countries anyway), you are legally obliged to offer "reasonable accomodations" – which doesn't have to necessarily be by making the application accessible, it could be offering access to an alternative application instead.
Also, you are only obliged to provide accessibility if a disabled user asks for it, which assumes you have a user with a relevant disability – if your application is sufficiently niche, that might never happen. Suppose you build an internal GUI for SREs to use. Do you need to make that accessible for the vision impaired? Well, are any of your SREs vision impaired? If the answer is "no", then legally you don't. Maybe one day you'll employ a vision impaired SRE – at which point, then you will be obliged to make it work for them. But, maybe you never will, in which case you'll never have that legal obligation.
I don't completely disagree with you, I think accessibility is often a spectrum, and it's good to be thinking in terms of what your specific accessibility needs are, rather than just going for a scattergun approach.
That said, I have some problems with this example.
Firstly, a lot of features end up in the "accessibility" bin that are more broadly useful (the "cut curb effect"). Your SREs might be fully sighted and mobile, but they will probably still appreciate being able to drive the software fully by keyboard. And even from a development perspective, I've found having correct aria roles etc can often be useful for end-to-end testing because it allows you to be more semantic with the tests ("click on the thing with a button role with the text 'submit'" rather than "click on the element with the ID '...'").
Secondly, we often see accessibility in terms of the extreme cases ("you are blind, you have unlocked the accessibility treatment now"), but accessibility is more broad than that. If you're working late at night in a dark room, or trying to do something on a small laptop screen, or you were out climbing and now your wrists ache, you might still appreciate accessibility features - zooming, keyboard navigation, accessible themes etc. Getting this stuff right helps everyone.
Finally, retrofitting accessibility is often a nightmare to actually implement, because (a) it's hard to notice what's not working unless you go through everything with a fine-toothed comb, and (b) if you don't have at least some accessibility plans to start with, you might have dug yourself into a corner that you can't get out of. For example, if your gui framework has no accessibility support, and someone blind starts in a month's time, what are you going to do? Rewrite the whole application for them?
We talk about accessibility in terms of extremes, because those extremes are where we have legal support that companies can't ignore. The vendors don't want to cut the curb, they want to replace it with a fence and a toll gate. Talking about wheelchairs and strollers lets us prevent the fences, and the toll gates. If they figure they'll skip the curb too, every user wins, but this won't happen without prioritizing wheelchairs, and won't happen if we emphasize regular people too much.
Or, in short, accessibility is the only weapon we have to fight dark patterns and maybe get some ergonomics for everyone.
Didn't downvote but I want to share a couple thoughts I've had about accessibility:
- I assume that browser/electron accessibility is not really optimal for users. It's sort-of free if you're building web/browser stuff, which is nice because it's available by default, but I can imagine most applications could be tailored for accessibility and be much more productive and enjoyable for end-users. For instance a "blind mode" for a text editor would probably be a lot different than what you get out of the box from a screen reader + VS Code. (I would love to get feedback on this from someone using a screen reader or accessibility features.)
- I worry you're providing a bit of a moat if you regulate/require accessible software, since for most applications that equates to using a native GUI toolkit or browser-based solution.
You said
>Anyone who would make software with your framework would be in violation of several laws.
Which just isn't true as there are ways you can comply with EQA without every underpinning library being accessibility-compliant.
Say I am a British person who decides to make a some common piece of software, like a spreadsheet, and offer it for download on the internet. Am I required by law to make it accessible? And if, for instance, the provision of accessibility features would require thousands of hours of redevelopment, does that fall under "reasonable adjustments"?
I understand that the moment my software becomes required for someone's job, for instance, there might be a problem, but a few posts up it was implied that making software without accessibility features was in itself illegal in some way, and that just sounds doesn't sound right to me.
Section 20: https://www.legislation.gov.uk/ukpga/2010/15/section/20
That does not call out in its table any reference to software. I can't see anything in it which would effect a binding to software products. Section 22 refers to regulations being able to define what are "auxiliary aids", so in theory an SI may have done so.
Scanning the whole Act, the closest thing I can spot is Schedule 25: https://www.legislation.gov.uk/ukpga/2010/15/schedule/25, however even that defines it as “any service normally provided for remuneration, [...]". Note that is for an online Service, not a Good (which software would generally be). I've also not found what references it.
So that may cover a web site, but only if it was a paid for thing (e.g. online newspaper sites accessed via subscription). I wonder if and how it applies to the likes of Amazon, where the site itself is not paid for, but goods available through it are.
So unless someone can find specific references, I see no reason to believe there is such a statutory requirement in existence for software, certainly not for free software.
(IANAL)
Approach it like that and you have a slight fighting chance because you know what you are getting into.
Funny that you mention Unity because they first had an immediate mode UI that was very difficult to work with, then they got a more flexible UI but decided to go the HTML way of having separate documents for elements and styles, making the whole design process way more complex.
The new input system is powerful and flexible but is slower to learn and makes lots of things that used to be easy much harder.
Same for the new UI Toolkit and several other recent additions. I think it's a company culture thing - nobody internal is rooting for casual devs or noobs.
The problem is - everyone is a noob on something. You might be an incredibly experienced Unity dev who rarely touches input or UI. You want to quickly implement something and you're going to feel the pain.
"Make the easy things easy, and the hard things possible."
People like Apple, Microsoft and browsers put lots of effort into this if you use their existing frameworks right.
Good accessibility capabilities requires flexibility to be at the core of the UI toolkit. You basically can't make assumptions anywhere.
The compositor. Dealing with ancient Win32 API, with his semi intelligent windows where seemingly every window feature is in weird undocumented ways coupled. Also, the race condition when rendering, just resize any window and see it jiggle. We seem to have collectively given up on fixing it.
Text Input. While not terrible difficult, so many features. From drag and drop, copy / paste, page up / down, CTRL up down left right, Home / End, wrapping. Text is by far the most complex GUI.
To do really nice form layouts in a desktop application, alignment and sizing of labels across the entire form becomes really important to me. Products like Developer Express that have a dedicated form layout work so well for this. And some of the swing layout managers I've used (I can't recall the name, too long ago) make complicated form layouts much more sane than grid, flex and other css layouts.
That might change with .NET MAUI, though. I'm not too deep into it yet, and it is significantly different enough from HTML+CSS to be a challenge to learn, but what I havelearned so far keeps making me question "why did the CSS version of this have to be so hard?"
I even implemented a fair subset of CSS grid:
parseGridTemplateRows gt, ["row1-start"] 25'pp \
["row1-end"] 100'ux \
["third-line"] auto ["last-line"]
However, text input is hard and tedious! Then you ideally would need to handle the different keybindings for each OS. You also loose any plugins the OS'es provide. Not to mention the lack of accessibility.
I recall reading that browsers shim out to native OS text fields and wonder how that's done. It really seems like the best approach for small GUI libraries to enabled first class text input.
Ideally the underlying system would offer a low level text rendering API that's independent from the system's UI framework and can be combined with your own rendering efficiently (haven't used it, so don't know how good it actually is, but for instance on Windows there's DirectWrite). The problem with this approach is that not all platforms provide such a modular text renderer (for instance web browsers), and you need different code paths on different platforms (maybe even still have to integrate FreeType because one of the target platforms doesn't expose a text rendering API).
Short of doing a whole CSS rule engine (best) you can also implement a Flutter-like layout system. It is described well by Flutter documentation and is quite flexible.
What he essentially did was he cracked open “Inside Mac” and ported it to a C library that ran on the PC.
He did the bulk of it too. Memory manager, Resource manager, QuickDraw, Window manager, Event manager, Font manager.
They changed it up, they didn’t copy the window gadgets and such, scrollbars worked (and looked) different.
But the task was to port a native Macintosh application to the PC, pre-Windows, and keep it portable. It was not a one way trip. The company maintained both versions.
Got the bulk of it done in less than a year. Obviously this was early Macintosh. B&W, bitmap fonts (though the application generated PostScript for printing).
Yup, they don’t call them The Old Days for nothing.
You could write a GUI, a game, a text processor, a compiler and even an OS as a one person team.
Compiler design is no harder today than 20 years ago.
Any text processor you would have created single-handedly 20 years ago is similarly doable.
Simple, performant GUIs are easy if you restrict yourself to a single platform. Windows 11 still runs win32 apps, Ubuntu/GTK is also straightforward.
For an OS, consider targeting a well-documented microcontroller. Some variants are approaching a similar amount of compute and RAM as 25-year-old PC hardware but have Ethernet ports and/or LCD displays.
And if you just want to build a game to say you did, you could build just about anything in the browser using vector graphics primitives reminiscent of DOS.
Depends on how much one takes interactive compilers on environments like Energize C++, Visual Age for C++, Strongtalk, Dylan,Oberon,Cedar... into consideration.
While similar to modern compiler based IDEs, their language servers, compiling while typing, static analysis while typing,...,weren't as feature rich.
You could write a GUI, a game, a text processor, a compiler and even an OS as a one person team. Now you can't and some of the most complicate aspects of software engineering is how to interface your work with the work of others, how to understand what others have done, to have a high level global view of the whole software application and tools your company works on while having a very low level view of the part of the software you are working on.
Yup.
And I had read earlier that Niklaus Wirth created a computer and many of the layers above, with just a small team. (I guess there must be other such examples from the early days of computing.) Don't have the reference. May be related to his work on Lilith, Ceres, Oberon, etc.
https://en.m.wikipedia.org/wiki/Niklaus_Wirth
Also see Wirth's Law:
https://en.m.wikipedia.org/wiki/Wirth%27s_law
The derived / related laws mentioned there are also interesting.
I am old enough to have used Native Oberon back when it was still a novelty, and compare Modula-2 to Object Pascal compilers.
Nowadays they feel out of place, both in OS capabilities, as well as the languages used to develop them.
And where exactly is the difference, when you write a OS, where you have to interface with the low level hardware and the choices of those engeneers?
Apart from that, what is stopping you? Just pick a simple plattform and start. But yes, the result will be probably hard to sell, unless you find the right niche.
Alternativly, you can join SerenetyOS, which started exactly as just a one man project and came quite far alone. And now they are getting momentum (100000$ donation to build a browser).
You still can and many still do, but we expect a lot more from our tools in 2023 than we did in 1983.
In JavaScript land people are scared shitless to write original software or learn the basic foundations of their common platforms. So yeah, that would make 10000000x more challenging. But that is a psychology and preparation problem, not a technology problem.
No, it's really not. Let's take just UI animations for example. I can create a fade-in transition with a custom timing function movement in 3 lines of CSS today. To do the same thing natively 20 years ago involved writing hundreds of lines of custom OpenGL code. I get that there's a lot of fatigue from framework churn, but the layers of abstraction are orders of magnitude higher today, to the point that most UI work has become an afterthought.
QT wasn’t just a video player but also an interactive environment, so this was a lot of API surface.
Yue was the only option that 1) supported macOS, 2) supported Lua, 3) was sufficiently comprehensive to build a non-toy GUI app, 4) and that I could integrate into my (static) build. I couldn't even get the wxWidgets Lua interfaces to compile, and Qt and FLTK wrappers had similar stories, whereas reverse-engineering the baroque Yue build (based on Google's internal build systems) was relatively simple. The first candidate I tried was libui, but it had some crashing bugs that didn't inspire confidence, the widget set was bare, and most importantly development had ceased. Yue had some sharp edges, but I was able to work around them whilst patiently waiting for patches and fixes upstream.
Immediate mode interfaces were a non-starter for me. For a non-trivial set of otherwise typical controls and window management you have to implement too much yourself, plus being non-native they not only felt wrong (which admittedly is somewhat subjective; the younger crowd seems to think non-native, immediate mode interfaces look more state-of-the-art), but lacked other interfaces for proper desktop integration, like theme change signaling (i.e. notification that a user switch between light and dark modes in the macOS system settings panel).
All in all I would highly recommend Yue.
>despite the very steep learning curve.
?
I used your quotes out of context, I know, because those phrases are in different parts of your comment, and I did it because I could, ahem, reuse your words, and it looked cool to do so, but the question is genuine:
Why do you recommend it despite the very steep learning curve? Because of support for Lua? macOS?
The only possibly unnecessary complexity with Yue is the build framework, but that sort of thing can be quite subjective and a function of various requirements (personal, third-party libraries--it uses Facebook's Yoga for automagic layout and reflow) and familiarity. Excepting libui (which is basically defunct), all the frameworks I investigated had terribly complex builds, so it's pretty much par for the course. Notably, Yue releases prebuilt binaries with every release. I couldn't make use of those because I wanted a fully static, fat application binary (x86-64 + arm64), and also I needed a convenient, local Yue build for writing and testing patches.
- Windowing
-- Tabs
-- Menus
-- Painting
-- Animation
-- Text
-The compositor
-Handling input
-- Pointer input
-- Keyboard input
- Accessibility
- Internationalization and localization
- Cross-platform APIs
- The web view
- Native look and feel
On top of that DOM and CSS implementations to achieve declarative UI. And JS as a languuage behind UI - declarative in some sense way of defining UI behavior.
Right now I'm using libgdx so I can have a cross platform UI and then specific code for USB communication for each platform.
Sciter would be great if I could use it.
When you say "Native compontents" it still sounds very UI focused :P
Been there. Done/sort of doing that. Have impressive collection of scars. There are no t-shirts to be had.
This document only barely scratches the surface of the work necessary. If this is your calling, by all means, but expect to sink a good chunk of your life into it. Things are glossed over in a paragraph that mean years of work. (My favorite is the internationalization section, closing with "But don’t worry: there’s a standard. All you need to do is implement it." - All you need to do, indeed :)
To be clear: Colin knows it's years of work. He's been working on a number of excellent projects in the space (xi-editor, druid, ...). But I really feel this deserves a warning sticker, lest you accidentally spend a decade or two of your life on it.
But "yes", you need some prior experience, in order to come up with flexible architecture that will allow to expand it further without major redesigns.
First version had DOM/CSS and used just GDI. GPU support (Direct2D, OpenGL|Metal|Vulkan via Skia) was added later with API staying the same.
I have no idea how other team succeed at it either.
<edit>
Recognize every item on this list. EVEN THE WEBVIEW ONE https://github.com/godotengine/godot-proposals/issues/960
Unicode variables names https://github.com/godotengine/godot-proposals/issues/916
So you want to write a GUI framework - https://news.ycombinator.com/item?id=28127492 - Aug 2021 (173 comments)
They want you to open a window and build a widget inside the window. The APIs for rendering to a byte buffer and injecting bogus input events seem kinda weak. If I make my own toolkit, I'll focus on that first. I don't want such a huge part of the app to be an untestable I/O leaf.
And, Win32 has a whole separate subsystem for unit testing UIs. Read more here: Win32 -> "UI Automation Test Library"
Man, there is so much damn misinformation about Qt on HN. It is tiring. I don't understand the source of this misinformation. Part of it probably stems from their various licensing mishaps over the years that seeded a bunch of FUD. It's a ridiculously good cross-platform UI library. KDE would not exist without it! I don't work for them, but I have used the toolkit for years. Nothing beats it. Yes, it is huge, but it does everything you could ever need.
Maybe I didn't check the docs closely, maybe it's better in Qt6. I stopped working on GUIs a while back.
Fwiw HTML and xaml were just as bad for that.
Also, Qt4 was last released in 2011. That was nearly 12 years ago. Maybe... try again?
Also, you can Google: qt widget without window border
Plenty of hits.
The real issue, in my opinion - as someone who's worked in a litany of UI systems and built my own UI frameworks and libraries: it's a lot of frankly boring as shit work that everyone gets into and then drops like a rock once the dopamine rush of implementing basic widgets wears off.
(Then you add on that the number of people who know the big three (Cocoa/Win-whatever-they-use-now/GTK-or-QT) really well is a much smaller pool)
While not trivial, nanovg shows that it can be done reasonably well. At least well enough for a GUI. Thankfully a GUI doesn't need a very advanced 2D rendering engine (see Dear Imgui or egui). Vector art apps can write their own engines.
I wanted to have a GUI inside the game itself (so you can edit games while you play), but that means I had to develop my own GUI. I created simple game GUI's before, so I thought this wasn't a big of a deal. Oh boy was I wrong. A couple of buttons and textfields is very different from a full widget toolkit.
And then another thing: I originally developed RPG Playground in ActionScript3, so I latched on to the native textfield they provide. In fact, most GUI toolkits you see are able to make use of some native textfield functionality.
So after Flash was killed, I ported my product to Haxe. And then realized that I can't really latch on to any native textfield functionality. Yeah, you guessed it, I had to write my own textfield code: word wrapping, mouse handling, ... . That was another project in itself.
The obvious downside is the lost of "native feel". But i think this has been overblown. Probably a reaction to horrible look for 2000/2010 java/jwt applications. But the web doesn't have a native feel, but still can look great everywhere.
Using dynamic content for tab is just only barely useful and still persists in many browser and applications (such as postgres or obisidan). What happens is that you will open a large number of things in this will completely clutter your tabview. A list which is filterable will always be better to navigate than some tabs with dynamic content.
The article specifically mentions Rust which does have Mac and Windows bindings. That’s necessary, but not sufficient. Where is the macOS or Windows Rust framework that makes writing fast, safe, beautiful apps in Rust the best and easiest choice?
Surely a developer with modern tools and knowledge can do better than Microsoft did in 1992 with C++ and MFC.
Implementing these APIs for Windows and macOS seem super arcane. It's hard to find docs on implementing them. I guess there must be a good reason there are not cross-platform libraries that abstract IME specifically.
Java still seems like the only language with cross platform GUI and swing/awt are on the way out with javaFX being “abandoned”? (Probably not the right term but it never was made main stream). You got to Python, etc and it’s an absolute mess. C/c++ have a few but cross platform still seems tough.
Don’t even get me started on the mess of electron.
It's curious what these languages have in common too: as languages almost nothing. But they're all venerable, proven & respected in their domains if not beloved, and relatively unpopular.
I don't know how to interpret that. Maybe it's just a coincidence. Or stdlib gui frameworks are just out of style and it's survivorship from the old ones. But then python is from around the same time, why doesn't it?
Edit: this is it, Ruby Tk.
IME is not Skia's responsibility.
IME handling must be implemented at OS level.
A different view: It might be possible to write an IME for X11 using Skia. :)
The lack of unified standards in X11 and especially in Wayland/libinput is a huge source of problems in accessiblity and user experience of traditional Linux environments.
Each of those IMEs (ibus, fctx,...) are a major pain to setup and configure correctly. They often don't look "native" to the toolkit of the desktop.
Those were the days!