The achilles heel of terminal UIs is that they can't display images.
The achilles heel of terminal UIs is that they can't display images.
Sure, it's something you can do with SSH via X11 forwarding since forever, but it seems the ecosystem never quite stuck the landing, and now we're here trying to figure out how to make CLI more like GUI for our own sanity.
Maybe we should give it another shot? It seems Windows is trying to do something similar with WSLg to achieve seamless integration of Linux apps within Windows 11 (using RDP under the hood), what is everyone else doing?
> "The achilles heel of terminal UIs is that they can't display images."
That statement is not entirely 100% accurate these days, thanks to some really creative hacks over the years. I'll list a few links where some fun examples live. ;)
[1] https://github.com/atanunq/viu
[2] https://github.com/stefanhaustein/TerminalImageViewer
[3] https://askubuntu.com/questions/97542/how-do-i-make-my-termi...
[4] https://unix.stackexchange.com/questions/35333/what-is-the-f...
JAWS might have dumped all their TUI accessibility features now that they think blind users should only use the GUI, but BRLTTY is based around the console. Orca does assume Gnome and/or a web browser, but it's no worse than usual with a console. (That is to say - I hate Orca, no matter where I'm using it.)
If you're making your own TUIs, there's a few things that curses sneaks in to make things a nicer experience. It uses the start of heading, start of text, file/group/record/unit separators from the invisible portion at the start of the ASCII table, which your sighted users can't see, but things like BRLTTY announce.
(Note: I don't use BRLTTY with an actual Braille display. I use it with espeak.)
TUI/GUI (qt,gtk,cocoa,...) & even has pas2js
We're all up this tree where we want all the features we're used to and we're willing to put up with software bloat and insecure software and sprawling supply chains to get it. I'm not. So I start with something minimalist and do what I can within it.
One of my inspirations is http://akkartik.name/illich.pdf
Is it an artificial limit you put on rendering?
Do you have button animations at 30 FPS?
The real achilles heel is they're not accessible. There is no terminal protocol for denoting actual objects being drawn in the window such that they can be exposed to accessibility APIs. Standard CLI tools are usable, but TUIs are not.
I have previously seen someone state that for this reason, every TUI program should have an optional CLI mode.
Also, if every UI is a TUI then the OS task switcher becomes useless, terminal programs don't have access to the full set of key modifiers, mouse support is spotty, etc.
Its amazing how one persons perspective can completely disagree with anothers.
What did you mean by the "OS task switcher becomes useless" ?
As for the OS task switcher, if everything is a TUI then you only have one app, which is your terminal, and so your OS task switcher cannot help you switch between apps anymore. You have to switch between windows belonging to the same process. It’s the same problem that you run into by doing everything inside a browser.
I have hyper meta super and ctrl work, sent through to my curses app. Are you referring to something like gnome or something else intercepting the keys ?
I do see on https://invisible-island.net/xterm/ctlseqs/ctlseqs.html that xterm has a resource modifyOtherKeys that is documented to turn keys like like alt-tab turn into CSI 2 7 ; 3 ; 9 ~, but this doesn't appear to work on Terminal.app, iTerm.app, or Kitty. Similarly there's a documented sequence that's supposed to make xterm set the 8th bit for meta instead of prefixing with ESC, but that also doesn't work on these three terminal emulators. I am not set up right now to try xterm itself (I don't have an X11 server).
Ultimately, the ways keys are sent to terminal processes by default disallow many modifier combinations (and may also have multiple key chords that send the same input). Some terminals may allow for customizing this on a key-by-key basis but most CLI apps won't understand the resulting input, and won't have a way to declare whether or not the terminal should do this. xterm documents a way to do this for most keys, but that isn't supported on the terminals I have access to on macOS.
It doesn't pass command through, which I don't personally consider a problem since I want the command key to do things to my terminal (like opening new tabs) but ymmv on that one.
It looks like kitty actually has a means of requesting this stuff dynamically with CSI = flags ; mode u (and a push/pop stack). It also says it will use CSI-u mode automatically for keys that don't have a "legacy" escape, but I didn't see this before because stuff like C-M-3 does not in fact echo as CSI-u but instead that inputs as just 3 instead of CSI 5 1 ; 7 u. Even using Kitty's "progressive enhancement" mode to request disambiguation of escape codes leaves C-M-3 broken which is rather confusing. Versus iTerm where its profile-wide CSI-u mode toggle does report C-M-3 correctly.
In any case, Terminal.app doesn't support this, and I'm not sure what else does. Looking at Alacritty right now, I don't see any direct support for this, the closest I've found is some issue comments suggesting setting up custom key bindings to simulate it (which I guess means defining every single combination of modifiers for every single keyboard key?).
As for Command, yes, it should be for actions _to_ my terminal. But for GUI apps they go to that app, and so replacing GUIs with TUIs means losing the ability to use Command for this sort of thing. For example, in MacVim I can press ⌘N to open a new MacVim window. In vim I can't, because that just opens a new Terminal window. Or ⌘A to select all, which doesn't even have an equivalent CLI key shortcut (e.g. in Vim I'd have to type something like ggv<C-End>, exiting insert mode first if necessary).
And of course proper usability of the terminal typically requires binding Alt to Meta, but doing that means losing the ability to type non-ASCII chars on macOS. This is another thing that GUIs don't have to worry about.
I don't see a way to split the difference between ⌘N for a new terminal window and ⌘N for a new application window. It has to do one of those things because it can't to two, and speaking personally, I want it to open the new terminal window. It's a tradeoff.
CSIu mode is new, it's pretty well supported though.
> And of course proper usability of the terminal typically requires binding Alt to Meta, but doing that means losing the ability to type non-ASCII chars on macOS. This is another thing that GUIs don't have to worry about.
This doesn't happen to be true though, right Option can be the compose key and left Option can be Meta/Alt aka prepend-with-Escape. I also use Karabiner to set pressing left Alt to send Escape for even more faithful terminal fidelity, but that's optional. Having both Compose and Alt as right and left Option is perfectly straightforward, and I suggest it, because much like with Command, it does have to do one or the other, but this time, we have two keys.
It's probably possible to split the left and right command keys but again, personally, I don't want the terminal app to see my command keys and am comfortable with the mere six(!) combinations of modifier + key I can send without it.
Right. I'm not saying CLI tools should be able to get command-keys, I'm saying using a TUI instead of a GUI means you cannot use command-keys. It's a notable limitation inherent to TUIs, and it's one I'm talking about since the context here is "the terminal is the most portable GUI platform available".
> Having both Compose and Alt as right and left Option is perfectly straightforward, and I suggest it, because much like with Command, it does have to do one or the other, but this time, we have two keys.
I like that this is an option in some terminal emulators. Unfortunately it's not an option in Terminal.app, and for me personally I'm using right-option for something else system-wide (though I may have chosen differently if Terminal.app offered this).
He was looking for a cross-platform GUI toolkit.
QT and Electron are the most capable cross platform tools for building UIs in my opinion. Both have their drawbacks of course, QT is a pain to work with and electron is famous for being not responsive enough for some users