Everything you ever wanted to know about terminals
xn--rpa.cc
xn--rpa.cc
Sure, if all you care about is coloring text on recent terminals, I do think just using ansi codes is fine.
However, I completely disagree with the assertion that hardcoding ansi codes is somehow “more readable” than using a well-named API call that abstracts such detail away from the user. Maybe I’m alone in this, but I really don’t want to waste brain space memorizing ansi codes. Yeah I can write my own library or little macros to do this but…why? If I’m doing anything more involved than coloring text why wouldn’t I just pull in the lib I need? There’s no way in hell I want to maintain a custom implementation of terminal cursor movement for kicks. I’d much rather reach for a battle tested library that scores of other devs have already used and improved upon, whether it’s ncurses or one of the more modern TUI libs.
Also, it’s become cool recently to hate ncurses for some reason. Are there things ncurses should do differently from today’s perspective? Sure. Is it’s api awkward to use in the face of modern programming features and paradigms that we’re now used to? Sure. But on the whole it’s design is really solid and it has plentiful documentation. Whenever I see rants like this I get the impression the author is a person that lacks an appreciation for history and the way technical development proceeds in general and imagines they are our technical savior, come down from the heavens to enlighten all us blindered fools about how terrible X is.
If you want to educate people about ansi codes, great. A really inefficient way to do that is to wrap your advice in an obnoxious rant.
/s
Sure, going with VT102-family codes probably covers somewhere between four and seven 9s of cases, but...
Then I noticed Pygments, a Python-based library for pretty formatting source code, has an option to output an ANSI formatted file. I quickly found a bunch of libraries that could render ANSI formatted text to a print canvas.
In the end, I put the original source code file through 'pygmentize -16m -o tempfile.an` (`16m` is the 16M color terminal ANSI formatter) and pipe the `tempfile.an` through a print-optimized renderer to actually print the source code.
ANSI escapes FTW!
[1] WinPrint - https://github.com/tig/winprint [2] https://pygments.org/
Nothing like as powerful as your app and entirely tangential to the topic of ANSI escapes, but my preferred way to generate HTML from source code is simply:
vim +TOhtml +wq +q path/to/source/file.ext
Definitely worth checking out the full landscape these days if you're going to dive into making your console programs prettier.
[1] My words, I just coined that name (although I wouldn't be surprised if others had said it too). I mean that in the last ~decade there has been a lot of TUI work in the background, with lots of new programs and some pretty stunning results. I blame unicode - once the web folks realized they could use something like font-awesome instead of sprite-sheets (or maybe as a pre-build sprite sheet?), and the "icons in a font" movement took hold, terminal programs got a lot prettier too. It makes sense, its the same font/font renderer provided by the system no matter if the app is a browser or terminal emulator.
I'm suddenly really excited for what can be done on TUIs. Like I was some 40 years ago.
Edit: and you can write CSS instead of curses? That's seriously cool.
This so much! Dank, Jart, Yatli and HPA are 4 people whose work is single handedly pushing TUIs forward.
There are others, but these 4 are the poggers, and I don't think the likes of me (or Christian, or Autumn, etc) can stand the comparison.
Notcurses is a unique work of love and art worth all the praises in the world.
The title should be changed to (2019) because this post came out at about the same time as the teletypewriter aristocracy kicked off the renaissance.
https://github.com/migueldeicaza/gui.cs
(Disclaimer, I maintain Terminal.gui).
I'd recommend that over the wikipedia link in the article, since wikipedia seems to be missing some things.
Then there's other specs not included in that doc even outside of the aforementioned proprietary codes. Like Sixel, conventions on non-POSIX terminals, etc. Also lets not forget the popular-but-not-standardised conventions like the hyperlink escape codes (which I personally think shouldn't exist in the first place....but that's another topic entirely).
Even standard ASCII characters can differ from one platform to another. Backspace being a classic example: ISO 646 describe it as ^H whereas ASCII 1963 has it as ^?
This mess of differing compatibilities is exactly why termcap is a thing.
Being an author of a readline library, this is a topic quite close to my heart :)
Would be good to add support for that to my image rendering library: https://github.com/lmorg/murex/blob/c77b893a58a682735ad7502c...
Fork used by iSH: https://github.com/ish-app/libapps/tree/master/hterm
I looked at the docs (https://github.com/ish-app/libapps/blob/master/hterm/doc/Con...) and it appears to use the same protocol as iTerm.
Here's an implementation: https://github.com/ish-app/libapps/blob/master/hterm/etc/hte...
> When the PC bucket keyboards came, IBM in their usual idiocy simply put the [BACKSPACE] key where the [DELETE] key normally is, and what's worse, they made it not only move back one space but delete too. Thus came the sadness of everlasting redefining of deletion to be ^H when using a PC bucket keyboard.
Indeed. And to confuse things even further, the [DEL] key on IBM keyboards sends the following ANSI escape sequence: [27 91 51 126]
So we now have 3 different standards for deleting characters (and that's before you start looking at the escape sequences for deleting rows, nor supporting vi and/or emacs/readline bindings).
There's still plenty left on that list, though -- including at least one escape sequence I'm fairly certain was included as a joke (SPQR).
The functionality is legitimate (although I doubt anyone uses it), but the name is pure silliness -- much like the PowerPC "eieio" instruction.
This isn't me being critical in what you've done though. More of an adjacent commentary about the state of ANSI escape sequences.
Either way, I wouldn't use it as an authoritative source uncritically.
The VT510 manual is also useful since it was designed to be used with a PC keyboard so it covers newer escape codes that aren't on older DEC terminals https://vt100.net/docs/vt510-rm/chapter8
* The ESC [ command start sequence is actually a compromise for 7 bit systems. The [ character is not chosen by accident. It has an obvious positional relationship to ESC in the ASCII code which is why, informally, Ctrl-[ is the same as ESC.
* If you have an 8-bit-clean channel to the terminal, only a single character is required: the "upper escape" from the C1 control character set (0x80 to 0x9F). This character, 128 + 27 or 0x9B is called CSI: control sequence introducer, which is basically its role in these terminal control sequences. Thus ESC [ is just an alternative way of encoding CSI for 7 bit. E.g instead of CSI 4 A, you use ESC [ 4 A.
I'd avoid using this. It conflicts badly with UTF-8's use of 0x80 through 0xBF as continuation characters.
Of course, if the terminal is ignorant of UTF-8 then this is a nonstarter, because whenever CIS occurs as a continuation byte, it will be misinterpreted. If the terminal is ignorant of UTF-8, why would you send UTF-8 to it, though? If it's going to be interpreting UTF-8 as some branch of ISO Latin, the display will be a mess.
If the terminal and host do handle UTF-8, then this CSI signaling is just an extension of the state machine. It's also nice and simple that, in the absence of any data loss or synchronization error, the CSI code is unambiguously not part of any valid UTF-8 character (except as the second or subsequent byte where the receiver is in the right state to interpret it that way).
In my experience, it's the terminal->host direction where you get mixups, whereby the terminal generates some escape sequence like for an arrow key, but the host is not in the right state, and interprets part of it as data. This is exacerbated by a situation in which the host supports ESC as a UI command. CSI solves the abmbiguity between the control sequence start and ESC just being ESC.
The ambiguous nature of ESC in the terminal->host direction (as you put it) is unfortunate, but is difficult to fix. Some terminals (like iTerm) can be configured to use C1 controls for function keys, but my experience has been that a lot of software fails to recognize these sequences, making it impractical to use.
> It can't run a UTF-8 decoder first, because that'll turn the C1 controls into replacement characters, and it can't run a terminal sequence decoder first either because that'll treat a lot of the UTF-8 continuation characters as control sequences.
It has to have a state machine which recognizes the combined language of UTF-8 sequences and control sequences. Which is the approach you would take anyway, even with C0 controls.
That combined language is an unambiguous, regular set, so you could code it with your eyes closed.
Starting in an initial state, the legal inputs are: ASCII character, Unicode character, or escape sequence headed by CSI. This is decidable from reading exactly one byte value with no further lookahead.
That's just one way. You can in fact follow a layered approach whereby the terminal decodes everything with UTF-8 before analyzing it for control or data.
For instance, say we decode UTF-8 into integer code points. A valid character decodes into its implied code point. An invalid byte like CSI can decode into some reserved range like U+DCxx. The higher layer of the terminal's firmware then looks for values in that U+DCXX range: that's where it finds the CSI.
I have years of experience with this exact encoding scheme, which I baked into the text I/O streams of a programming language.
For instance, oh, /proc/self/environ is NUL-separated, right? No problem:
1> (file-get-string "/proc/self/environ")
"CLUTTER_IM_MODULE=xim\xDC00XDG_MENU_PREFIX=gnome-\xDC00LANG=en_CA.UTF-8\xDC00;D
[...]
-go:@/tmp/.ICE-unix/2065,unix/sun-go:/tmp/.ICE-unix/2065\xDC00GTK_IM_MODULE=ibus
\xDC00_=/usr/local /bin/txr\xDC00"
2> #\xDC00
#\pnul
3> (take 4 (spl #\pnul *1))
("CLUTTER_IM_MODULE=xim" "XDG_MENU_PREFIX=gnome-" "LANG=en_CA.UTF-8"
"DISPLAY=:1")
Putting a terminal into C1 mode is a great way to have it do unpredictable and weird things unless printable input is constrained to the lower seven.
If they are being encoded, this is still useful. Though space isn't saved, any ambiguity between the control bytes and UTF-8 bytes is eliminated. The advantage is still present that CSI is different from ESC, and so in the terminal->host direction, you don't have the ambiguity between ESC as a UI command character versus control sequence signal byte.
Actually, yes, understanding the tty in detail seems to become a dark art.
However it's the best way to do complex things quickly: I did use some of these tricks like storing and then restoring the cursor position to have the time at which a command stop executing ABOVE the command itself and next to the time it started executing in https://github.com/csdvrx/bash-timestamping-sqlite
I had to, because I was using MSYS2 and the time to execute a command was a limiting factor in Windows before WSL2.
No, WYSIWYG: I don't bother much more than that with explaining how it's done: the source code + the comments + the github page already give all the details away to whoever wants to dig deeper.
I'm not much into social media or self promotion either: if people like what I do, they'll use it - I don't care much more than that, as everything was written for myself and my selfish needs first.
The tty/sixel world is very small world anyway, we generally know and recognize eachother, so we know where to look for cool new stuff :)
To be fair it was more like a joint venture between us!
If you keep an accounting of how many lines you have displayed since then, you could alter the response to RCP by also doing the appropriate amount of scrolling: it should only take one variable, the deltaY to scroll.
If you want to test the idea, I think you could even use tmux and send commands to control the scrollback cf ahttps://superuser.com/questions/209437/how-do-i-scroll-in-tm...
I've used similar tricks with RCP/SCP but for simpler things: the only slight difficulty is the deltaY accounting, like when you are executing commands near the bottom of the screen because you must take into account that scolling will happen - but it's essentially similar to your idea.
So check https://github.com/csdvrx/bash-timestamping-sqlite/blob/main... and make sure you understand both how the __notbottom function works, and why PS0 needs an extra Esc[2a
Actually, now that I think more about your idea, it would be sweet to keep a SCP/RCP stack with multiple values, where you can push values with each SCP then pop them with RCP, say in sequence, or maybe just access the nth value with a different command that wouldn't pop them? That could be done nicely by augmenting RCP.
Also you could augment SCP with an optional flag to specify whether the terminal should scroll back upon RCP of this nth entry, and you'd have a great function that would be quite useful (ex: SCP with a jump bool when the return is non 0: you could make a shortcut to jump to the commands that have returned errors)
There's no reason to stop adding cool features to terminals: we're in a terminal renaissance!
How does it work with buffered stdout? For example, if you wanted to colour a single word?
I was also surprised when I discovered this. IMO this should be used everywhere but I've rarely seen it.
> How does it work with buffered stdout? For example, if you wanted to colour a single word?
This is not affected by buffering. What matters is what the output device/file/stream gets eventually.
How coloring a single word works depends on how you use it.
You can do
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_dev_tty("word");
write_and_flush_to_dev_tty(END_RED_TEXT);
Redirection won't have any effect on these lines of code. You always get a red "word" on the terminal. The redirection target doesn't get anything.
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_stdout("word");
write_and_flush_to_dev_tty(END_RED_TEXT);
write_and_flush_to_dev_tty(BEGIN_RED_TEXT);
write_and_flush_to_stdout("word");
write_and_flush_to_stdout(END_RED_TEXT);
When stdout is a tty, /dev/tty is the same as stdout, so a write that goes to one goes to the other. In this case, even if you don't flush after write immediately it's likely not a problem. Still it's something that the developer should pay attention to.
-------------
Edit:
Please ignore all the pseudocode examples. They don't convey what I wanted to express.
Just use /dev/tty in the same way as how you would use stdin/stdout/stderr that is connected to a terminal. Only difference is that it is a rw device that can be used for input and output at the same time.
This isn't guaranteed to be serialized: Someone trying to log the output of your application might try: | tee /dev/tty | logger ... or they might be running under kubernetes/docker (which does much the same thing).
I suggest the following:
1. If fd 0 and fd 1 are both ttys (isatty), and they point to the same tty (ttyname) then use it a tty (like your first example)
2. If fd 1 is not a tty, and fd 0 is a tty, write plaintext to fd 1. The user wants to filter output.
3. If neither fd 0 or 1 is a tty, write twice: do interactive stuff and send your vt-sequences with the text embedded to /dev/tty and a plaintext copy to fd 1.
Now the user doesn't need the extra tee, and we no longer need to worry about synchronisation. Users don't typically (meaningfully) grep interactive applications, so I think it's a good compromise for interactive servers and tuis.
But if you're not reading input or doing absolute cursor movement, and you just want some spicy log lines, I don't think you should be doing any of this fiddling: Just write sequences to stdout if it's a tty (or if the user specifically tells you to).
I also recommend checking for $NO_COLOR in the environment and honouring the users wishes here: Some users are colourblind so this represents a real accessibility issue for them. One of the advantages of ncurses/terminfo is you get some accessibility features you might not have known you needed.
We're basically recreating Flash for the terminal, where we got a fancy UI but we lose the original text functionality, i.e. output is non pipable anymore. Not ideal when working on UNIX systems.
Yes I have. Using /dev/tty doesn't stop an application's author from adding a --color that lets their program send color codes to stdout. But if the author doesn't use /dev/tty either stdout or stderr of their application can't be redirected.
Search color, its the first result as an abbreviated reference and the second as a full explanation. Its also the first result on google/ddg/etc.
You have a good argument, but claiming you have to "flip through a 623 page manual" and "with my sed solution, anyone who knows regex could write it in a few minutes" detracts from your point.
Also, it will include the null terminator. Probably won't do any harm but quite silly if you're redirecting to a file for example. I'd subtract one, or use strlen which would cover the pointer case above and I'd hope a modern compiler would elide the call on a string literal anyway.
[1] #define szstr(str) str,sizeof(str)
Almost every day the mandatory prayer "I wish they had used curses"
anyway the content looks good so will try to find time to sit down and read this "essay" soon
The final example is both a total treat and a beast. I encourage anyone with any interest to chuck the code into GCC and behold the interactive experience for themself.
It's interesting, because I sometimes also find colors distracting, so for my Vim and other terminal tools I make an intensive use of text attributes (like bold, italic, underline and their various combinations)
Your comment may incentivize me to release my monochrome vimrc: it looks quite good on mintty/msys2 or Windows Terminal, and still very decent with xterm!
And terminfo, pty, ctty, process groups, and the rendering width of a utf-8 string, and, and and…
> i might be one of a handful of people left on earth who actually has the knowledge to
Uh, no. Anyone on BBSes in the 90s is very aware of ANSI, thank you. And we've not died off yet.
And honestly it's really not that hard at all. Even on the topic of terminals this is not even the hardest aspect of it.
> i have effectively zero pull in the tech community
If you think there's all there's to terminals, yet you act as if you know it all, then you shouldn't have any pull.
I would also not accept any pull request that looked even remotely like their example code.
Here's a more interesting article about a less understood aspect of terminals: http://www.linusakesson.net/programming/tty/index.php
https://hn.algolia.com/?query=The%20TTY%20demystified&sort=b...
You guys should be more vocal. When you have great knowledge like that, you can't just keep it to yourself as a fond memory. People wouldn't be saying what the OP said if more of the oldskool crowd was out there blogging and mentoring the younger generation.
It's not obscure.
One thing I actually agree with the author about is that these escape codes are the only relevant thing. Outside of retro computing nobody should care about supporting anything else.
Other programs doing this are not exactly in short supply. Anyone can do "ls --color /bin/ls | hexdump -C" and see the secret sauce.
I blog about various things, partly to help out people who are less experienced. But I don't pretend what I write about is some sort of lost art, that "only a handful of people know".
Like "how do I make bold text in linux terminal" gives as second result this:
https://askubuntu.com/questions/528928/how-to-do-underline-b..., which also links to https://misc.flogisoft.com/bash/tip_colors_and_formatting.
Searching on youtube I immediately got this, for those who prefer in video form: https://www.youtube.com/watch?v=OL21-EnsNjQ
And it manages to do this without the pure arrogance of "i might be one of a handful of people left on earth who actually has the knowledge".
Seriously, this whole article could just have been just:
"Q: How do I output more than just print black&white text to the terminal? A: ANSI escape codes. See https://en.wikipedia.org/wiki/ANSI_escape_code"
Not "everything you wanted to know about terminals".
Like, "how does Ctrl-C work? What's flow control?". No, this post is entirely about ANSI codes.
What's extra frustrating is that you too are calling this "oldskool crowd". I'm just not that old, and this just isn't forgotten. It's simply another tool that people use when they need to solve the problem of colors, etc.
Just because many people don't know how a malloc()/new becomes a mmap or sbrk doesn't make it "oldskool". It's simply a thing that many people have not learned yet, because they haven't needed to. If and when they need to it's quite documented and many others know it, if they want more hand holding.
Like say I didn't know how garbage collectors worked. I don't go "Oh you older generation of lisp programmers, you need to blog more and teach us younglings, so that we can understand the languages that we use". Sure, blogging etc about GCs is good, but who would be arrogant enough to just write an article about "old gen and new gen" and call it "everything you ever wanted to know about GC" and claim that they are one of the handful of people who understand GCs.
I don't even believe that you believe that this is an accurate description of the post.
> That's not arrogance
Straw man argument. I was not commenting on those things, and you know it.
> and your comment comes across as very condescending which is worse than arrogance.
Clearly the original article is both condescending and arrogant. And elitist.
> "oh, you think this stuff is new? it's been documented a thousand times before, don't bother".
I didn't say that. There's value in writing it again. I don't pretend that my blog post are pushing the envelope of knowledge either, but maybe I'll explain it in a way that fits better with how some reader will better absorb it.
Anyway: I am familiar with you from the past, and how even your friends have described you as someone who likes to live life at the edge of trolling. I sense that this is what's happening now, so I will not engage further.
I wish you good day.
From the tone of this piece I gather that the ANSI escape codes are actually standard enough to target. Cool! Thanks for the heads up.
termfo[1] comes with a "termfo" CLI utility which – among other things – can group terminals by escape code; for example "termfo find-cap save_cursor" shows that almost all terminals use "\x1b7", with just a few very old ones using something different (full output is a bit long, but it's at [2]).
It's useful to check "can I safely hard-code this escape code?" But like you said: for ANSI it's pretty safe to just hard-code most codes, especially the common ones, but never hurts to check.
Correct. Hardware terminals are extinct in the wild, and essentially all software terminals (including the Windows terminal!) now support a reasonable subset of "extended VT100" terminal control sequences. Some of the weirder features of the VT100 (like double-high/double-wide text or VT52 compatibility mode) are usually omitted, and some features which were added in later DEC terminals (like color) are often added.
Even now there's big differences between terminal emulators. Especially when you take modern features like images into account, there's like 6 different ways of doing it as some terminals have invented their own (kitty for one) and there's more standard ones like sixels which come originally from the Dec VT340 series. Though they were ridiculously slow on that hardware so it didn't take off until much later.
So long as you're only concerned with placing and formatting characters on the screen, support for the subset of "extended VT100" which is required is essentially universal. There is no disagreement over what escape sequences can be used to move the cursor, for example, or to clear the screen. Using an abstraction layer like termcap is effectively a no-op here; it will output the exact same sequences for any modern terminal application.
And if you want to support modern and/or esoteric features like embedded graphics or custom characters ("sixels"), you're probably going to handcraft that anyway, as support for those features is very limited, and most abstraction libraries won't support them at all. So I'm not sure I see the case for an abstraction at either end of the spectrum.
Only since Windows 10 version 1511 (released 2015) which may or may not be too recent for you to assume support.
-F or --quit-if-one-screen
Causes less to automatically exit if the entire file can be displayed on the
first screen.
(Also handy: add in `-X` to leave the contents on the screen when you exit less)
You'd be better off piping into less / more / most. These are called "pagers" and are designed to do this. eg
cat large-file | grep common-phrase | lessyour_command_here | less
It will capture the output and let you scroll up and down.
Otherwise, this would really be a feature of the terminal, and not necessarily required to be supported (it makes sense when you consider that many early real terminals literally printed the output to paper -- who needs scroll-back, just look up the tape! And repeating previously shown output would result in a confusing print-out).
However, if your terminal emulator doesn't have a scroll-back, you could try something like tmux, or alternatively gnu screen. These add lots of little features to the terminal (a buffer to scroll back, split terminals, persistence so you can detach and reattach) (tmux is more modern feeling, IMO).
Assuming you tried moving the cursor vertically, have you tried the alternative approach of Store Cursor Position / Restore Cursor Position cf https://stackoverflow.com/questions/28986776/ansi-escape-seq...
The scrollback buffer may be a limitation, so you should try with enabling or disabling the alternate screen (ti/te)
If that fails, you may have to tweak your terminal emulator to "hook" the SCP/RCP to a specific point of the scrollback buffer, to allow this scrollback.
It's not hard to find yourself with an incorrectly configured terminal on a modern Linux distro. For example, try running emacs inside tmux inside rxvt-unicode and find out how Ctrl/Shift/Ctrl+Shift with arrow keys are bound.
The tone and presentation seem calculated to annoy HN readers (nice) but this article has a bunch of difficult-to-uncover details with just enough context to apply them if you actually have a use for them and a desire to.
In fact they mention various issues with their approach that nurses would fix automatically and without the user having to configure stuff in the app:
> sadly, true color isn't supported on many terminals, urxvt tragically included. for this reason, your program should never rely on it, and abstract these settings away to be configured by the user.
Apparently they don't care about the great terminal-independence ncurses offers because these ANSI sequences will only work fully on terminal emulators. And as they mention even those differ in supported features. Ncurses was created precisely to abstract these differences. It's not just good for "obscure dumb terminals from 1983". It would also be quite a pain having to deal with terminal resize events etc when building a TUI. And no, doing a full screen rewrite for each minor update does not make for smooth TUIs. Yes modern hardware is fast but sometimes you're on a slow SSH connection.
Personally I still use real terminals too at times (I own a real VT520 and love it) and apps that are totally ignoring termcap/terminfo are super annoying. But I know this is niche.
PS: Also from the article:
> also, i have effectively zero pull in the tech community and am also kind of a controversial character who is liable to give projects a bad reputation, which i don't normally care about
Gee, I wonder why...
It seems less and less people want to use the terminal.
All the new developers I notice want to stay in the IDE and almost never want to get their “hands dirty”.
Thanks for writing that great resource.
Is that correct?
I seem to recall having a hell of a time trying to figure out which key combinations Emacs could understand and why (keyboard -> OS -> terminal emulator -> protocol -> program.)
Are there any proposals? Workarounds? Proofs-of-concept?
It's not exactly like that - it's more like there are competing "standards" and interpretations of these standards, and sometimes supporting one means not supporting the other.
Fortunately, such things are rare, and can be addressed by GUI options.
Take for example SGR1 for "bold/intense" text: read the whole issue that came to Microsoft Terminal team in 2018 up to its most recent discussions on https://github.com/microsoft/terminal/issues/109 then check the "simpler" version in wikipedia https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_.28Select... and you will realize there's no "right" answer between bold and bright (or both!), just different preferences and interpretations of the standards.
Are there standards to represent, for example, Ctrl+Alt+* (I.e. Ctrl+Alt+Shift+8) ? Additionally with cursor keys, Insert, Delete, Backspace, F1 - F12, etc.
I’ve searched several times for such standards before but never had any luck…
Sorry if I misunderstood your question!
However, for inputs the situation is essentially similar: the standards again depend on the terminal, and there can be multiple interpretation than can be conflicting.
Take for example Control-H and Control-? : which is delete and which is backspace depend on the terminal you use!
> I’ve searched several times for such standards before but never had any luck…
There are sources, mostly Thomas E. Dickey (of xterm fame) but personally, I try to take an emulator and make it behave to what my fingers are doing with the minimum level of configuration.
To follow up on Control-H and Control-?, here's what my .inputrc does:
### Delete to the right will be remapped to Delete
"\e[3~": delete-char
# keypad on .
#"\eOn": delete-char
# Special case for VT100 (borks backspace -> make it conditional)
$if term=vt100
"\C-?": delete-char
$endif
### Delete to left will be remapped to Backspace
"\C-?": backward-delete-char
# keypad on 5
#"\eOu": backward-delete-char
# Special case for VT100 (borks backspace -> make it conditional)
$if term=vt100
"\C-H" : backward-delete-char
$endif
(yes, the numpad delete has its own case!)
Here's another one, for Control-Backspace and Control-Delete
### Line cuts will Ctrl-Backspace and Ctrl-Delete
"\C-_": backward-kill-line
"\e[3;5~": kill-line
# Ctrl-Delete variants for tmux and urxvt
"\e[M": kill-line
"\e[3^": kill-line
And last time I checked, Windows terminal needed proper remaps cf https://github.com/microsoft/Terminal/issues/755 like { "keys": "ctrl+backspace", "command": { "action": "sendInput", "input": "\u0017" },
This would piggyback on "\C-h" backward-kill-word - which might be incompatible as it's got a different meaning for VT-100 as noted above.
Quoting you again:
> I’ve searched several times for such standards before but never had any luck…
You shouldn't bother too much about that: terminals are just tools to help you do your work, and while sometimes you can use a lot of pre-existing configurations or documentation to have as much compatibility among the various standards as possible, there are other times when you need to break the standards that stand in your way.
For example, I'm a Windows fan, so I believe Ctrl-C should be copy, Ctrl-V paste, and I don't really care about the traditional unixisms that would cause me try to learn to use Shift-Control-C for copy and Shift-Control-V for paste (just... no!!!)
You may disagree with that, so let's take a more staightfoward example: I care even less of the more obscure unixisms like Ctrl-S/Ctrl-Q that I rarely need. Who needs to "freeze" the tty output in 2022?
So in that case, I've remapped Ctrl-S to "kill word forward", something usually done with Esc-d by default.
To achieve that, in my .bashrc I first clear the defaults with stty:
## Remap ctrl-c to ctrl-x to copy/paste with ctrl-c and ctrl-v, and disable ctrl-s/ctrl-q
stty intr ^X stop undef start undef rprnt undef discard undef swtch undef -ixoff -ixon
# The above doesn't show ^S,^Q,^R and ^O anymore in stty -a
Then I can do the remappings in my .inputrc:
### Backward-kill-word is ctrl-w, while kill-word is alt|esc-d,
# so instead, map ctrl-s which makes ctrl-k|u and ctrl-w|s very close
"\C-s": kill-word
All that to say, standards can be helpful, but there's nothing set in stone: with Control-H and Control-? already causing such headaches (but following all the standards) I don't feel so bad for abusing the standards with my use of Control-S for forward kill word, as it complements nicely the existing Control-K, Control-U and Control-W (all the 3 of which are standard BTW!)
Yes, this is correct. In particular, there are certain key combinations which cannot be distinguished (for example, Control+I is indistinguishable from Tab) and some keys aren't affected by modifiers (such as Shift and Control on Space).
There's an extended discussion of this issue, and a proposed solution, at: http://www.leonerd.org.uk/hacks/fixterms/
keyboard -> OS -> terminal emulator -> protocol -> shell -> program
And that’s for local IO…
(Too late to edit my above comment.)
$ wget https://justine.lol/ttyinfo.com
$ chmod +x ttyinfo.com
$ ./ttyinfo.com
"\001" is CTRL-A a.k.a. ^A
"\000" is CTRL-@ a.k.a. ^@
"\002" is CTRL-B a.k.a. ^B
"\f" is CTRL-L a.k.a. ^L
"\r" is CTRL-M a.k.a. ^M
"\033\001" is ALT-CTRL-A
"\033[A" is UP
"\033\033[A" is ALT-UP
It'd be great if the standard bodies could promulgate some kind of well-defined solution to this problem. Having read the ECMA standards documents, I think they really could be doing more to focus on the people who use it for terminals!
I’m not even geriatric.
<meta name="viewport" content="width=device-width, initial-scale=1">
This author is a much much better writer than average for free technical content! They have an interesting, unique style! The typography and punctuation are part of that style! It would be tangibly worse if they adhered to the (bad, and also arbitrary!) typographical conventions of raw html just to please a bunch of square-ass nerds. Shit just makes me sad seriously.
Citation needed. Please cite your source and make sure you strictly follow MLA format
Her 'interesting, unique' 'typography and punctuation' makes me want to poke my eyes out instead of focusing on the content itself. You sneer at conventions, but they were developed after centuries of typesetting and typewriting.
Her entire style sheet is an abomination:
- The serif typeface she has used, EB Garamond, is best used for printed text. I am working on a 27" 1080p monitor, and the low DPI makes for very painful reading. Sans-serif humanist typefaces like Verdana (used at Hacker News, thank goodness) or Open Sans are best.
Better still, use the OS typeface in `font-family` (`-apple-system`, Segoe UI, SF Pro, Roboto, Lucida, Ubuntu, Open Sans, etc) instead of dragging in your own. If one so desperately wants a serif typeface, please use something that's nice and blocky, like Droid Serif or IBM Plex Serif.
Incidentally, I have the same qualms with the default typeface in LaTeX: the Computer Modern provided is far too spindly for digital reading. There is an alternative, MLModern[0] that is thicker, but it only has Type 1 glyphs rather than OpenType.
- The colours she has chosen are not as bad, but they are certainly distracting. Can't go wrong with a monochrome dark grey/white, or even straightforward black on white.
- Uppercase/lowercase letters and proper punctuation help break up the prose, and improve the reading experience by differentiating proper nouns, beginnings of sentences, etc etc. Dismissing this by saying 'Oooh, look at me, I'm different for different's sake' is just exasperating.
Incidentally, I'm probably of a similar age group as the writer, but I like to follow grammatical and typographical rules in long prose (I don't care as much in personal texts), because they make sense.
“Oh look, I am a rebel ignoring what they taught me to do in third grade”.
Much like all capitals is seen as shouting.
It’s a way of saying, you are not worth talking to properly.
As a father trying to teach two small children to read and write, I've begun to think it's kind of annoying and unnecessary to have two different forms. It's easy enough to teach when they are just bigger glyphs, e.g. O -> o, C -> c, S -> s, etc. But having uppercase and lowercase that are entirely different is just a pain in the ass, e.g. A -> a, D -> d, G -> g, R -> r, etc.
I do wonder if literacy might be improved if we got rid of uppercase, to little to no detriment elsewhere. (I'm sure someone here can probably sight some data on that!)
[1] apparently an "overcompensation" coming from a language in which all nouns are capitalised.
You don't necessarily need ncurses to have a portable smart terminal experience, but you do need terminfo.
ncurses magic relies on the lore stored in terminfo, which contains all of the obscure escape sequences and other information about the disparate world of terminals.
They're maintained as a combined package, but you don't need ncurses to drive the screen. You can get the codes yourself if that's what you want to do. You can use tput to make colorful labels in shell scripts.
While much of the world has moved on to the One Terminal running on the One OS, not everyone has.
> being compatible only with ANSI-capable terminals is a feature, not a bug
Driving old hardware terminals (as opposed to terminal emulators) is a fun stunt, not something of actual modern value. Every modern terminal emulator supports ANSI escape sequences. Some features are supported by a subset of terminals, but you can either 1) probe for those features by asking the terminal, which some terminals support, or 2) try them and have graceful degradation if they're not supported, or 3) have configuration options to use them, or 4) don't use them.
For every one case in which terminfo allows you to support some obscure non-ANSI terminal, there are many many more cases where terminfo won't happen to have a definition of the user's terminal (or won't know all the capabilities of the user's terminal) and you'll have less functionality than if you just used ANSI escapes. This is especially true over SSH and similar.
Is this true on Windows? My understanding is that quite a few TUI libraries don't have windows support.
Color ansi codes might be everywhere today, but that's no reason to disgrace terminfo, there are a ton of other features that need checking and are not universal.
The application should be checking isatty by default, unless the user overrides that.
> or your ssh session does not display full 256 colors
I've most often seen this kind of thing because the remote end doesn't have the right terminfo entries for the local terminal.
> or you question why we don't have true images in our terminals when we have bitmap escape codes for decades now
Sixel, specifically, doesn't seem to have much support other than in xterm itself (as opposed to otherwise-xterm-compatible terminals). And other bitmap extensions are something that many terminals support autodiscovery for.
> No terminal was created equal and never will. terminfo is about future as it is about the past.
Part of the problem is that terminfo doesn't have much info about the future, as opposed to the past.
For going beyond terminfo, I think autodiscovery protocols would be a better future path.
And even truecolor was added to it about 10 years after terminals started gaining support.
And many terminals just claim to be "xterm-256color".
>2) try them and have graceful degradation if they're not supported
Note: There are many cases where degradation isn't graceful. Many terminals on windows currently spew garbage on your screen if you send bracketed paste (alacritty, for instance).
>1) probe for those features by asking the terminal, which some terminals support
This requires waiting for a reply, which often isn't useful. E.g. if you want synchronized output, you want it from the very first paint (because that's when the terminal is most likely to still be resized, e.g. by a tiling window manager). So you would have to delay your startup until you've either gotten a reply or "enough" time has passed that you believe it's not supported.
Frankly, this is all a big mess and terminfo isn't very helpful, but we don't have a good alternative either.
> being compatible only with ANSI-capable terminals is a feature, not a bug, go the fuck away. terminfo is a fucking joke. nobody needs to target obscure dumb terminals (or smart terminals, for that matter) from 1983 anymore.
I don't have the expertise to opinion on this. But AFAICT most terminals that people still use[1] will self-report as xterm or xterm-256color so they'll only use the xterm escape sequences anyway.
[1] kitty is a notable exception. But it's rarely supported on preinstalled terminfo's, and manually adding them to all the remote servers that you access is a PITA.
Terminfo is not a de jure standard, only de facto. It is not in POSIX.
POSIX specifies a tput command with exactly three operations: clear, init and reset.
Right here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...
There is a Rationale section giving reasons for why terminfo wasn't standardized.
Bottom line: ECMA/ANSI is the real standard that can be used to drive a conforming terminal from any OS, without any special library other than basic I/O.
Also not just ECMA. The standard was ratified by ANSI X3.64-1979, ISO/IEC 6429, and FIPS-86 in addition to ECMA-48. The whole civilized world wants these codes (e.g. \033[A for UP) to be a standard. Because it makes console development blissful when you can ignore all the accidental complexity of curses and terminfo when you can just assume the terminal behaves sanely and isn't the leftover byproduct of bygone commercial rivalries.
The author additionally uses the writing technique of "enallage", defined as a "slight deliberate grammatical mistake that makes a sentence stand out," [2] from the article excerpt: "in other words, we need to use termios. the ugly side of termios."
In other words: the author writes in a highly informal, colloquial tone with a stream of consciousness narrative style. The article is notable for its vulgarity and usage of enallage to achieve its exceptionally high degree of informality. The author is reminiscent of James Joyce as an experienced C programmer.
[0] https://liberalarts.oregonstate.edu/wlf/what-stream-consciou...
[1] https://english.stackexchange.com/questions/29720/whats-the-...
I wonder if Chrome considers the upside down "LATIN SMALL LETTER TURNED K" potentially deceptive.
They've apparently got a whole document about what Chrome will and won't display
https://chromium.googlesource.com/chromium/src/+/main/docs/i...
punicode was a terrible idea to allow in DNS.
This is a very western-centric viewpoint. Most of the world does not use the Latin alphabet as its primary writing system.
In practice, browsers already check for this and display the "raw" punycode if they detect mixed script usage, but I wish such domains would not be registrable at all. These checks are somewhat complex and difficult, and easy to get wrong.
This would still let some homographs through. In particular, Cyrillic has a lot of characters which are confusingly similar to, or even indistinguishable from, Latin characters (e.g. "авсекморѕтѵху").
Those should be valid domains though IMHO; maybe show the used script in the address bar? I think users might be confused by that though and/or just ignore it, so idk. Then again, displaying "xn--80a6aa.com" and "xn--80ak6aa9058r.com" is pretty confusing too.
.cc isn't part of this list and I only see the punycode.
No, terminal emulators differ all over the place, especially for new features. Do you want to lock yourself in and only use the ancient ANSI codes? In, like you say, “THE TWENTY FIRST FUCKING CENTURY”? Great, you do that.
What’s that you say? You want to use modern features on terminals which support it? Do you then write “if getenv("TERM") == "spiffy-terminal"”? Congratulations, you’ve just begun implementing your very own ad hoc, informally-specified and bug-ridden terminal UI library.
> my hope is that this tutorial will curtail some of the more egregiously trivial uses of ncurses and provide others with the knowledge needed to implement a 21st-century terminal UI library
If you don’t like curses specifically, then don’t use it, but there are now myriads of alternatives.