What happens when you press a key in your terminal?
jvns.ca
jvns.ca
Also interestingly, the use of \x03 for this purpose is a default, but it's not hardcoded. You can change it with the stty command.
For example, if you run
stty intr '^X'
then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!).
In order to make this change, the stty program actually has to call into the kernel with an ioctl call (TCSETS or a related ioctl, for "terminal control set settings").
You can learn more about this ioctl on Linux in the ioctl_tty(2) man page, and you can see the various settings that can be changed this way with
stty -a
(It's a little bit confusing which things are handled by the readline library and which things are handled by the kernel, as the kernel's ability to support some rudimentary line-editing features on an interactive terminal long predates libraries like readline. I think readline might actually disable kernel interpretation of some of these control characters when it starts accepting input, and then re-enable them when it stops, but I've never looked into that.)
That's the canonical mode bit (ICANON). Canonical mode means the kernel/terminal is line-buffered and line-editing is handled by it; non-canonical mode means user input is pushed to the application immediately.
> stty intr '^X'
> then your interrupt character sequence will be Ctrl+X instead of Ctrl+C (!).
I have a feeling that if you actually do this in a real Linux system, a sysadmin will hunt you down and dismember you alive, but I might be wrong.
stty intr yIt cannot be a multi-byte sequence, which means it cannot be any keyboard key.
On SCO systems intr/break is the Del key, which on a scoansi terminal emits ^? (I don't remember the ascii value just that ctrl-? is another way to produce it) So to break out of programs is the Del key instead of Ctrl-C.
But on a vtxx terminal like the linux console or xterm, even if you were perverse enough to want to, you can not assign break/intr to the Del key like that, because on a vtxx terminal the Del key emits a multi byte escape sequence, not a single byte.
hot keys like ctrl-c require multiple fingers, but what's produced is a single byte.
(you can actually modify both the console and xterm to change what a key emits, but then the resulting terminal no longer matches the definition of a linux or xterm terminal)
At least on Linux, indeed you can't use 0x00 (aka ^@), because _POSIX_VDISABLE (the thing you use for disabling special characters) is 0.
^? = ASCII DEL[elete] = 0x7F. Terminals where the Delete key sends Delete are doing it right.
Why is DEL 0x7F, when other control codes are <0x20? Because the American Standard Code for Information Interchange descended from teletype codes, and teletypes often used paper tape ‘storage’ where a 1 bit was a hole. So teletype codes would normally have a delete function punch all holes, because that would obliterate any other possible character (and typical of punches, advance to the next position, making DEL semantically a forward delete operation).
> on a vtxx terminal the Del key emits a multi byte escape sequence
Only VTxxx where xxx ≥ 200. The VT100 series and earlier had ASCII Delete and Backspace keys, but in the VT2xx era DEC got some funny ideas and provided only a ⌫ key, which left us an enduring mess.
You could type "stty 9600 /dev/tty4"; cat file >/dev/tty4" and it wouldn't work because when stty exited, the system would reset the terminal baud rate.
The proper way to do this (assuming you weren't the sysadmin and couldn't modify the default per-terminal baud rate), was to type the following
(stty 9600; cat file)>/dev/tty4
There is a nice tutorial that walks through how one might write it from scratch: https://viewsourcecode.org/snaptoken/kilo/
Of course the client may be really dumb! Here is a 1930s teletype used as a Linux terminal:
https://youtu.be/2XLZ4Z8LpEE?t=656
If you would not do it that way, you would run into all kinds of synchronicity problems. How could you be sure that, after rapidly making 50 keystrokes, every keystroke was received by the other end, in that order?
Exactly. I implemented a client that emulated a VT100 early in my career, and this is a real problem. There are various strategies you can use but by far the simplest and safest seemed to just be the echo and for the client to always display exactly what it receives[1].
There's nothing worse than typing out a command that you realize is wrong and potentially destructive, only to Ctrl+U it and have the client kill the line but the server didn't get the instruction, so when you press enter it runs the evil command. If the command doesn't echo anything you may not even know! I once accidented a space in the path I was deleting when (recklessly with -rf) trying to remove my ~/bin directory, like this:
rm -rf ~ /bin
Good Lord that was a bad day. Thankfully I still had the installation disc to restore /bin, and a relatively recent backup of my home directory to restore that. I lost a few days of uncommitted code, but that felt like a trifle compared to what it could have been :-)[1]: I love how mosh[2] handles this to get the best of both worlds. It will smartly show you what you typed, but sill underline it until it actually receives the echo from the server, so you can type and feel like there's no delay between bytes, but still be confident that client state matches the server state.
[2]: https://mosh.org/
https://programesecure.com/ascii-values-table-generator-in-c...
Unfortunately if you just see the character codes with the decimal, hex, and octal next to them this logic is obscured. Remember, it had to be implementable in (mechanical!) hardware.
Could you elaborate on what does the bit pattern reveal?
For example, I understand that Ctrl-C generates 0x30, which stands for ETX (end of transmission), but what is it there in 011?
Older terminals (like the VT05) used bit-pairing also with shift. They just flipped some bits depending on the upper two bits. Compare column 2 with 3 in the linked graphic, and 4,5 with 6,7.
#define CTRL(x) ((x) ^ 0100)
// e.g. 'C' ^ 0100 is 3
// e.g. '@' ^ 0100 is 0
// e.g. 'M' ^ 0100 is 13 a.k.a. \r a.k.a. enter
I like this explanation better. It also explains what notation like ^C means. It's shorthand for 0100^C.Crtl-C is shorthand for 0100 xor ‘C’
Also, the 0 prefix means this is a base-8 (octal) number
eh, the article linked does a very poor job of an explanation of ASCII
And no wonder, it's a deep subject - if you want answers the you want to watch this talk: https://m.youtube.com/watch?v=_mZBa3sqTrI
Maybe I should give that a go in some classic Tracy Kidder style. I certainly can't fill in all those steps. I'd have to do some learning myself
This is characteristic of one of the common interview traps I fall in. I don't know who my audience is or what kind of answer they're looking for.
I can answer the software world version of that all day along with internationalization and the history. But hardware? No not really
There's also a whole lot that happens when you press a key to generate said input (and I don't even mean at the hardware level), which is perhaps much less known.
So until I fix that no "emacs -nw" (emacs in terminal mode) for me as I rely way too much on my Hyper key.
Sending "F18 a" with QMK or KMonad (a random prefix-combo I picked for example purposes) instead of "H-whatever" (for whatever keystroke combo you have H-whatever bound) would work with a terminal in your case, but you'd have to change all those bindings and setup QMK/KMonad accordingly. That's altogether too much work.
If this is incorrect, (whether caused by line noise or whatever), then the user knows this immediately. Unix has a lot of 2-character commands - you wouldn't want an innocent command accidentally mistranslated to "rm" and then press <Return>.
2004l -> “Turn off bracketed paste mode”
What is the purpose of caret notation then? Is it just for human readability? e.g. my terminal shows ^H sometimes when I press backspace
"0x08" is "^H". H is the eighth letter of the alphabet. There is no difference between those except for the notation. In the ASCII character set, the first 32 characters are called control characters. This is why many keyboards have ^ on their control keys. The 26 control codes 1 through 0x1A correspond to ^A through ^Z.
PS This relates to the 7-bit ASCII character set, American Standard Code for Information Interchange, composed of control characters for communication control, letters, numbers, and punctuation.
(about 1981 or so)
They didn't last long, they were quickly replaced by PCs on a lan so we could play Doom.
Although if you asked one of my 900 coworkers in 1981 who wrote flight simulation software in Fortran, how the VT100 terminals interacted with the computers they were connected to,only a few dozen would know.
One of my projects at the time was to create a library to support the creation of 'screen oriented' applications without knowing the escape sequences.
But she blogs eloquently from the standpoint of someone learning things from first principles, in an accessible, actionable way. As a self-taught person this is near and dear to me. She’s like the 3Blue1Brown of systems programming, and there’s not much higher praise.
Striking the balance between rigor and practical applicability is tough, especially while assuming little prior knowledge.
Keep up the good work @jvns!
https://jvns.ca/ is a treasure trove
It’s an awesome company, and they’ve got solid folks, but “Staff” at Stripe isn’t what makes Julia cool: Julia’s work with Recurse is way cooler.
I mean we’ve got Consul: https://stripe.com/blog/service-discovery-at-stripe
And then etcd: https://stripe.com/blog/operating-kubernetes
Both courtesy of Julia incidentally.
Chubby has worked since like 2003. We’re just talking a different level of ball game.
IIRC telnet instead uses urgent TCP packets to indicate SIGINT.
Your setup fails to distinguish keyboard interrupts (intended for the remote machine) and real SIGINTs generated by kill(1). It also uses the local termios(4) settings instead of the remote ones.
Hard to say without looking at the actual code, and right now I am not particularly in a mood for reading C sources at the moment. Maybe someone else is and will tell us the true story!
P.S. Now that I think of it, ssh implementations have to "sync" local and remote tty parameters or at least make it look sane for the user: if you resize your local xterm, arguably the remote e.g. vi should get notified, but what if it's the remote process that changes the terminal dimensions, should your local xterm get resized as a result?
Hadn’t thought about this before, but I think only window size needs to be synced (maybe baud rate and parity? I really have no idea how those would work)
> if you resize your local xterm, arguably the remote e.g. vi should get notified, but what if it's the remote process that changes the terminal dimensions, should your local xterm get resized as a result?
Huh, TIL processes other than the pty master (the terminal emulator usually) can change window size. Glad I checked my sources before writing off a comment …
TTYs or teletypewriters have been in use since the 19th century. I'd love to see a blog post that talks more about the early history.
For example, the ls command example from the article:
sent: "l" recv: "l" sent: "s" recv: "s" sent: "\r" recv: "\r\n\x1b[?2004l\r" recv: "file\r\n" recv: "\x1b[?2004hbork@kiwi:/play$ "
The interpretation and processing of "\r" to return the final output is actually bash processing this, not the terminal?
So what's being sent from the terminal to bash here is "ls" (which is echoed back) and then the return/enter key, which bash interprets as "run the command".
So it sends "\r\n" to the terminal (this is "recv" in that notation), which moves the cursor to the beginning of the line and then to a new line to get the cursor off of the prompt line, and then "\x1b[?2004l", which is the sequence to turn off bracketed paste.
Then ls runs and prints "file\r\n", which is the filename "file" on its own line.
Then bash takes over again, reenables bracketed paste and prints the prompt. Notably it does not move the cursor to get the prompt on its own line, so when the command didn't end in a newline the prompt hangs in a weird spot - try `printf '%s' foobar`, it'll show your prompt like "foobarbork@kiwi:/play$". There are tricks to get around this.
CR - carriage return - escaped as \r - move the carriage to the beginning of the line (the "carriage" is the print head of a line printer, think an old dot-matrix or a typewriter)
LF - line feed - escaped as \n - advance the paper one line.
Since all on-screen terminals are "virtual", these are translated to cursor movements. But their origin is in paper output.
If you've ever
seen text that
gets printed like this
...it's because the \n LF line separators in the output aren't being translated to terminal instructions, just dumped raw.MS decided that both should be kept in text files; Unix-ish dropped the carriage return to use \n; MacOS before OSX used only \r.
After we press the return/enter key to tell bash to "run the command", is bash doing everything from here (ls is not part of this bash part right?) including switching off the bracketed paste and re-enabling it?
And yes, then bash starts ls, which is an external program. It might be /usr/bin/ls.
And then ls quits, and bash re-enables bracketed paste because the command might have not enabled it or enabled it and disabled it before quitting. So you get this weird bracketed paste sandwich.
In general, “smart” programs like bash or vi or fzf or readline configure the termios state so that the tty driver doesn’t handle any keys. This gives them more control. When they exit, they restore the termios to the original state, so that you can still backspace in dumb programs like cat and grep.
So you might have a dance like this when you run vi:
- bash restores its saved termios, so that whatever program it’s running starts with a blank slate
- vi saves the original termios
- vi switches the termios into “raw mode” (simplification)
- you edit text …
- vi switches back to the state it saved
- vi exits
- bash saves the termios state
- bash switches to raw mode
No typeahead while a previous command is running, like you can do in UNIX et al