Hacker News Reader

Comment by jdlyga | Hacker News Reader

I love how so many behaviors we depend on were mistakes or temporary fixes

"In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move"

View on HN

The standard UNIX file system layout exists because hard drives were small at the time and files had to be split across several.

The VI keyboard shortcuts only make logical sense on a specific Tektronix 4014 terminal that was popular at the time: 46 years ago.

The reason the delete key is broken in like half of all UNIX / Linux systems is because TTY is short for “teletypewriter” — literally a remote type writer banging out text on dead trees. The carriage can go back one character at a time and overwrite an error but it can’t shift printed text on paper.

Windows was developed in an era of CRT displays and delete works with it consistently.

This is why it cracks me up whenever someone talks a out “Linux being the standard”…

> The VI keyboard shortcuts only make logical sense on a specific Tektronix 4014 terminal

The Lear Siegler ADM-3A, actually:

https://en.wikipedia.org/wiki/ADM-3A#Legacy

Why would you do this to me I'm going to be in a rabbit hole trying to recreate this machine for the next two years. I've been in the hobby long enough that I only get excited about weird things like this keyboard that have historical significance.

FabGL can emulate an ADM 3A on an ESP32:

http://www.fabglib.org/group___enumerations_ga67edd9a04814c2...

Until very recently windows terminal couldn’t select a wrapped line without adding \r\n in between… And it still messes up default color brightness making grey text white until something else messes it back.

Until recently "cls" and "Clear-Host" in Windows Terminal used to actually clear everything so you couldn't accidentally scroll back and see junk that you wanted to get rid of.

The Windows Terminal team copied the UNIX terminal "standard" of only clearing the current viewport.

Now, scrolling back "works" in the sense that it shows partial(!) output from previous commands.

Say you have a script that produces 47 screens worth of output, and what you want to see is somewhere in the middle. You can't judge this visually because the scrollbar is inaccurate and there may be other history.

Before you could just 'cls', re-run the script after a change, scroll back, and see only the updated output.

Now, if you do this you'll get 46 screens of superficially identical looking output, with 47 of the output you wanted to see appended to it. This is super confusing.

It has confused me. I've seen other admins be confused as well and copy-paste the wrong thing.

The Microsoft Terminal team debated this and decided to do the wrong thing on purpose and copy the "Linux way" to pander to Linux admins on Windows.

Meanwhile if you go back through the old newsgroups and mailing lists, the same argument was had by Linux people as well, with the same logic I just outlined. They also decided to do the wrong thing. Why? Because they were pandering to UNIX admins coming across to Linux.

Why does UNIX do this? Because of an error in the coding. That is all.

That's why the shiny new Windows Terminal is broken now, to copy an error that was already a copy of an error.

ncurses' clear defaults to clearing scrollback and you have to use `-x` to not clear it. I would think most distros use it.

Busybox clear does not clear scrollback nor does it provide an option to do it. But for example on Alpine where Busybox clear is the default, the ncurses package can be installed to get the ncurses clear.

"clear" clears scrollback on my machine. alacritty + zsh on arch linux.

Another thing, is how now we tend to get CLI scripts for what used to be nice Visual Studio wizards.

Microsot teams are tainting the Windows developer experience trying to cater to the crowd that has been buying Apple gear for GNU/Linux development.

And that's good

Not from point of view of Windows devs used to what "Developers, Developers, Developers" meant in practice.

And the worst part is those of us who resemble your remark just download Git Bash et al because Powershell is too verbose anyway.

Because parsing text is so great.

Why does UNIX do this? Because of an error in the coding. That is all.

Are you saying this specifically or metaphorically? Because I suspect that there was a rationale behind that decision.

See also https://man7.org/linux/man-pages/man1/clear.1.html which explicitly states that it clears scrollback iff a terminal has a specified capability. Which windows terminal being of CRT origin should have. I think the issue here is actually an intersection of some terminal nuance and windows team who blindly copied the behavior that some linux guys were used to because this knowledge was buried under their gnome defaults.

Iow, no need to blame unix and linux, blame those who picked one (stupid) mode of many and used that as the only mode of operation.

Which is why it makes more sense for \r\n to be the *nix line ending - because there is a distinct "carriage return (on same line)" from "line feed (at same carriage position)" operations possible, and combining them together makes carriage return and line feed to put the cursor at the start of a new line.

And why it makes more sense for \n to be the common Windows line ending because it doesn't have that TTY ancestry.

That would make sense except you have them the wrong way around.

I have them the right way around, it's everyone else who has them the wrong way around.

Sometimes when theory doesn't match reality, it's reality that's wrong.

I don't see anything that says which is which in that comment. Just what makes sense.

What's even more nuts about Unix 300-layers-of-tty-emulation \n acting like a CRLF is that \r still works just fine as CR. e.g., you can get a nice little clock with:

  while true; do echo -n "  $(date)\r"; sleep 1; done

I need to use printf to interpret the escape character:

    while true; do printf "  $(date)\r"; sleep 1; done

It might also work with "echo -e".

I like my terminals a little bit cleaner:

    printf '\033c';  while true; do echo -en "  $(date)\r"; sleep 1; done

(don't forget "-e" for echo)

  while true; do echo -en "  $(date)\r"; sleep 1; done

Depends on your echo. Zsh's interpretes escape sequences by default unless the "BSD_ECHO" option is used, bash's doesn't unless "xpg_echo" is on.

And xpg_echo is on by default for bash on macOS, so for users on macOS it looks like echo interprets escapes by default.

This is one of the reasons why echo is hopelessly unportable.

See https://unix.stackexchange.com/a/65819/55665

Learn something new every day!

And I wish we had a stronger word than unportable for echo. You see that word tossed around on shell scripting resources where the only system it applies to is a 30 year old version of ksh and it only blows up on a full moon when passed a filename with a literal form feed \f, but echo is a practical portability problem.

> The VI keyboard shortcuts only make logical sense on a specific Tektronix 4014 terminal that was popular at the time: 46 years ago.

vi is more than hjkl, and those four non-mnemonic keybindings make up for it by being extremely easy to press for QWERTY users. The vast majority of other commands are perfectly “reasonable” (easy to memorize and not dependent on any antiquated hardware) — append, backward word, change, delete, end of word, find letter, go to (OK, this one is just an arbitrary prefix), and so forth.

You know vim has been around for like 30 years now right? And the Windows thing has nothing to do with CRT displays. Windows was built around IBM compatible PCs which had standardized keyboards. It was the only keyboard layout Windows needed to support.

> The reason the delete key is broken in like half of all UNIX / Linux systems is because TTY is short for “teletypewriter” — literally a remote type writer banging out text on dead trees. The carriage can go back one character at a time and overwrite an error but it can’t shift printed text on paper.

Which Linux system/terminal/editor are you using where insert mode is not the default?

HTTP "Referer" is the one that makes me sad.

The fact that terminals being an emulation of a teletype comes a close second.

And the origins of /usr/bin a definite third.

A teleprompter? Is that a typo? Did you mean a teletype?

Sorry yes teletype.

Honestly, the typewriter interface is pretty useful. A program can make text appear on the screen by just writing ASCII (or these days, UTF-8) bytes. The only sad part is the design of escape sequences, but that's more a consequence of chaotic evolution rather than the fact that it's emulating a typewriter (and most of the evolution happened with the early software terminal emulators).

Honest question: How would you design a simple text outputting protocol? Would it be radically different to how it's done today, or would it be the same basic ideas but cleaned up a bit?

Hm, I've actually thought about that a lot. I'd probably keep the bytestream format. It is insanely useful. However I might open up multiple streams. Stream one is basic IO. Stream two could be formatting info. Streams above that implementation dependent, but you could probably start defining them for keyboard/mouse/HID IO or audio and video. Then to make it work over the internet just shove it over SSH or TLS.

The one downside that has is if you separate the formatting from the data you can not recreate the session without keeping time info somehow. So, might need a fix for that.

The only other real alternative I'm aware of is moving the formatting into the API "stream" that communicates using the machine ABI. This is a terrible solution. It's what Windows uses.

I think that text output protocol is the wrong way of thinking about it. It should be a rich data stream capable of advanced features like objective stuff. Think ps, but then you’re able to navigate the output and perform additional functions. I think a subset of html would work for this.

I think the biggest danger would be scope creep, but a terminal shouldn’t just be thought of as text.

Unix chose the midpoint between raw unformatted byte input and output (*argv being a raw byte array and stdout being binary or not existing at all and outputs would be configured by whatever was passed into argv) and structured results or a typed-function interface (like COM or CORBA or XML). Either extreme would have been untenable for a cohesive simple operating system, but I do wish Unix had leaned a bit farther toward the typed-interface design. We even had all the ASCII file/group/record/unit control characters.

Think ps, but then you’re able to navigate the output and perform additional functions

We already have gtk, qt, appkit, win32, web based frontends for almost everything, and none of them produced anything remotely interoperable. The main advantage of just text is that you see its structure right there in a terminal and may reason about it without referring to the documentation of ps result-related set of structures with all the possible bells and whistles embedded into a large xml-like document schema (which is of course full of legacy and references to other specs, because compatibility and standardization).

I’ve always been curious though, can VT escape sequences be used to perform user actions? E.g. can you coerce stdout to execute an additional command?

There is a thing called the answerback message: computer sends Ctrl-E (0x05, "ENQ"), and the terminal response with the pre-programmed answerback message. Maybe the message is a shell command, like "rm -rf *" or "chmod a+rwx ."

You can program this message in the setup screen of the terminal. Maybe you can set up all the terminals in the computer center at night...

You can send Ctrl-E to a user with "write" or "wall"..

This answerback message is actually really old, exists even in mechanical ASR-33 teletypes:

https://en.wikipedia.org/wiki/Enquiry_character

It can also be triggered by the mysterious "here is" key on the ADM-3A.

https://dave.cheney.net/2017/08/21/the-here-is-key

I think you can also request the terminal type and at some point you could also request the title of the window in some terminals. Which is why it’s not just annoying but also dangerous to output untrusted characters to the terminal without a filter.

TTY emulators have a long history of being vulnerable to all kinds of nasty things. For example: https://seclists.org/fulldisclosure/2003/Feb/att-341/Termula...

One would hope that we're past the bad times, but at the same time brand new projects like the kitty terminal emulator let stdout read and delete arbitrary files. And verifying something is safe is just hard, these are not even e.g. memory safety bugs, they're "features".

No, kitty does not let stdout out read or delete arbitrary files. First of all stdout is an integer in a lookup table in your kernel, It cant read or delete anything. I assume you mean it lets programs read or delete arbitrary files by writing to and reading from the tty device. And even that is not true. The kitty graphics protocol does let a program delete arbitrary files in the TEMP directory. It does not let anyone either read or list the contents of any part of the filesystem.

> First of all stdout is an integer in a lookup table in your kernel, It cant read or delete anything.

Oh come on, you know what everyone means. "Output with potentially hostile content".

Right now it seems kitty will happily delete e.g. /tmp/.X11-unix/X0. That sounds fun.

https://sw.kovidgoyal.net/kitty/graphics-protocol/#the-trans...

https://github.com/kovidgoyal/kitty/blob/5541e3c2ff441aea319...

Everyone?? No just you. And read the rest of my post. Yes, kitty will delete files in /tmp if asked to. That's what I said. And no unlike your claim, it does not read them. If you cannot tolerate the theoretical possibility of something deleting a file in /tmp ever, then set TEMP before you launch kitty to /tmp/my-mommy-told-me-not-to-put-valuable-things-in-tmp-but-i-didnt-listen

Holy excessive combativeness, Batman!

> And no unlike your claim, it does not read them.

    strace -e open,openat kitty
    printf '\e_Gf=24,s=10,v=20,t=f;%s\e\\' "$(echo -n /etc/passwd|base64)"
    openat(AT_FDCWD, "/etc/passwd", O_RDONLY|O_CLOEXEC) = 14

Are you being deliberately obtuse? Yes obviously when asked to display a file, kitty will read it. But it wont send the file contents on the TTY, so the *program sending the request* cannot read it. Which is what we are discussing here. You are trying to make it seem like the ability of a program to get kitty to read an arbitrary file is a security issue. I suggest you go do the following: strace your editor, then ask it to open /etc/passwd, and then be shocked when you see it doing so in the strace log. Seriously, why am I wasting time replying to you.

I'm trying to make it seem like kitty doesn't have very strong security boundaries, and it's a bit of unknown what it can and cannot be made to do.

Me asking my editor to open /etc/passwd is me asking my editor to do something. My terminal emulator opening files and parsing them and deleting them just because I viewed untrusted content is something different. If you want to make that argument, you need to come up with an example where my editor accesses attacker-controlled files because I opened a random README.md.

Also, please read https://news.ycombinator.com/newsguidelines.html, and shove your attitude.

Ah so classic FUD, "its unknown what it can do". Why dont you spend some effort figuring out what it can actually do instead of making driveby comments with an obvious axe to grind.

You very conveniently left out the fact that pretty much anything that views untrusted content has to parse files. If parsing files is where your "security boundaries" lie, I suggest you drop your computer off at the garbage dump. Hell if your editor has a preview function it too will parse untrusted content. Basically, to do anything software has to parse untrusted content. And just by the way, /etc/passwd is not attacker controlled. If the attacker has access to your filesystem already, she really doesnt need to use kitty to do anything. And "opening" README.md will not cause kitty to parse files, unless by "opening" you mean catting without -v. In which case we are back to your mommy told you to know better but you didn't listen.

At this point its obvious you are deliberately trying to spread FUD. I am done interacting with you. Good bye.

Because security is hard nobody should even try?

The general trick with parsing is to put it in a sandbox that cannot touch files, can only do limited syscalls, etc. This is what e.g. Chrome does.

But yes, it's obvious the Kitty author has no interest in making it more secure ( https://github.com/kovidgoyal/kitty/issues/2084) and it's obviously you take discussions about improving software security as attacks on your person.

Have a good day, sir.

Oh man, this is rich :))) Let me just leave this for posterity. kitty has had a socket for remote control for years: https://sw.kovidgoyal.net/kitty/conf/#opt-kitty.allow_remote... (see socket-only). It can allow remote control from specific windows while disallowing it generally: https://sw.kovidgoyal.net/kitty/launch/#cmdoption-launch-all...

Furthermore the next release of kitty has capability based security for remote control with public key crypto to keep the data safe: https://github.com/kovidgoyal/kitty/discussions/5320

I suggest you find a better place to move the goalposts in your attempts to smear and spread FUD.

> How would you design a simple text outputting protocol?

If I were to design terminal control today, I might have a separate file descriptor for a pipe that is used for control communication instead of sending control codes inline with the text. And I would definitely have a way to query the capabilities of the terminal directly, instead of looking it up in a terminfo database.

> I might have a separate file descriptor for a pipe that is used for control communication instead of sending control codes inline with the text.

How do you keep that synchronized with the text for things like moving the cursor between words?

Having all data in a single steam also makes it easier to buffer messages like a complete log entry with colors and then output that by just writing one buffer to stdout.

At least that one saves a byte on every request. The cumulative savings are in the terabytes by now.

It saves 2TB/year, just on Google searches! (4e6 requests/minute)

It's a big world. I'm sure there's someone offsetting this by keeping a 2TB hard drive just filled with 'r's.

> keeping a 2TB hard drive just filled with 'r's.

r-chive.org

That’s to balance out Traveller.

The thing that irritates me is that people then venerate these accidents. Yes, some accidents are happy; many are at best neutral or least-worst or what have you. And then *ix culture dies in a ditch that we must keep all of them, forever, as though they were the result of a carefully thought out process that produced the best of all possible worlds.

What would you change and how, ignoring the cost of re-standardization?

Plan 9.

plan 9 got the same treatment your username did. Problem solved XD

Rephrase. As in, no thought went into the name?

UNIX pipes. No more plain text, it has to be typed stream of values. And the terminal will by default print details about it e.g "list of records with this or that format".

That’s PowerShell. Everything is an object, not text. But a lot of people hate it because it’s different.

Hm, have to check it out again. Last time around installing in Linux wasn't straightforward, plus I hate Snap. But will revisit.

Humans are the result of billions of happy accidents. It’s not a bad thing to use them as successful experiments.

Another one that circulated recently was that Qualified Immunity (which means police in the USA can effectively commit civil rights violations without suffering repercussions) partly originated in a clerical error: https://twitter.com/ReichlinMelnick/status/15571788326790144...

Nothing more permanent than a temporary solution.

"The fix is temporary, unless it works" -- Red Green

Like Quake bunnyhopping and the SID 6581 being able to fake a 4th voice.

Any links about the SID thing? I found Quake bunnyhopping, but “sid 6581 four voices” turns up nothing relevant.

"Due to imperfect manufacturing technologies of the time and poor separation between the analog and digital parts of the chip, the 6581's output (before the amplifier stage) was always slightly biased from the zero level. Each time the volume register was altered, an audible click was produced. By quickly adjusting the amplifier's gain through the main 4-bit volume register, this bias could be modulated as PCM, resulting in a "virtual" fourth channel allowing 4-bit digital sample playback. The glitch was known and used from an early point on, first by Electronic Speech Systems to produce sampled speech in games such as Impossible Mission (1983, Epyx) and Ghostbusters (1984, Activision). The first instance of samples being used in actual musical compositions was by Martin Galway in Arkanoid (1987, Imagine), although he had copied the idea from an earlier drum synthesizer package called Digidrums. The length of sampled sound playback was limited first by memory and later technique. Kung Fu Fighting (1986), a popular early sample, has a playback length measured in seconds. c64mp3 (2010) and Cubase64 (2010) demonstrate playback lengths measured in minutes. Also, it was hugely CPU intensive - one had to output the samples very fast (in comparison to the speed of the 6510 CPU)."

-- from Wikipedia: https://en.wikipedia.org/wiki/MOS_Technology_6581

The Garden of Eden was transitory.

Every omnipotent creator implicitly grasps that initial configuration matters as much as the automaton rules.

Messages from the Unseen World

The Universe is the interior of the Light Cone of the Creation

Science is a Differential Equation. Religion is a Boundary Condition

- Alan Turing

Reply on news.ycombinator.com