Also, the fact that I can type "cat binary-file" and mess up my terminal settings tells me that they didn't clearly separated concerns.
Yes, concerns aren't separated ;)
A much more recent and far superior design can be found in Windows NT, where things like coloring are done entirely through an ioctl-equivalent (whereas Unix has amounts of in-band signaling, while some later extension use tty_ioctl).
> Unix has amounts of in-band signaling
Their tech beats OSS in many areas and there is definitely a reason that people continually pay to use it.
You can't separate concerns that you can't clearly define.
What cat does is what you tell it to do. If you need escaping -- well, do it (base64 or whatever).
It'd be a failure to separate concerns if the shell detect and treated cat output differently.
As an example on how to do it right, consider how commands like "git diff" determine whether to paginate/colorize their output: they call the isatty(fd) stdlib function on stdout, which uses ioctl TIOCGETA to get terminal properties. Iff that fails, there's no TTY on the other end of the fd and "git diff" just writes the raw output.
If only there even was a clean, consistently implemented, well-documented protocol... but there is not - few tools properly escape their output, and you can't get much right without a terminfo database, even with you can't get consistent results, and the protocol is a mess.
> out of band, e.g. via ioctls
Out of band? Do you suddenly double everyone's wiring costs? That won't fly. How about multiplexing control and data streams onto the same channel? Yeah, that's exactly what ISO 6429 / ISO 2022 / ASCII does.
If you want to express terminal control using function calls rather than inserting raw multiplexed control sequences into text — Yeah, that's a good idea… which Unix has had since 1978.
Those who don't understand Unix are doomed to reinvent it, poorly.
You should try to implement it the way you think it should be. The problem you will notice: "content" and "presentation" can't be easily separated because it's important when changes in presentation take effect. Also one man's presentation is the next man's content. They are just not independent streams, period.
It's actually great that you can capture streams including control characters and replay them later. It works pretty well, especially these days where there aren't many competing instruction sets anymore.
Even in HTML you need to tag elements with class names inline. (And the HTML approach could never work for general purpose terminals -- as opposed to crippled, specialized ones, like Mathematica / Jupyter notebooks).
--
> If only there even was a clean, consistently implemented, well-documented protocol... but there is not - few tools properly escape their output, and you can't get much right without a terminfo database, even with you can't get consistent results, and the protocol is a mess.
You can go GUI. But many of the programs you want to use aren't available for GUI, guess why...
isatty() is a hack, but it's convenient and has very low false positives. If you filter output into another program before sending it to the terminal, the hack falls flat. You then need to provide a means for the user to be explicit, like `git diff --color=always`.
Let's avoid hollow arguments to authority that preclude any interesting criticism or discussion. Unix has its fair share of flaws.
> it's important when changes in presentation take effect. [...] They are just not independent streams, period.
Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent.
ioctls could very well apply to a specific position in the data stream instead of being immediate. And if you don't like ioctls, there could well be a third output fd instead. In that case, the synchronization problem could be solved with a "barrier" feature.
Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress.
It's not an argument. It's just a category for the rest of the comment.
> Not independent, but separate. Your argument would also apply to stdout and stderr - these are separate even though not independent.
No - technically they are completely independent. There is no ordering or interleaving or association defined.
Actually stdin/out/err are nothing but conventions introduced by the shell.
> ioctls could very well apply to a specific position in the data stream instead of being immediate.
The general idea of a stream does not include positions - they don't have a clearly defined beginning or end. Streams are not like files on a filesystem, which do indeed have a size and can be indexed.
Stream positions are another specialist concept that eases development for some apps or environments, but gets in the way of development of general purpose environments.
> Can we at least agree that the actual terminal protocol situation as it is now quite frankly sucks? The poor documentation and completely inconsistent implementations have caused me quite some stress.
Frankly there are not many problems with Unix terminals. They are quite simple. As Unix in general they just won't keep you from shooting yourself in the foot.
Job control is hairy, though, but mostly because it's an ill-defined problem. In particular there was never a good agreed on mechanism for starting processes in independent environments. Systemd might have solved that to some degree (not a systemd fan).
The situation with incompatible instruction sets (escase sequences) has also dramatically become better, partly because there are libraries like ncurses, partly because pretty much everything is VT100 compatible these days, partly because we have GUIs for complex graphical specialist applications.
If you have a concrete problem - get in touch with me and I am happy to help.
But control commands shouldn't be commingled with general input/output. That's how you get security bugs like the recent bash CGI vulnerability. There should be an interface that allows ncurses etc. to use colours, but it should be via a distinct control channel, not "magic" sequences embedded in normal data.
Keep in mind that many of the original tty devices were teleprinters, and the physical actions of the printer head upon being commanded to backspace would involve moving the actual printer head back a character - potentially using an eraser head - and then potentially moving the pointer head back again once more if that advanced the head. It's perhaps complicated an error prone, but it's also the optimal movement of the printer head.
You do not want to bake "complicated and error prone" into the peripherals of the time. You had no firmware. I'm not sure if you even had a proper ROM, necessarily. So you do it on the software side.
There's no abstraction here per se - but a 1:1 mapping of commands to common sets of physical hardware actions. That's not even a bad thing - what exactly would the abstraction gain you back then? You might abstract the escape codes to use for a given terminal via a terminfo or termcap database, to support multiple devices, but the core actions were always the same.
Hopefully you're not still using teleprinters in production - but someone is. And the VDUs that replaced teleprinters reused the same control mechanisms - keeps the complexity and thus unit price down, easier to retrofit / make backwards compatible with software expecting teleprinters, etc... I know at least one place still using fairly dumb VDUs in production, and these sequences are the simplest fastest commands you can send them to effectively implement "backspace".
If you ape the same logic in your latest and greatest single-page javascript webpage "app", and you're not writing a terminal emulator, it might be the wrong abstraction. It's sane for ttys though.
> Also, the fact that I can type "cat binary-file" and mess up my terminal settings tells me that they didn't clearly separated concerns.
It's a gun with the potential to be a footgun. Benefits, drawbacks.
Writing a shell script that wants to read passwords? cat a file which disables input echoing. After invoking readline, cat a different file to re-enable it. "Feature".
Want to cobble together a quick script to use your laser printer to print barcodes? Create a binary file that creates the barcode you want, cat it, pipe that through sed a few times to replace the number, and then pipe straight to the appropriate /dev/ttyXY. I've actually seen this, fixed this, and updated this. Figuring out the correct keyboard inputs to give vi was 'fun'. "Feature".
If you don't want to mess up your terminal, "view" or "vi" is likely the correct command - not cat.
Sending control characters to the tty won't do that. Input echo is done in the kernel and you need an ioctl to disable it.
Several terminals have support for, and escape codes to manage, local input echo:
http://www.vt100.net/docs/vt102-ug/chapter5.html#S5.5.2.13
If the kernel is operating under the assumption that the device is operating under local input echo mode (e.g. the ioctl has already been invoked to disable kernel input echo, as the terminal is assumed to be handling it), sending escape codes to the tty will do that.
EDIT: In case my air quotes around "Feature" weren't enough of a hint, I'm not actually recommending anyone do this.
All of them were, tty is short for teletype after all.