Hacker News Reader

Comment by vidarh | Hacker News Reader

I use my own terminal, and early made the decision not to give a shit about terminfo, pretend to be rxvt, and deal with the fallout (now, admittedly, the only user I care about for my terminal is me - at least for now). Modern terminals supports enough of a shared subset that it's less painful doing that than having to deal with applications get all confused because they don't recognize your TERM variable, to the point where I've typically found the closest match whenever I've used a less common terminal in the past too. (This matters less if you don't regularly ssh into new systems, of course)

View on HN

Absolutely agreed. The main problem with that is providing new features that xterm doesn't support, but terminfo pretty much refuses to add new features anyway, so modern terminals have had to work around that with some combination of autodetection, separate environment variables, or YOLO approaches. So, at this point, I'm not sure it's worth supporting terminfo at all for most applications, either as a terminal emulator or as a CLI program. (The article makes a similar point about the negative value terminfo is providing for modern applications.)

If you're building a program that takes over the terminal screen, you probably should use one of the established libraries that abstracts over terminfo and over old non-ANSI Windows consoles. But if you're doing light terminal handling in an application that doesn't take over the screen, such as emitting colors or doing simple cursor control, forget about terminfo and just handle two or three cases: ANSI, optionally old pre-ANSI Windows consoles, and files/terminals with no support for anything. Rounding ancient terminals or terminal emulators to "no support" and giving them the same thing you'd give a redirection to a file seems entirely reasonable for a new program.

Just... please please please call isatty() and don't barf ANSI cruft into my buffers.

I'd frankly far prefer people not to call isatty(), and require switches for machine-readable output. The kind of tools that "barf ANSI cruft" are usually the tools I want to do so even when I'm piping their output into something else. It's a real nuisance to e.g. deal with options to preserve colours through a pipe for every element. There might well be exceptions where I'd prefer to default to checking isatty(), but they are few.

If there was any remotely standard way of signaling the preferred default, and a remotely standard command line option to toggle that default, it'd make things a lot nicer, because I totally understand where you're coming from; I get equally annoyed just in the opposite scenarios, so the situation is a nuisance for both of us.

I'd love to have a standard command-line tool for "run the following program with its output set to a pty, and then pipe the output to stdout". That's the most reliable way to get programs to display color and similar.

Any tool whose output is ever parsed, including "parsing" as simple as "tool | grep xyz", shouldn't emit terminal escapes to a pipe by default. If `xyz` has an embedded color sequence in it, that grep will fail. Or worse, produce unexpected results. (The standard color sequences end in `m`; a grep for 'msomething' could match 'something' preceded by a color sequence.)

I agree, of sorts. The problem being that we have developed a culture of making user-facing user interface and APIs the same in Unix-y tools. I think "everyone" recognises the problem, but we may have different preferences about what the default should be. Personally, I'd prefer a default way of indicating "I want API behaviour now" but defaulting to more human-friendly output, because even in most cases like "tool | grep xyz" from the command line I'm still more likely to want colours etc. to stay intact through the pipeline. Most of the time that's fine, but you're of course right it will give unexpected results some time.

I think my ideal expected behaviour (which would still not be perfect) would be something like:

* Tools defaulting to unescaped output in scripts, but with a standard short option and/or ENV var to trigger colour output.

* Tools default to colour/escaped output when run in an interactive shell even if in a pipe.

* Tools being escape sequence aware, maybe with switches to turn that behaviour off if you genuinely e.g. do want to grep for sequences that may include escapes and you want them considered.

But I'm not sure there is a good solution to this other than decoupling UI and API and having different defaults for tools that are expected to be "user facing" vs treated as API. I have an "ls" replacement on my system, for example, which changes formatting and adds more colour to my ls output, and it's obviously not named ls because the amount of stuff that breaks if "ls" isn't reliably the same as always is significant. It's still not great to have to separate this given that part of the ease of composing pipelines etc. is familiarity with the output, but maybe if coupling that with reasonably standard switches to turn on/off machine-friendly output.

I can imagine that a world like that could work (with some care to make it less error-prone for common command-line scripting, since command lines can sometimes be small scripts and evolve into large scripts). But I think it's too late to do that for CLI tools without causing widespread breakage.

I think we might be able to do better in something that isn't a traditional shell, and that uses ptys instead of pipes, together with builtins that replace standard UNIX tools with escape-aware tools.

Yeah, I think it at least means abandoning any idea of changing the API of any of the standard tools. Whether you silently replace them in a custom shell or provide alternative named replacement.

I'm halfway tempted to replace my shell with one that is more integrated with my terminal and do something like the last bit you suggested, given it can be very trivial[1] if you explicitly make the choice that for any scripting you'll use a "regular" shell.

[1] there is, in fact, a tiny single-file Ruby shell that I might be tempted to extend.

You could write a wrapper around something like https://github.com/cvolny/faketty, which can manipulate isatty() calls when used with LD_PRELOAD on Linux (and presumably DYLD_INSERT_LIBRARIES on macOS except on Apple's binaries).

Why use an LD_PRELOAD wrapper rather than running the program with a real pty?

Yes, absolutely.

Also, as a pet peeve: always read your input from stdin, and if stdin isn't readable, do not assume that if stdout/stderr is a tty you're allowed to use that for input. (This assumption is broken in batch/CI/etc systems where stdout/stderr may be a tty so that a program emits color/etc, while making stdin /dev/null because there's no user interaction possible.)

TTY queries are written to stdout but read from stdin. That's not user interaction. E.g. if you're system doesn't have ioctl for window size (or you're over a remote serial etc), setting the cursor to the bottom right and asking it's position. Those programs break with no stdin because a tty is inherently bidirectional communication!

Such queries are not universally supported, and a program using them has to be prepared to expect a lack of response to the query. (This is one of many issues with such queries.)

> pretend to be rxvt, and deal with the fallout (now, admittedly, the only user I care about for my terminal is me - at least for now

It's fine when it's just you. Unfortunately, there are some widely used terminals that default to claiming to be xterm, but are not xterm compatible.

Frankly, I find most work well enough with xterm or rxvt termcaps to prefer that over having apps just refuse to even try to work until you've changed settings.

But conversely, if distributed like that, I'd also feel that this implicitly means any failure to act like xterm is a bug they've implicitly suggested it is reasonable to report or complain about (and if the version number doesn't imply it's an early stage release, and it still doesn't work well with its defaults, I'd get cranky)

If you send feature-detection queries it doesn't matter.

The nasty part is that RXVT violates ISO 2022 structure in weird ways. It's not alone in that, but most of the other-program violations are much more easily fixed.

When I was on ITS, I just implemented SUPDUP in my terminal. It made Emacs usable at 300 baud. That protocol is a nice piece of engineering.

I got a big dose of Telecoms standards early in my career, including some X.25 and networking stuff. After the clarity and simplicity of SUPDUP, my reaction to Big Standards could best be described as "allergic".

[Marshall Rose and Michael Padlipski are good reading on this subject. To this day I use the word "octet" as an epithet]

use standards

not too many

mostly plain ones

That's fascinating! Did you have a specific itch you wanted to scratch by writing your own terminal? Or was it a learning project that you found useful?

A combination of a procrastination project, wanting a pure Ruby library I can use for output from hybrid applications (e.g. my editor) to be able to have a mostly terminal UI but being able to add in limited graphics when available, and wanting a testbed for other terminal changes I want to experiment with in terms of creating more of a hybrid UI.

A basic terminal can be fairly small - mine is ~1800 lines of Ruby at the moment, which I consider disappointingly large. For comparison st is ~8k lines, I think, and xterm is ~88k. But you can do a working terminal in far less than my 1800, even in more verbose languages, so it's a nice space to play in where you can get something that works in very little (set TERM to a dumber terminal than rxvt, dump any escape sequences your terminal doesn't yet understand to a log, and run the apps you need, then iterate...), and build up in whichever direction you want to something quite usable very quickly. Then you can spend a lifetime polishing quirky little details nobody with you will ever care about... ;)

I’d like to know more of the motivations for building your own terminal. Are you using a library or written all from scratch?

See a lengthier reply elsewhere in this thread, re: motivations etc. but it's almost all from scratch. Frankly, a simple terminal is not a huge amount of work, and you can get something working "well enough" to start with with very little code.

The huge amount of work comes from nailing all the quirks you will have to deal with if you let a bunch of other users loose on it, with their expectations of running all kinds of applications I've never tested that does weird stuff.

Yeah, the last 10%. I saw your reply after posting mine, thanks.

Famously the last 90% of the project :(

Reply on news.ycombinator.com