> [0] As I mentioned in a recent comment on a thread[1] here:
> "Motivated by the thought that at the current point in time perhaps the 'essence' of a 'terminal' is its linear 'chronological' presentation of input/interaction/output history rather than its use of 'text'."
I somewhat object to this.
If you think of a TTY/PTY as an Abstract Data Type, it's a very specific (and clever) one. It's not just a transcript of bytes that flowed through the wire.
Rather, a TTY/PTY is five distinct but inter-related things:
1. a (passive) character-cell viewport grid, of (at any given time) a fixed size, where each cell holds not only a character as data, but also a set of metadata/annotations attached to it (the particular metadata being dependent on the implementation)
2. an active "brush" state for the viewport grid — this includes things like a cursor position, an active foreground and background color, etc.
3. a byte pipe "but better" — in that it's actually a multi-subscriber byte message-queue, plus (only in baked mode) message persistence into an expandable ring buffer, with logical line-oriented consumer-group cursoring to trigger forward-truncation of said buffer
4. a rendering agent with a rendering ruleset (or, in old-school TTY terms, a "line discipline") that sits as one consumer of the byte pipe; reacts to characters in the byte pipe by writing them to the cursor position in the viewport grid; and reacts to in-band messages by manipulating the "brush", making big changes to the viewport grid (e.g. clearing it entirely), and/or swapping out the rendering ruleset itself
5. (the part everyone forgets about) a replay stream-conversion function, that can convert the current state of the viewport grid + its brush, into a stream of characters + in-band messages, such that the rendering agent would parse that stream back into the current viewport + brush state
The clever function you get from this set of components, can be seen in the case of attaching a "client" TTY/PTY to an existing persistent "server" TTY/PTY, such as is created by something like tmux or docker — or even by /bin/login on an old-school terminal server that you've dialed into.
When a client like telnet, ssh, docker-exec, tmux-attach, etc. attaches to an existing backing PTY (either locally or remotely), the client either establishes its own "replica PTY" or reuses the one it's running attached to, and then wants to establish replication from the backing PTY into this replica PTY.
To get a coherent replica, that doesn't just paint garbage at a random position with a random brush, several things need to happen:
• the client needs to reset its own local PTY a known-neutral brush state;
• the client needs to set an ACK position in the byte-stream, and a memorized brush state, for the remote PTY (usually both of these are done implicitly by the syscall on the remote end that opens the PTY for reading);
• the client needs to fetch from the backing PTY, an implementation-neutral byte stream representing the current contents of the character viewport grid of the PTY — and replay that (i.e. it needs to trigger the replay stream-conversion function) — this also happens implicitly, such that when the remote end open(2)s its its PTY, the current state is serialized and dumped into a buffer for read(2)s on the PTY to read;
• the client needs to set up its local terminal's brush to match the memorized one that was set as of its ACK position on the remote.
Once these four things happen, the client can in theory simply shunt the remote PTY's byte-pipe into the local PTY's byte-pipe, select(2)ing and write(2)ing in a loop on a thread; whereupon the local PTY's rendering agent will handle reducing that stream to the visual grid.
In practice, though, the rendering rules / line disciplines / syntaxes of local vs remote PTY aren't guaranteed to match (especially if the local PTY is actually a real physical TTY); and so in practice, clients doing PTY replication like this actually have their own translation function — logic that acts as a hybrid of a rendering agent and a replay stream-conversion function, taking the byte-stream from the remote, reducing it into a model of immediate state-change effects, and then re-serializing those immediate state-change effects as a byte-stream that the local PTY/TTY will understand. (With the first half of this usually being app-specific code, and the second half of this usually being done using some version of libcurses, to know exactly what rendering-rule syntax the local PTY/TTY claims to understand.)
---
Given all that, you could generalize a TTY/PTY to something other than "text". I've often myself considered a PTY where the "lines" are each (a binary, minimal encoding of) HTML body markup; or even where the "lines" are JSON-encoded log events with parent-node-IDs, and the "rendering" is of a default-collapsed hierarchical event history.
But you'd be losing a lot if you didn't bring across the concept of there being a message-stream + a "canvas" + a brush-state for that "canvas" + rules for two-way conversion between the message-stream and "canvas." Without the stateful canvas, you would need an unlimited-size transcript — and potentially hours of replay — to be able to "attach" a client to an existing backing PTY.