a library that I can feed data directly into (without a pty)
LGPL, MIT, or BSD
able to quickly serialize its internal state (make me a blob)
able to restore state if I feed it a blob of saved state, even across different software builds
portable to recent versions of Linux and Windows
Moreover, the session state saving is not possible either since you need to save the internal state of running programs as well as their cached output (saving the output is really very easy but it's the least of your worries given the specifications you've described).
You might also find this problem is better solved with a $SHELL replacement rather than a terminal emulator.
So the emulator gets bytes from the emulated virtual machine, and then what? This needs to get fed into a terminal. While it is possible to invoke xterm with the very buggy -S option, there is no way for the emulator to suck the state back out of xterm for coherent snapshots. Simics uses xterm with the -S option. It sucks.
A library interface is required. The whole point of a library interface is to allow session state saving. Using a $SHELL replacement is way off. That has nothing to do with the problem.
So, if you were hacking on an emulator (qemu, MAME, Simics, VirtualBox, etc.) and you were trying to emulate a device with a serial port or even a modem, how would you get snapshots to contain the terminal state? To make this work, you need that state within the emulator. Passing stuff over a PTY is not going to work.
You can then also forward any output to a file/terminal emulator/whatever is convenient, but relying on the terminal emulator to save part of your emulator's state seems weird to me, unless you are going to ship the two bundled together.
The point of a library is that the two are more than just bundled together. They are linked together, making one executable. The terminal window is a part of the emulator.
The only thing that wouldn't be inlined via control characters nor escape sequences would be the PTY operating mode (which are defined via kernal syscalls). Im talking about stuff like local echo, flow control, how CR/LF characters are handled, etc. However that's solved with a multiplexer (screen or tmux type thing) or a custom shell, thus you could save that running state and reset it when needed (shells actually already need to do this to some extent so they can pass control of the TTY to forked processes. But we'd obviously need to take things a step further and store any state changes made by forked processes as well). However now we are back to my original solution you dismissed.
The emulator needs to pop up 2 terminal windows so that we can log in to the BBS. We do that, and a colorful menu is displayed on each of them. At this point, we save a snapshot and shut down the host computer. Later, we start the host computer, then start the emulator, and then try to resume the snapshot. Each terminal window remains blank, which is not the correct state. The guest OS (a DOS BBS) is in the state where it is showing a colorful menu, but we can't see it at all. Our terminals, being separate from the emulator and without an ability to save/restore snapshots, are inconsistent with the emulated guest OS.
In the above example, the guest OS (a DOS BBS) never did any kernel syscalls related to PTY operating mode. It directly acted on serial port hardware. There is no PTY, and adding one (where?) doesn't help.
Fundamentally, the emulator needs to be able to save/restore all terminal state. That even includes stuff like incomplete escape sequences, such as when just the first byte of the escape sequence has been passed to the terminal.
I don't know why you even mentioned PTYs (now I understand your requirements). A PTY is just a detachable interface for local terminals, but since you're connecting via serial anyway there's no need for a PTY. There's also no need for a special emulated serial interface like you suggested because the DOS VM would have one already baked into it, just use that and have the BBS listen on a couple of COM ports.
Regarding storing the previous session, there are already terminal emulators out there that will restore the scroll back session from a previous instance, so you might find your idlogical terminal already exists.
It still seems like a daft problem you're trying to solve though. Putting aside you're suspending a server for no apparent reason (why would you even want to do this?), you're effectively running a forum for one local user. But I'm sure you also have your own reasons for these anti-patterns as well.
I have implemented (wrote C code for) what you refer to as "a special emulated serial interface" and thus "the DOS VM would have one already baked into it". That is my code and it works fine, and I do indeed "have the BBS listen on a couple of COM ports".
I mentioned PTYs because I don't want them. Most terminal emulators will only accept data from a PTY, with themselves on the master side and a shell on the slave side. This alone makes these terminal emulators essentially worthless; to use them I'd really need to have my emulator act as a telnet server and then ask the user to telnet into the server! Perhaps a really screwy $SHELL could perform that automatically, but it still doesn't solve the replay/snapshot issue.
An xterm is very special because xterm supports the -S option. With this, my emulator can fork off a process and then exec an xterm with a file descriptor that I can feed data into. This works OK until I want to load a snapshot.
There may be "terminal emulators out there that will restore the scroll back session from a previous instance", but this gets unusable. My emulator supports multiple named snapshots. Using a terminal program with distinct snapshot functionality would require that the user save/restore snapshots both in my emulator and in each of the terminal programs. I have a case (other than a DOS BBS) where my emulator has dozens of serial ports, so this would place a heavy burden on the user.
Suspending a server is good for debugging it. I can go back to before a crash. I can load a snapshot to bypass slow and annoying start-up.
The reason I commented about hacking the BBS software is because that's the only way to solve the weird requirements you're asking. You can complain about my suggestion all you like but it's your unusual requirements that are at fault The fault here, not my solution.
Maybe you need to stop and think for a moment that perhaps what you're asking is silly rather than just assuming that everyone else on the internet is stupid.
The closest work around you're going to get (and is buildable with your experience) would be to build a terminal server (like a minimal Linux distribution running on a VM or Raspberry Pi) which you can SSH onto from Linux or Windows (eg via PuTTY) and which has a serial interface to your BBS software (be that virtualized or physical). Install 'tmux' or 'screen' onto the terminal server and when you SSH onto the terminal server reattach to your screen session. That screen session will be connected to your BBS and you'll then have your detachable but persistent terminal sessions.
Edit: If your terminal server is a VM and the BBS software is a DOSbox instance running on the terminal server then you also only need to snapshot one host and have persistent scroll back PLUS the ability to restore from random snapshots. The only thing you would lose is your TCP/IP handshake to the xterm / PuTTY session but they're detachable anyway via tmux / screen so you'd just have to SSH back into the host and reattach to your multiplexer (screen / tmux).
I do restore snapshots in random orders, and I really want my terminal to know exactly where to resume. I don't expect the terminal to guess. I expect it to accept a blob of serialized state data, and of course I also need it to produce that sort of data on demand. I want that data in the snapshot files that my emulator creates.
I could indeed put the emulator, along with separate terminals (xterm) and the OS they require, within an emulator. This works, but adds the overhead of a whole extra OS. It gets confusing to use, takes up lots of memory, runs slowly, and takes up extra space on the screen.
Honestly, just stick this stuff behind a terminal multiplexer and move on with your life. You're trying to over-engineer a solution to a problem you're inventing.
The state of an entire OS, with all running programs, gets saved. It's more complete than when a laptop/notebook does suspend-to-disk. The emulator saves the CPU registers, the RAM, the disks, the parallel port mode, etc.
The trouble is that a connected terminal is not included in that list.
You keep trying to reinvent terminals with features that already exist to solve a problem that needs to be fixed in the server itself.
The serial port is an emulated 16550 chip. It is much more than a PTY. It has clock divider registers, an interrupt cause register, a transmit holding register, etc.
Scroll back buffer history won't let me untype something. If I take a snapshot, then a character is output to the terminal, and then I load that snapshot, the character needs to disappear from the screen. Of course, "character" could be something more destructive, like a scrolling operation or the clearing of a line.
If you disable local echo and have the server side software handle that (like how the SSH protocol works) then it becomes even easier because you then also manage the terminal state (to some degree) via the server as well.
I've written quite a bit of software around this kind of area and I really doesn't sound like a problem that hasn't already largely been solved. I mean if you're feeling really lazy you could just build a terminal server that supports detachable sessions (eg tmux or screen) and that would get around the scroll back issues without you having to write a single line of code.
Like I said, I've done a lot of work with serial interfaces much more archaic than what you've described and have been able to get them working without having to write too much magic. So I suspect you've over thought your problem because it still seems really straightforward to solve from this latest description. Like I said before, terminals literally are just flows of text (ironically there's even less complexity there since you're dealing with serial interfaces rather than PTYs) so all you actually need to do is ensure your terminals output is persistent - which is actually a surprisingly easy job.
I emulate lots of different things. I'll make up an extra-simple case to illustrate the problem.
The guest OS running in the emulator is just a bootable floppy image for testing. All it does is print a counter to the serial port, once per hour. Just after 3 hours it has written 0, 1, 2, 3. At this point I make a snapshot file named 0123.snap and let the software continue running. More numbers get printed. Eventually the terminal is showing 0, 1, 2, 3, 4, 5, 6. I decide that I wish to go back in time, so I issue a command to load the 0123.snap file into the emulator. The internal state of the emulator warps back to the point at which I took the snapshot. The CPU registers, the RAM, and everything else within the emulator are now as they were earlier. The terminal retains the old state however, because it is a separate program with no awareness of the fact that I just loaded the 0123.snap file into my emulator. The guest OS carries on from the original point of course, since that is what the CPU and RAM state dictates, so a "4" will be printed next. The terminal then shows 0, 1, 2, 3, 4, 5, 6, 4. The numbers are simply wrong. They should be 0, 1, 2, 3, 4.
I can only avoid this fate by saving full terminal state whenever I save the emulator state, and of course restoring it at the same time too. There is no reasonable way to extract terminal state from a separate terminal program. (attacking it with ptrace is not reasonable) I thus conclude that the terminal must be built into the emulator.
It's called a "terminal multiplexer" and that is exactly what I suggested with the tmux solution right from the bloody start. I've lost track of how many times I've said this needed to be solved on the server yet you repeatedly pushed back on both points when they were made, insisting it was a terminal emulator issue on the client side.
I don't see tmux being available as a library that I could link into my emulator. If it can be a library, then yes my emulator could implement the tmux protocol. This turns out to be almost exactly what I was proposing, with the terminal state implemented by a library within the emulator. I'd just be missing the user-friendly aspect of automatically popping up the terminal windows when my emulator starts, but perhaps that could be arranged by having the emulator start xterm running tmux.
This is comparable to using vnc protocol for VGA. I have this implemented, but nobody likes to use it. Everybody prefers the built-in video window to show the guest's VGA.
If you meant to not link in the tmux code though, that won't work.
Instead of insisting that there is a terminal emulator issue on the client side, I insist that there not be a distinct client. The code that tracks terminal state (cursor position, current character attributes, etc.) pretty much needs to live in the emulator. That is the only reasonable way to ensure that the state can be saved and restored by the emulator. Actually displaying this state could be done by connecting a client (annoying) or just by popping up a window.
Another nice thing about having the terminal windows built into the emulator is that they all go away when the emulator does, even if that is a crash.
It is typical for emulator authors to just give up, letting the terminal be inconsistent after loading a snapshot.
So far I've done just that... but my users love loading snapshots.
It is useful for virtual machine snapshots. You can restore the virtual machine with the terminal state intact.
It isn't true if the VM is communicating over an emulated serial port. In that case, the VM software must provide a terminal as hardware. Typically the VM software only provides a serial port to the VM, which the user will then associate with something else via the VM configuration. This fails when loading a snapshot because the terminal isn't part of the snapshot.
For example, suppose that the VM is a PC-AT running DR-DOS and a BBS. (from the pre-internet days, with a modem for users to dial in on) At minimum, the VM software must provide a serial port for the VM. You could configure this to connect with a real physical modem and then connect to that with another modem and some dial-up software, but then you'd have a problem with snapshots. Loading a snapshot would place the BBS in a state that is inconsistent with the dial-up software. What you really need is for the terminal emulator to be part of the hardware that the VM is emulating. It would then be properly snapshotted.