For those like me with the opposite inclinations, try dtach. It is only a session handler. It does not pretend to be a terminal, or do splits or tabs or anything but I/O. That's left to your actual terminal.
I mean, you could avoid knowing anything about the terminal state machine if you dumped the entirety of the session’s bytestream from the very beginning on each attach event, but unless you are willing to do that it seems to me that you’re going to have to track enough state to basically amount to maintaining a screen buffer (and then you do have to care about sixels).
For example, suppose the user fills the screen (maybe even with an image), then spends a lot of bytes overwriting the last line over and over (think progress bar). Either you’re tracking the first byte to have affected the current screen (and need to store the entirety of the “a lot” and dump it onto a newly attached client) or you’re trying to discard all the stale updates while keeping the last one and the initial screen contents (and that’s just a screen buffer with extra steps). (Incidentally, I suspect Muratori’s Refterm fails this test to the point of requiring a redesign, though I haven’t checked.)
I use tmux exactly to split panels in any way I want. My terminal (iTerm2) doesn't give me the full power to manipulate windows/panels using shortcuts.
But unfortunately I'm gonna have to stop using tmux because it's not compatible with sixel. I spent a lot of time ricing it tho...
(I used to use tmux, but ultimately found it more trouble than it was worth.)
Which tmux pane management features do you miss in iTerm2?