The serial port is an emulated 16550 chip. It is much more than a PTY. It has clock divider registers, an interrupt cause register, a transmit holding register, etc.
Scroll back buffer history won't let me untype something. If I take a snapshot, then a character is output to the terminal, and then I load that snapshot, the character needs to disappear from the screen. Of course, "character" could be something more destructive, like a scrolling operation or the clearing of a line.