Linux would be faster with a PS/2 keyboard, non-compositing WM, and plain ASCII Athena Xterm with a bitmap font.
https://en.wikipedia.org/wiki/PS/2_port#Legacy_port_status_a...
Is Unicode support in xterm really adding the latency?
But in practice, I'd expect non-ASCII to be associated with more expensive features, which has more of an effect.