It’s almost always more usable to run a desktop session in Xvnc on the server and connect to it with a VNC client, because if you get disconnected, you can just reconnect.
This isn't very true. The X protocol is very async and lets you batch plenty of things when a response is required.
1. TCP streams require stalling when packets are dropped to keep the stream in order
2. X _requires_ by design that commands are performed in order
Which means that using something that can do UDP and manages it's own sequence ordering can do significantly better. This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years ago and you still can't with X protocol over the network.
Breaking up the screen into small 16x16 chunks or so, encoding on the GPU, and shipping that turns out to be significantly faster.
Especially when you take into account that virtually _nothing_ draws with X using X drawing primitives. It's almost all using Xshm for anything non-trivial.
> Which means that using something that can do UDP and manages it's own sequence ordering can do significantly better. This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years
No it isn't. It's because those things actually compress the video, and X-forwarding generally doesn't. The transport protocol is completely irrelevant, it's just a bandwidth problem.
I've X-forwarded Firefox between two desktops on 10G Ethernet. I can watch 4K video, and I genuinely can't tell the difference between local and remote.
But: > This is why things like RDP, PCoIP, etc could do full frame rate HD video 15 years ago and you still can't with X protocol over the network.
Compression is going to have a much bigger impact over a large motion than most anything else; you can stream video over HTTP 1.1 / TCP thanks to video codecs, but X (sadly i think, seems like such an easy thing that should have been in an extension, but even png or jpeg never made it in) doesn't support any of that.
> It's almost all using Xshm for anything non-trivial.
Xshm is not available over a network link and it is common for client applications to detect this and gracefully degrade.
Xvnc is fine if you want a desktop session, but usually I just want one program, so X11 is a better fit for me. Once you have the infrastructure setup, it's really easy to go from a normal shell to running a graphical program. For me, I use a Microsoft Windows desktop environment, have X11 forwarding enabled on my usual shell targets, so I just start the X server and run the program. I could have the X server autostart (it's unobtrusive if there's no clients), but I haven't yet. On a mac, the Xserver starts automagically (once installed), so it's just a matter of enabling X forwarding in your ssh config to hosts where it's likely. Even less work if you run an X desktop.
This is a huge improvement over most "web applications", whereby your client (web browser) sits there looking like it's doing something when your network connection goes away. Worse still, you don't know whether the last (trans)action you sent to the server was processed or not.
Interestingly, that is xlib's behavior moreso than inherent in the protocol. xlib assumes connection lost is a fatal event, but if you're doing your own socket, you can choose to do this differently. (or, even with xlib, you can throw an exception from the connection lost callback and regain control before letting it abort)
Some server state is lost, but it is possible, with some care, to recreate that from your client upon reconnecting. You can even connect to a different server and somewhat seamlessly migrate windows over.
But yeah it isn't commonly done.
Emacs is the one program I know of that can actually do this. It can pop up frames (Emacs lingo for windows) on multiple displays at the same time and even mix tty and X11 frames. The Emacs session survives connection loss fine, frames on other displays continue working and you can reconnect if desired.
The one caveat is that Gtk used to have a bug that caused it to uncontrollably abort on connection loss but I build my Emacs with --without-x-toolkit (so it uses raw xlib, no toolkit) and that configuration has always been robust and performant. If I remember correctly the Gtk bug might be fixed now too.