XRDP with H.264 codec brings immense performance boost

VNC and the like are basically just transmitting the framebuffer to the client. There are some extensions for stuff like better support for video though.

RDP works a bit differently:

https://learn.microsoft.com/en-us/windows/win32/termserv/rem...

On the server, RDP uses its own video driver to render display output by constructing the rendering information into network packets by using RDP protocol and sending them over the network to the client. On the client, RDP receives rendering data and interprets the packets into corresponding Microsoft Windows graphics device interface (GDI) API calls.

RDP supports various mechanisms to reduce the amount of data transmitted over a network connection. Mechanisms include data compression, persistent caching of bitmaps, and caching of glyphs and fragments in RAM. The persistent bitmap cache can provide a substantial improvement in performance over low-bandwidth connections, especially when running applications that make extensive use of large bitmaps.

https://learn.microsoft.com/en-us/azure/virtual-desktop/rdp-...

To reduce the amount of data transferred over the network, RDP uses the combination of multiple techniques, including but not limited to - Frame rate optimizations - Screen content classification - Content-specific codecs - Progressive image encoding - Client-side caching

Remote Desktop Protocol is a modern protocol designed to adjust to the changing network conditions dynamically. Instead of using the hard limits on bandwidth utilization, RDP uses continuous network detection that actively monitors available network bandwidth and packet round-trip time. Based on the findings, RDP dynamically selects the graphic encoding options and allocates bandwidth for device redirection and other virtual channels. This technology allows RDP to use the full network pipe when available and rapidly back off when the network is needed for something else. RDP detects that and adjusts image quality, frame rate, or compression algorithms if other applications request the network.

Good old RDP sent the GDI calls directly. It also had fairly efficient bitmap caching. Then it got the H.264 stuff similar to what's mentioned here.

However in both cases I think the key reason RDP works so well is that it's very integrated in the stack.

I fiddled with FreeRDP server some 10 years ago, to see if I could improve performance. First off it ran on top of Xorg as effectively just another program. So getting a copy of the framebuffer was very CPU intensive, taking hundreds of milliseconds each time. Since this was before hardware H.264 encoding, it then had to do a delta pass to figure out the "damaged" regions, ie those needing an update. A full pass over the framebuffer comparing it with the previous framebuffer, very memory and CPU intensive. Then it could compress and send those regions to the client. The performance was very similar to good old VNC, ie horrible, so I expect it worked similarly.

Compare with good old RDP. It was integrated into the OS. It could send the draw calls to the client, or if needed, it could send bitmaps of regions. However if it had to do that, it already had the information of the damaged regions because Windows kept track of that anyway for redrawing purposes, so it didn't need to do any comparison pass like to find the damaged regions, and it could capture the draw calls from the applications, so no need to do a full framebuffer dump.

I really think Linux desktops like KDE and similar need to go a similar route if they ever want to get a decent competitor to Windows RDP. Remote Desktop needs to be designed into the solution, from the lower levels and up, not just tacked on top afterwards.

Very well. Assuming you have identical CPUs (say Intel with QuickSync) The performance constrains are really more dependent the desktop environment you choose (I have good results with GNOME on a LAN but LXDE works better in Azure since it has less decoration).

It is very good, and much less hassle than Parsec to setup today.

I’ve been doing this for a long while now, here’s one of the most complete writeups I did last year: https://taoofmac.com/space/blog/2022/10/23/1700

TL;DR: install xorgxrdp-glamor and make sure you are using the Xorg session in xrdp-sesman.

It is almost as fast as GPU-accelerated RDP in Windows, although that relies on deeper acceleration support (I have notes on that here: https://taoofmac.com/space/protocols/rdp )

A big concern I have right now is that Wayland is nowhere near as useful for multi-user RDP access (it doesn’t even seem to support offscreen GPU accelerated rendering, but that may just be my testing on Fedora).