Comment by saltcured | Hacker News Reader

Comment by saltcured | Hacker News Reader

Parent Full thread

To be honest, I think remote rendering is unnecessary and is mismatched for actualy application workloads. The amount of data and IO needed by the renderer in many applications outstrips that needed to send the rendered result to a display. I.e. the textures, models, or font curves loaded into GPU-local RAM and then resampled or reinterpreted to draw frames. This stuff is the heart of a graphical application, and doesn't really make sense to split across a network.

When the application host is really resource-starved and does not need to supply much content to a renderer, we are better off using more abstract network interfaces. On one hand, HTML+CSS+js are the remote rendering protocol to the browser which becomes the remote renderer. On the other hand, sensor/data interfaces become the remote data source for a GUI application that can be hosted somewhere more appropriate than on the resource-starved embedded device.

What I will miss from SSH X-forwarding is not remote rendering. It is per-window remote display. I would be perfectly happy with remote applications doing their rendering within their own host, and just sending their window contents to my display server, where my local compositor puts the window content onto my screen. Protocols more like VNC or RDP could be used for each of these windows, adaptively switching between damage-based blitting, lossless and lossy image codecs, or even streaming video codecs. What is needed is the remote window-manager and display protocol to securely negotiate and multiplex multiple windows opening and closing rather than one whole desktop.

What I won't miss from SSH X-forwarding is window and application lifecycle tied to the network connection. Let me disconnect and reconnect to my remote application and its windows. Let me share such a connection to have the same windows displayed on my desktop and my laptop, or to window-share with a coworker down the hall...

That's pretty typical. "I don't need it, so therefore nobody should need it, so therefore I will remove it".

Unix is different things to different people and to remove the network capability from X is a step back, not a step forward. There are a whole pile of neat things that you can do with networked graphics, so much, in fact that Plan 9, in many ways a successor of Unix used that theme as the core of many of the system services. All of them work locally as well as remote.

The thing is, you don't need explicit support in the display server for remote. RDP works just fine on Windows, yes even single-window forwarding, without it.

RDP is a very weak implementation of what X is capable of functionality wise.

In what way? Please elaborate.

I’m also interested in hearing GPs thoughts on rdp and x.

In a nutshell: RDP is a good way to access remote systems that are running a window manager of sorts, X is a good way to have remote clients connect to a local X server (the display).

RDP is more closely related to VNC and something like 'screen' than X, which is more like a networked resource that you can access through remote clients.

RDP is an application sharing protocol, X is a screen sharing protocol, the two are radically different from each other which gives each advantages and disadvantages that the other lacks and a bunch of overlap. Most notably, with X security was bolted on as an afterthought. RDP, originally reading the screen contents and sending those over (compressed) does not allow for much interaction with what gets sent over, it is rather low level whereas X sends over display primitives. Again, X's initial - rather naive - implementation made a ton of assumptions because the original implementors had nice fat workstations and fast networks to work with and so the real-world utility of these features was rather limited.

Anyway, you could write books about this and not get all the details, besides the various implementations of both but all that there is to this is that due to their history and intended application the two are very different beasts and X offers a versatility that not a whole lot of people need but when you need that versatility you need it badly.

I thought with the later versions of RDP it somewhat works like remote X, where can pass through a lot of the context and the rendering is done on the clients GPU instead of being rasterized and sent over the network? I've found RDP very snappy in later versions, with very crisp rendering even over relatively low bandwidth connections (order of magnitudes better than VNC).

There's also remoteapp which allows you to just run one program remotely, but haven't got much experience with this.

RDP got a lot faster due to tapping in at different levels in the rendering engine, but those are mostly optimizations and not how the whole thing is meant to be viewed, it is a 'remote display protocol' that transports what is already visible on some remote screen or memory buffer to another computer for viewing. Shortcuts and optimizations don't really change that, whereas 'X' is at heart a client-server solution.

Really? Where is the GPGPU support of X?

Last time I checked it was a X extension, if at all.

> Really? Where is the GPGPU support of X?

What does an optimization have to with anything? So what if X does not have a native GPGPU driver, that's really more a function of hardware manufacturers support than anything else. And if you are not too picky about all your stuff being 'open' then NVidia's X driver and CUDA on linux co-exist just fine with accelerated graphics and GPGPU support.

It has to do with the protocol, drivers alone aren't enough.

https://en.wikipedia.org/wiki/RemoteFX

I am suggesting that people need to distinguish remote rendering and remote display. I do not advocate removing the network capability. In fact, I personally have avoided using Wayland for this very reason. But, I am also not volunteering to maintain Xorg myself, so I try to understand where things are going.

Hardly any of the use cases I have heard for networked graphics actually require or even care where the rendering occurs. They are just concerned with application software installed and running in one place causing pixels to appear for a user elsewhere. These could work just fine with remote display streams, with all rendering happening on the same host where the application executes.

I ponder what it is that I really dislike about VNC or RDP, and I find it is the nested desktop and lack of integrated window management. I think I'd be fine with some kind of "rootless" variant to let me SSH-forward a set of windows to a Wayland compositor. I would prefer it to have more flavor of tmux/screen to let me connect and disconnect, and easily define named sessions or groups of windows that I would want to grab together.

> to remove the network capability from X

They haven’t removed it from X have they? You can keep running X if you want to (you may need to start maintaining it yourself.)

They don’t use it so they don’t want to implement or maintain it. Seems super reasonable to me?

In practice this is a very niche use case, and the maintenance cost and architectural constraints it imposed are just not worth it.

The Unix philosophy: do one thing, and do it well. If you want a remote renderer, write a piece of software that does just that, specifically.

> The Unix philosophy: do one thing, and do it well

You're trying to argue that there should be two distinct pieces of software, a local and a remote renderer, for simplicity.

Rendering does not fit in that space well. The physical resource of a display and graphics card really only allows one renderer to run, so it's not really useful to say "just write two renderers, one local and one remote".

The unix philosophy is not a good argument. It's a principal by which you can make decisions, but it doesn't always lead to good decisions and it's quite open to interpretation as well.

X11 did that. It has clients, doing their client thing, and a server, managing the display details.

Yes, this means that unlike many C-S architectures, the server is the part nearest the user and the client (may be) further away. But each did its respective job and stuck to its knitting.

Keep in mind that when X11 was first created, and for much of its early life, through the 1990s, it was used for remotely accessing another (or multiple other) systems. The networking capability was baked-in. Nevermind that security was bolted on later, here, have a MAGIC_COOKIE....

Yeah, I sure liked being able to run RPi apps rendered on my laptop with X.

Per a discussion the other day, my RPis are in a drawer. Have we swapped out screwdrivers for RPis in the modern junk drawer?

> Have we swapped out screwdrivers for RPis ...

No, they've been added to the screwdrivers. I've found it challenging to use RPi to drive screws.

Surely they can at least manage a flathead.

I can do the same with WebGL, RDP, VNC.

Doing graphics at the level of AAA games wasn't surely a Plan 9 distributed network feature.

It did not necessarily preclude it either, but the networks of the day would have been the limiting factor, not Plan 9.

Nor the ones from 2019. It remains to be seen if Stadia will succeed where many others failed.

I think remote rendering is unnecessary and is mismatched for actualy application workloads

Nowadays, perhaps, but that's software bloat in action, software getting slower faster than computers gaining in speed. In the last millennium if one had a login one could connect to the computer in Daresbury, UK that had a copy of the Cambridge Structural Database and search crystal structures. Many people at British universities did that. It would render graphics on an X server. Once I tried it on holiday at the East Coast of the US, and it wasn't a painful experience at all. Then again, it was the old version that still had PLUTO, which was a program from the computing stone ages.

VNC has a significant performance penalty compared to network X.

This is true but VNC doesn't exactly win any awards in the performance department in general. Sending bitmap diffs is pretty much the least efficient most naive way to do remote desktop. What VNC has going for it is that it's dead simple to implement because of that fact.

X forwarding goes the complete opposite direction to pipes the whole X protocol over the network. Convenient -- yes, also dead simple to implement -- yes, actually works -- well no. You see the X ecosystem also assumes all rendering is local, that fact is just hidden in edge cases. For example Nvidia graphics work on X with an extension that both has a client and server component -- as soon as you forward X traffic that's looking for the server-side extension... crash.

RDP by contrast does the right thing(tm) but also the hard thing by defining an intermediate object model which is semantic enough to kill the need to send pixels over the wire most of the time but flexible enough to allow for it when necessary.

RDP something like RDP is the correct answer - I thought it worked by effectively streaming GDI calls over the network - well that and whatever they used to replace GDI for 3D enhancements.

I'd agree with you for modern applications that are not well written, I've been running X applications over the network however for 20 years or so, and outside of things like.. well 3D applications, network X works pretty well - my only real complaint was not including sound too.

NCD (Network Computing Devices), who made X Terminals, came up with a sound server called "NetAudio", based on the X11 server, protocol and client library, with the graphics ripped out and sound stuck in.

I used it in the early 90's to support the Unix/X11/TCL/Tk multi player version of SimCity, with a scriptable TCL/Tk audio server that could drive either /dev/audio on Sun/SGI/etc or NetAudio on NCD X Terminals. Other TCL/Tk clients like SimCity (but possibly others) communicated with it via the TCL/Tk "send" command, which bounced messages off the X server.

http://www.art.net/~hopkins/Don/simcity/simcity-announcement...

NetAudio had its problems, but basically worked. Except that it insanely mixed audio by AVERAGING (add waveforms then divide by the number of sounds) instead of adding and clipping, and I couldn't convince the NCD engineer that was the wrong way to mix sound, and to just add and clip instead, like sounds mix in the real world: When you talk over music, the music volume and the volume of your voice doesn't magically lower by half. Hopefully they've fixed that problem by now.

https://donhopkins.com/home/archive/NeWS/sound-mixers.txt

It evolved into the Network Audio System (NAS):

http://www.radscan.com/nas.html

I can't say how popular it is, or if anyone actually uses it, though:

The Release version is now 1.9.4 (10/07/2013).

I've always wanted one of those NDC boxes to use - they're just kinda hard to get working on a modern network.

X performs better than VNC for stuff like xscreensaver, but not usually for video or applications that do dumb stuff like cute animations of internal windows opening. VNC just drops out the frames and catches up. X tends to render to perfection each graphic context call.

I’m fine with that tradeoff. For people who need network rendering, they can pay a performance penalty when they use it. For the 99% of people who 99% of the time don’t use this feature, we aren’t paying the performance (and architectural complexity) cost of this.

Electron is panned by many these days for requiring an expensive network system for a local process. Why doesn’t X11 get the same scorn? Have we forgotten the days when a Mac could run a GUI in 128K while a Unix system struggled to run x11 (with no clients) in 4MB? X is heavy.

> Electron is panned by many these days for requiring an expensive network system for a local process. Why doesn’t X11 get the same scorn? Have we forgotten the days when a Mac could run a GUI in 128K while a Unix system struggled to run x11 (with no clients) in 4MB? X is heavy.

If you actually properly understood both the argument you're making and the reason why most people dislike Electron and friends, then you'd not be making that argument. I'll give you the benefit of the doubt and assume you're joking.

Actually, Electron (and most other web browsers) on the Mac OS/X and iOS use IOSurface to share zero-copy textures in GPU memory between the render and browser processes. Android and Windows (I presume, but don't know name of the API, probably part of DirectX) have similar techniques.

It's like shared memory, but for texture memory in the GPU between separate heavy weight processes. Since simply sharing main memory between processes wouldn't be nearly as efficient, requiring frequent uploading and downloading textures to and from the GPU.

Mac OS/X and iOS IOSurface:

https://developer.apple.com/documentation/iosurface?language...

http://neugierig.org/software/chromium/notes/2010/08/mac-acc...

https://github.com/SimHacker/UnityJS/blob/master/notes/IOSur...

Android SurfaceTexture and GL_TEXTURE_EXTERNAL_OES:

https://developer.android.com/reference/android/graphics/Sur...

https://www.khronos.org/registry/OpenGL/extensions/OES/OES_E...

https://docs.google.com/document/d/1J0fkaGS9Gseczw3wJNXvo_r-...

https://github.com/SimHacker/UnityJS/blob/master/notes/ZeroC...

https://github.com/SimHacker/UnityJS/blob/master/notes/Surfa...

Why are you responding with this to me, rather than my parent poster.

Because when attacking your parent poster, you neglected to explain your argument, and the reasons why you believe most people dislike Electron, and why X11 doesn't get the same scorn, which caused you to be downvoted.

Care to elaborate, please?

Do you know if X11, like Electron, supports shared memory textures in the GPU like IOSurface and GL_TEXTURE_EXTERNAL_OES are for?

> Because when attacking your parent poster, you neglected to explain your argument

I can't force someone to understand something that they do not want to. More than enough people have written, much more eloquently than I, about why electron is bad. None of the critiques I've seen are 'because it relies on an expensive network system'.

EDIT: Hell, even if we assume that the critiques are about "expensive networked system"s, Electron is still incomparable to Xorg so the root-parent's post is making a false comparison.

In the late 80's, context switching was so slow and local area networking was fast enough that it was sometimes actually snappier to run your xterm or emacs on another system over the local network, than both on the same workstation.

That's because running over the network didn't require ping-pong context switching back and forth between the X11 server and the client at every keystroke, so both client and server could run smoothly without getting switched out.

The X protocol is so chatty and ping-pongy that it could require several context switches per keystroke to handle the event and update the screen, when running both server and client locally!

> I think remote rendering is unnecessary and is mismatched for actualy application workloads

What do you think of products like Google Stadia? https://store.google.com/us/product/stadia_founders_edition?...

Well, I haven't paid much attention to these gaming products, but this is a really clear example of the difference between remote display and remote rendering.

This Google Stadia is using a remote display protocol. All the rendering is happening in the machine in the datacenter, where the CPU and GPU lives and the game executes. It's a bit like someone sitting in the datacenter playing the game while you live-stream it at home, except they also pass through input controls so you can steer the game from home instead of just watching it.

A remote rendering protocol, which is no longer really sensible, would try to keep the GPU in your living room while running the game in the datacenter. Instead of streaming the screen updates at a nice steady pace based on your display resolution, it would have to stream the drawing commands from the datacenter to your GPU, adjusting the geometry as the game simulation updates the world model and stalling with nice loading screens every time it needs to shuffle in different texture and model data from the game computer to your GPU via the internet.

Thank you. I really get it now. You have a fantastic gift for educating!

Reply on news.ycombinator.com