Hacker News Reader

Comment by bjourne | Hacker News Reader

You can't do opengl efficiently over the network. At least xorg can't. Most applications uses opengl these days.

For me, I've been using Linux fulltime for the last 15 years and I have never, not even once, had the need to connect to a remote X11 server. ssh has always been enough for me.

View on HN

I think it is likely that your lack of need for running applications remotely largely reflects another change: the rise of the web. We run fewer native applications than we used to, for better and for worse. I run mostly a browser and a terminal. The browser has taken much of the role X11 had, and things like VNC have taken most of the rest.

Sure you can, it's just called WebGL! ;)

They just added another layer, flipped the words "server" and "client" around, and added more hardware.

Now you run the web browser client on top of the local window system server, through shared memory, without using the network. And both the browser and GPU are locally programmable!

And then the local web browser client accesses remote web servers over the network, instead of ever using X11's networking ability.

One way of looking at it in the X11 sense is that a remote app running in the web server acts as a client of the local window server's display and GPU hardware, by downloading JavaScript code to run in the web browser (acting as a programmable middleman near the display), and also shader code to run in the window server's GPU.

Trying to pigeonhole practices like distributed network and GPU programming into simplistic dichotomies like "client/server," or partition user interface programming into holy trinities like "model/view/controller," just oversimplifies reality and unnecessarily limits designs.

> the need to connect to a remote X11 server

You mean client, not server.

(ducks for cover)

> the need to connect to a remote X11 server

You mean X-Windows, not X11.

(ducks for cover)

XFree86 had problems with OpenGL over network (to the extent of "it does not work at all" for any reasonable definition of "work"). Since the X.org fork this got mostly fixed and the related infrastructure (DRI, AIGLX, GLX_EXT_texture_from_pixmap, ...) is used extensively by other parts of the X server (for example to implement XVideo or Render in terms of OpenGL).

On the other hand, typical applications (ie. games) usually expect that the channel to GPU has quite large bandwidth and no meaningful latency, which simply isn't true for any kind of network connection.

> Most applications uses opengl these days.

Now 'most' is a relative term, but I do not think that's true at all. There are very few applications that use OpenGL. Obviously games are an exception, but if you count in all GUIs I would say mostly target X11 (via Gnome or KDE).

GNOME and KDE can also use opengl for some tasks if it is available. But right, most applications doesn't directly use opengl. I believe if it weren't such a pita to integrate opengl in desktop appliations (thanks to X11's bad design), many more applications would be using it.

To give you an example, think about desktop effects. On Windows, several applications such as Explorer makes parts of their windows semi-transparent. It's a nice simple effect that is impossible (without tons of hackery) to replicate using X11.

https://www.x.org/releases/current/doc/compositeproto/compos...

I don't think that this qualifies as "tons of hackery" given the fact that other contemporary UI systems with truly transparent windows implement it in same way (IIRC in pre-Vista Windows truly transparent windows are supported on the OS level but the implementation involves hacks with backing buffer and synthesized expose events).

Hm right. That is indeed possible because the client asks xorg for a surface with an alpha channel. What I don't think clients can do is doing the transparency effects themselves. Because that involves using opengl and doesn't work (very well) in a networked context.

Clients doing transparency themselves is the hackish solution to this problems because it involves the client sohehow knowing what it paints over.

And in all this has nothing to do with OpenGL except that GLX_EXT_texture_from_pixmap is particularly efficient way to implement it on OpenGL supporting hardware. For simple transparency, the compositor can do the blending completely in software (which involves getting the drawables to the client and back), do it via XRender (which may get translated into OpenGL by AIGLX-supporting server) or by calling OpenGL directly. For compositor that only cares about transparency XRender is probably better API, for 3D-ish effects (as in Xgl or Sun's Looking Glass) OpenGL makes more sense.

At Sun we experimented with implementing an X11 window manager in NeWS. We didn't have transparency at the time (1992), but we did support shaped windows!

The NeWS window manager supported cool stuff (for both X11 and NeWS windows!) like rooms, virtual scrolling desktops, tabbed windows, pie menus, was easily extensible and deeply customisable in PostScript, and ran locally in the window server so it could respond instantly to input events, lock the input queue and provide feedback and manipulate windows immediately without causing any context switches or dealing with asynchronous locking, unlocking and event handling. You'd never lose a keystroke or click when switching between applications, for example.

I touched on some of those ideas in this ancient window manager flamey-poo:

http://www.art.net/~hopkins/Don/unix-haters/x-windows/i39l.h...

Also on that topic (I can't believe I still love flaming about this stuff so many years later! Sorry if I sound like a broken record.):

https://news.ycombinator.com/item?id=5861229

https://news.ycombinator.com/item?id=5844345

https://news.ycombinator.com/item?id=8039156

https://news.ycombinator.com/item?id=13198492

https://news.ycombinator.com/item?id=11520680

https://news.ycombinator.com/item?id=11319498

https://news.ycombinator.com/item?id=11319783

https://news.ycombinator.com/item?id=9977226

https://news.ycombinator.com/item?id=13196983

https://news.ycombinator.com/item?id=11481604

And here's how I think you should design a programmable "window manager" these days -- but it would be much more than just a window manager! It would be great for integrating legacy desktop and mobile applications into VR, for example!

aQuery -- Like jQuery for Accessibility

http://donhopkins.com/mediawiki/index.php/AQuery

Don asks Peter Korn: Hey I would love to bounce an idea off of you! I didn't realize how much work you've done in accessibility.

There is a window manager for the Mac called Slate, that is extensible in JavaScript -- it makes a hidden WebView and uses its JS interpreter by extending it with some interfaces to the app to do window management, using the Mac Accessibility API.

So I wanted to make pie menus for it, and thought of a good approach: make the hidden WebView not so hidden, but in the topmost layer of windows, covering all the screens, with a transparent background, that shows the desktop through anywhere you don't draw html.

Then just make pie menus with JavaScript, which I've done. Works like a charm!

THEN the next step I would like to do is this:

aQuery -- like jQuery, but for selecting, querying and manipulating Mac app user interfaces via the Accessibility framework and protocols.

So you can write jQuery-like selectors that search for and select Accessibility objects, and then it provides a convenient high level API for doing all kinds of stuff with them. So you can write higher level plugin widgets with aQuery that use HTML with jQuery, or even other types of user interfaces like voice recognition/synthesis, video tracking, augmented reality, web services, etc!

For example, I want to click on a window and it will dynamically configure jQuery Pie Menus with the commands in the menu of a live Mac app. Or make a hypercard-like user interface builder that lets people drag buttons or commands out of Mac apps into their own stacks, and make special purpose simplified guis for controlling and integrating Mac apps.

Does that sound crazy? I think it just might work! Implement the aQuery "selector engine" and heavy lifting in Objective C so that it runs really fast, and presents a nice high level useful interface to JavaScript.

Here is an issue I opened about it on the Slate github page, describing what I've done, but I haven't written up the "aQuery" idea yet. That's the next step!

http://www.donhopkins.com/home/archive/piemenu/uwm1/hacks.f

That's the FORTH source code from 1987 of a programmable multi threaded X10 window manager that lets you throw windows around so they bounce off of the edge of the screen!

I think he means "most by total runtime".

Chrome (and I guess most web browsers) do.

Stupid, stupid Nvidia drivers...

And that's basically the only application that counts any more these days.

Maybe you used Synergy to share a mouse and a keyboard over multiple machines. It's a use case that partly overlaps with running remote desktop applications on your display. Or VNC.

Reply on news.ycombinator.com