Wayland in Gnome: two progress reports | Hacker News Reader

Wayland in Gnome: two progress reports

lwn.net

117 pointsluu65 comments

I'm typing this from a wayland session of the GNOME 3.14 beta (via Fedora 21 Branched). While there are still some serious bugs, I have to say I am very impressed. The synchronization problems and graphical glitches of X always made the Linux desktop feel second rate compared to Mac or Windows. With wayland, interactions with elements on the screen such as moving and resizing windows or dragging apps to the dash are perfectly in sync with the cursor. Not even Android has achieved this.

In addition, GNOME 3.14 is looking outstanding. With the latest design refinements, I find the interface significantly more attractive than OS X (before or after redesign). It's not quite there yet, but if the community can deliver a system with fully functional wayland, portable sandboxed app containers, and a stable development target, the Linux desktop truly will stand a chance against the proprietary competitors.

I've been using Linux for two decades and not experienced the graphical glitches you speak of. Moving and resizing windows? Nope. Some infuriating compiz bugs, sure.

You didn't notice in two decades that moving and resizing windows in Linux was an ugly affair plagued by graphical artifacts? I wrote this almost a decade ago and as far as I can tell it's still true today unless you use Wayland: http://www.osnews.com/thread?93850. Obviously a lot of pieces (graphics drivers, etc) have fallen into place since then, but it seems like ditching X was a necessary step to getting a clean, artifact-free experience.

Nope, not seen that.

Sounds like resizing on a poorly written MacOS Classic application.

Possibly it happens and my brain filters it out.

look closer. you mouse cursor will move ahead of the window. check out the Xorg mouse handler code. it performs the update in a signal handler while the rest of the window movement is in the normal path (which could be blocked by chatty Xlib clients).

That's normal and a feature. Mouse pointer is driven by hardware input and rendered independently (was a hardware overlay in the old days) and is soft realtime (hopefully). Because the window manager is a normal process, operating asynchronoysly, the window frame will lag a bit (depends on system load), and the app contents even more (this can't be avoided, even if the WM/compositor was realtime). Making the window frame wait for window content is broken by design.

Its not a feature, its an implementation artifact of the window frame being drawn by a separate process than the window contents. Its broken by design. OS X does it the right way, with the window contents and window frame being redrawn in lockstep and double-buffered so the user never sees partially drawn windows.

This is probably a legacy of X's network oriented history. If your window contents are being rendered by another computer on the end of a high latency connection you want to be able to reorganise your windows on screen without waiting for the other end to finish the reflow.

I assume that window contents are drawn to a buffer before being rendered on a modern system anyway. I can't remember the last time I saw partially drawn window contents.

It's not a historic artefact. You can't trust the app to render in time... (that's also why client side decorations are worthless).

But wouldn't that also hold for OSX/Windows?

Well, even if you assume the app will take too long to render, is it really worthwhile to render the "rest" of the window (pointer, frames, etc). The main reason you're working with a particular window is the content, not the chrome, so drawing the chrome asynchronously only serves to make things look weird, not actually get things done faster.

when i talked to keithp about this last year, he wasn't as enthused as you seem to be about the implementation. it was purely a side effect that causes people to think that Xorg isn't locked up when it actually is.

Nope, smooth here. But now that you mention it, resizing (only) is behind the cursor a bit. Honestly never noticed, I'm not in the habit of resizing my windows often enough for it to matter much.

What serious bugs have you encountered so far? Let me know, as I need to fix them before release.

I suspect many of the issues are actually problems with Fedora (it seems I have a mix of 3.13.9 and the previous release) but I'll list them anyway, just in case.

* Apps are actually launching under xwayland by default. Everything segfaults if launched with GDK_BACKEND=wayland.

* I use dvorak. Xwayland honors the setting but the shell is still qwerty.

* Two-finger scrolling works in xwayland (though twice as fast as usual) but not at all in shell.

* Restarting shell kills running apps and afterwards only Web will open. And it has the app menu and min/max button in the headerbar.

* External monitor not recognized, works fine under X

Question if I may since you seem familiar with gnome. I use 3.12 and really enjoy it, but gnome shell (or mutter, or the js engine) seems to leak memory pretty badly. That is, if I mouse to the top left hot corner, then escape, I see its RSS increase, as probably expected. However, it increases each time I do this. I can get it to 500mb by doing nothing more than going into and out of the dash for a few minutes. In real use, after an eight hour day I see it anywhere from 700m to 1g rss. Any idea what's happening behind the scenes, or if this still happens in 3.14?

What happens if you run looking glass and perform a garbage collection. (<Alt>F2, lg<Return>, click on trash icon).

If you see RSS drop significantly back down, there is a chance it will be fixed in the future. It's pretty difficult to let the JS engine know how big the resources are that are not in JS-land so that it can take things into account for a GC. (There are some patches related to this in bugzilla IIRC).

Each time I've attempted to use Linux as my main OS it's always X11 that scares me off. Booting into console to edit xorg.conf gets tiring after a while, spending hundreds of dollars for graphics card that has more stable drivers, the general ancient 90's feel. I've tried it multiple times, but gave up each time.

You may be surprised by the latest versions, then.

I haven't edited xorg.conf in a long time (years?). And I'm using a power user oriented distro, Arch Linux.

As long as you use the open source drivers, you'll be fine. And lucky for you, the cheapest graphics cards (Intel integrated and AMD) have the best open source support.

Per default, you don't have xorg.conf anymore on modern distros. I have been in the "edit xorg.conf" boat as well, but there have been tremendous improvements since the bad old days.

Again this is very subjective. I have never had to mess with xorg. Things like Nvidia Optimus are obviously a tipping point of the problem.

There seems to be a lot of limitations here. It is odd, sandboxing just the graphics stack. It isn't true sandboxing so all the benefits of sandboxing are not gained, but a lot of the disadvantages still come with it.

It sounds like something as simple as a keyboard macro manager would not be doable, nor would the Windows utility AutoHotKey.

I'd also hate to try and write UI test automation for Wayland apps. It seems like they've pretty much ruled that out all together.

Heck what about services that do browser test renders? Would those be possible?

And not allowing plug ins is just foolish. "I'm sorry, we dictate that you must divide your application up this way." Flash, for years, drove innovation on the web, billions of dollars of value were created by Flash, trying to say "That won't be allowed to happen again because I say so" is hubris of the worst sorts.

It is odd, sandboxing just the graphics stack.

Since Wayland is a redesign of the graphics stack it looks like they decided to redesign it in a secure way rather than waiting for some system-wide security architecture to be designed. GNOME is working on full sandboxing: https://news.ycombinator.com/item?id=8219309

It sounds like something as simple as a keyboard macro manager would not be doable...

That's doable in the compositor/window manager.

I'd also hate to try and write UI test automation for Wayland apps.

That's possible either through accessibility interfaces ( https://developer.gnome.org/accessibility-devel-guide/stable... ) or the compositor/window manager. Note that many OS X "haxies" also use the accessibility API.

Heck what about services that do browser test renders?

That should be no problem using either a modified browser or modified compositor.

AutoHotKey-like behavior is already severely limited on X (can't send to non-active windows, same with XTEST). That is, if you want to use it for macros. For UI test automation it's fine I guess.

ui automation will have to be there, as accessibility toolkits require it (that's how atk works). it's just going to require an additional wayland interface.

There is also ongoing work to figure out how best to support taking screenshots of the whole desktop. Users expect the feature, but it has major security implications. If an application can take a screenshot unattended, it can also use that capability to record username/password combinations and other sensitive data. The straightforward solution is to force screenshot capture to require a user to verify the action; that would at least allow the user to detect if an unauthorized screenshot attempt is made.

I know how this will come across, but .. sounds like UAC?

And that is perfectly fine by me. There's a very limited subset of features which would require this kind of explicit authorization.

There was never anything wrong with the idea of UAC; the problem was the implementation. When the feature was first introduced so many Windows programs did things that caused UAC pop-ups that it because nothing more than an annoying click-through that no one paid attention to. It became nothing more than a default "okay."

The problem wasn't the elevation request itself, it was the amount of requests for a single [user-perceived] action. My record was 5, but a friend managed to get 7 requests in early Win Vista... I wish I could remember what for.

yeah, something like that. I guess it'd be cool if you could just whitelist applications to do stuff, and have others pop out when requiring the permission.

Sounds like a security hole: non-whitelisted malicious app launches whitelisted app as an external process, and then mayhem ensues.

User permission should be required when launching external processes.

And whitelisted processes don't have to take input from anything other than the user himself; a command-line screenshot tool that can be run by any program and doesn't ask for interactive confirmation through a trusted channel (ie. keyboard events directed into its own window by the compositor itself) would be a pretty stupid thing to whitelist unless you've got proper sandboxing for any untrusted apps.

An application can do all kinds of shit, can't it? It could probably replace your Browsers icon with an icon that starts a custom version of your Browser which uploads everything you type to some server.

Which is why we¹ are working on sandboxed apps.

https://news.ycombinator.com/item?id=8219309

¹I say "we" as in DE developers, I'm not a GNOME dev.

I don't know... we've generally used suid binaries for this in the past? I suppose the display server will run as root, so the sane approach is to add a screenshot api, possibly writing the screenshot to a file/folder/socket with special permissions? ("special" because the gist here seems to be a wish to limit capabilities more finely than traditional unix user/group/others -- after all the user has access to the whole desktop (say with a digital camera) -- so the goal here is apparently to limit which applications can access other applications. At any rate, root should be able to read all process' memory, so taking a screenshot shouldn't be a problem... (I'm not saying take a screenshot by reading video memory, just that the level of access is equivalent).

Sure, but UAC doesn't have to be as badly designed and implemented as it was in Windows. Allan Day has some great plans for adding this to GNOME: http://blogs.gnome.org/aday/2014/07/23/sandboxed-application...

Could another approach be to allow applications to flag a graphical element as secure and it will be excluded from any attempt by another process to read the screen? Don't even allow the superuser to take screenshots of passwords, credit cards, etc. They'll just show up as a black box.

Don't even talk about not allowing something to superuser. It could be nontrivial to enable though.

Lack of global keybindings would keep me off wayland for a long time. At this point I am completely dependent on my keybinds in fluxbox for doing everything from resizing windows to opening programs. If I can't specify real defaults that stupid programs cannot interfere with it will be useless.

No, Wayland only makes it difficult for individual applications to define global keybindings - so, for example, a global Ctrl-Space to play/pause your music player would need some extra plumbing to be implemented. Your WM owns (well, is, in Wayland) the display server, so it can define whatever keybindings it wants.

It sounds like it's not Wayland making it difficult, but other systems making it way too easy. If I was a GUI application developer I would totally expect to have to interface with some WM API to register global key bindings. Being able to just listen to the keyboard even when the app is not in focus really does sound like a security problem waiting to happen.

The problem with putting everything in the now-unified compositor-windowmanager-desktop thing is that it creates a huge barrier to implementing desktop customizations. Instead of being able to swap out just the window manager, or just the dock, or just the keybindings, you have to either fork the standard implementation (and be constrained by its architecture) or implement all the interfaces expected of applications.

That's not accurate. A wayland compositor can implement plugin support and delegate stuff to external processes; this is how weston works. You can swap out one shell for another, and if you want you can implement a shell that itself delegates out the dock, window decoration, etc. each to different processes. But the defaults will and should be that arbitrary apps cannot in any way snoop on or interfere with the user's interaction with other apps, so obviously any app that wants to do so will have to get authorization and use some interfaces that aren't part of the normal protocol for presenting an app to the user. You're not "being constrained by [the] architecture" of the standard implementation in any way that's not completely justified.

You're suggesting that compositor- and shell-specific plugin interfaces can replace X11's desktop protocols? I understand that X11's security issues had to be fixed: I just wish that the new system had preserved some of X11's support for heterogeneous environments and not relied on the extensibility of specific compositor implementations.

Wayland tries to be a simpler and smaller system than X11. For it to specify equivalents to ICCCM and EWMH would be way beyond its scope. But there's nothing preventing such equivalents from being standardized; it just won't be part of Wayland, since Wayland also wants to target use cases where such things would be inapplicable. If there ends up being a proliferation of wayland implementations that support pluggable components, I think we'll definitely see some quick standardization of the necessary interfaces. But at the moment people are still working on successfully using one desktop environment at a time, so good support for the mashups will have to wait.

So, what you're ultimately complaining about is the lack of a standard API for creating window managers and the like - the X11 protocol includes this, the Wayland protocol does not. But this doesn't stop you from using the API provided by specific Wayland implementations to create a WM, and the ability to innovate on just this aspect without touching the rest of the protocol should make it a lot easier to create a good WM API. Yes, you have to choose the architecture/API of one of the existing implementations, or write your own. But this is much better than having no choice, and being forced to use the X11 protocol!

https://github.com/walac/blender-wayland

So, this is a "Wayland back-end for Blender". Does that mean it renders real-time to a window in wayland? I'd think what most would want is a front-end for Wayland? What am I missing?

It's a rendering backend for blender's frontend (The user interface is the same across Windows, Linux and OS X despite each of those using their own ways to draw things to the screen, after all).

Writers (including the author of Wayland's home page) get enthusiastic about Gnome and KDE ports to Wayland, but what Wayland really needs is a port of either Firefox or Chrome/Chromium, and I've seen very little progress on that score.

Really? Chromium already has a fairly functional Wayland port [0] and Firefox just needs to upgrade to GTK 3, which is (admittedly optimistically) targeted for 32 [1].

[0] https://01.org/ozone-wayland/blogs/tiagovignatti/2013/chromi...

[1] http://worldofgnome.org/optimistic-target-for-firefox-gtk3-i...

EDIT: Now running the GTK 3 Firefox port in a GNOME Wayland session, and it's working great!

Firefox GTK3 builds are still pre-nightly. We won't uplift such major work, current nightly is 34 with the uplift to 35 on the day after labor day. I would say that 36 would be possible but obviously depends on the work load of Glandium and Emilio Pozuelo.

If you want to test the builds the are available at ftp://ftp.mozilla.org/pub/firefox/tinderbox-builds/elm-linux64/ however there is no updater, so you would need to manually do that.

I admit I let my new-desktop-linux* evangelism get the better of me there, since I definitely knew that target was outdated. I should really put my money where my mouth is and try and help with the port.

* There should really be a term for the new emerging desktop linux platform based around systemd and wayland.

>I should really put my money where my mouth is and try and help with the port.

It's more fun to work together than to work by yourself. Maybe you and I should meet up and spend a day working together on the port. (Contact info in my profile.)

Awesome! Just emailed you.

Well, GNOME Web and Konqueror are both very usable.

As interested as I am for Wayland, Gnome 3 is still Gnome 3.

I have grown to really like Gnome 3.

Can't wait to see it working in (or, under?) KDE...

off-topic but genuine question:

Is it not a bit ironic that this source of information about display technologies (among other things) uses tables for page layout?

LWN is old school and it's all about the content.

So does the site you're commenting about that from.

Not really: LWN is about Linux. Linux/Unix users care more about content than form (usually)..

Reply on news.ycombinator.com