Improving Xwayland window resizing
blog.vladzahorodnii.com
blog.vladzahorodnii.com
It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.
Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.
It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.
Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.
This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing.
Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m...
This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently.
That said, recent discussion on that latest one does look promising so maybe it'll finally happen.
Keep in mind that this is also a request by the application, not a requirement of the compositor to obey it. If there's not sufficient space where the application requests things then the compositor can just ignore it and do what it believes makes sense.
This is the source of the visual artifacts this article is trying to prevent, however. Sure, you probably don't want to block resize for multiple seconds, but in general the compositor is very responsive. The app may not be. If you just let the window chrome resize as fast as you get, what do you do with the rest of the window? Leave it transparent? Draw a default black or white area? Both are very ugly and very noticeable in practice.
This is the part of the problem space I know the most about - for a number of years I owned window chrome on Windows (don't blame me for the 1px border and way-too-subtle shadows, but after being overruled by PM/design, yes, I was responsible for implementing them).
As far as custom decorations - that is a lost battle. Companies and apps want their special design language and will simply not build for your product/operating system/compositor/whatever if you don't give them that kind of support. Twitter and Facebook both want their specific shade of blue and their specific font. Adobe is....well, let's not talk about Adobe. Browser tabs in the title bar are a P0 requirement these days and anyone who doesn't support that will be laughed out of the room. Etc., etc.
On the bright side I'm finally learning how Vim's digraphs work.
I did lose my custom mappings though, but I only needed them when I was in emacs and obviously there's already a command for inserting weird stuff, so I just added a binding for it.
It looks like the default en_US.UTF-8/Compose includes mappings of the form:
<dead_greek> <a> : "α"
Could you say more about this?
IME, it's just a matter of adding lines to ~/.XCompose — is there something I'm missing?
I guess it's up to each Wayland compositor, which calls for inconsistency :/
But I think there are specific exceptions still, such as this Chromium bug: https://issues.chromium.org/issues/40272818
I'm guessing that maybe I forgot to restart the machine to make sure everything got to read it. Or that whatever was broken long ago got fixed.
BTW, I can't reach my backup right now, but this seems like a good start to build up custom mappings in case anyone gets interested in this, https://github.com/kragen/xcompose
The Wayland book recommends XKB: https://wayland-book.com/seat/xkb.html
It does come down to the libraries used by a given app, though (see sibling comment).
[edit]
This is probably not relevant (assuming correct wayland libinput config), since this is not where mult-lingual input transformations live. libinput just handles the physical keyboard mapping and behaviour.
On more careful reading of the parent this sounds more like a buggy input method editor.. or maybe an issue with switching between X and wayland apps.
GTK_IM_MODULE=fcitx
QT_IM_MODULE=fcitx
SDL_IM_MODULE=fcitx
What happened to separation of concerns?
"Do one thing and do it well"?
Snark aside, it's still orthogonal to my question. I wasn't questioning whether it should be done centralized, but why it's supposed to be part of the display server protocol/compositor.
Note that more complex input methods do somewhat bypass the display server and communicate via dbus instead.
Did you miss the do it well part? Nearly every unix tool does more than one thing, the alternative would be a usability nightmare.
* grep, which primarily does pattern matching has dozens of file traversal related flags that could be handled by calling it from find
* find, a tool supposed to find files for some reason has dozens of flags related to executing applications which could probably be done by using xargs
* did you know that xargs can do pattern matching and replacement on its input string? there are probably dozens of unix tools that are specialized for that
Or in other words, your comparison is like saying food and shit are basically the same thing because both are made up of similar elements.
KDE just elected improving the Input story in its bi-annual community wide goals election: https://kde.org/goals/
If you're wondering why I don't use the native Wayland build of Emacs, it's because it is massively more laggy on HiDPI screens than the X11 version. I reported the issue upstream, probably won't be fixed for a decade when they'll decide to port to GTK4 and its hardware accelerated rendering backend. It took me a long while to notice that the typing lag I was experiencing was not because of native compilation or single-threading, but pgtk being a bit weak at rendering 3840x2160 pixels at 60 fps. In fact, it was not until I tried the Xwayland build that I experienced how much faster Emacs can feel.
Also consider running an xwayland app under gamescope which smooths out some issues.
We’ve long stopped living in a world where you should need to fully trust every piece software that you run as your user on your computer. And even probably trustworthy software can go bad at an update due to supply-chain attacks.
The thing about xeyes isn't that its privacy invasive, is that it shows that xeyes knows what you're doing in other clients.
Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
Unless you're sandboxed up the ass, Wayland won't save you when that browser escape happens. Something I did 20 years ago to a friend as a prank that still works today on a typical Linux desktop with Wayland; wrap sudo to log the users password the next time they use it. I didn't use a browser exploit for that, but it can easily be done if you have write access to the user's environment however that happened. Wayland won't protect you from that sort of thing unless you're willing to commit to extensive sandboxing.
Relatedly, now random apps can't record the whole screen. This is a good thing, now they get explicit user permission and go through xdg. I don't know how anyone could be opposed to this - it's objectively more empowering for you, the user.
But if you wanna argue Chrome should be able to read all your keyboard inputs whenever it wants be my guest. I can't fathom why people want that type of setup.
I won't expect stdlib.h to provide me a magical get_cursor_pos() function in any way.
GUI toolkits? (This includes the Win32 GUI.) Maybe.
It's not a universal thing though -- for example, you may receive mouse position _only_ when the cursor is in your window.
I did a quick google before I asked you that question, and it looks like wayland is one of the few exceptions to completely exposing current cursor position. Most "stdlibs" do expose it.
Are you referring to the native GUI toolkit of a platform (e.g. Winforms, Cocoa) or something else?
Just google it. You are incorrect and you're not pumped about it, I get it. The horse is fully beat to death. If you want to discuss further, my responses will probably just be lmgtfy links.
> In computer programming, a standard library is the library made available across implementations of a programming language.
You should show me that, among the mainstream programming languages, their standard library will provide get_cursor_pos().
I do agree that probably no-one was confused, though.
Not that it makes a huge difference in the discussion, but if you want to be smug about it then at least make sure you're right first.
It's an exception, not the rule. Even Python, well known for its batteries-included std, do not provide this magical get_cursor_pos() function that you insist is found in stdlibs.
What I'm trying to get across is that the need for that kind of intense security model, every process a threat, is not intrinsic to modern computing.
The most popular operating systems all do that (ios and android), and they have carved out safe APIs for all of that to work. You can’t patch up a Swiss cheese after the fact.
Is it hard to create standard APIs in a bazaar style of development? Yeah. But that doesn’t mean that it’s not the correct approach.
Sure Android and iOS are secure but in practice they kind of suck for making anything non-standard which limits creativity and freedom.
Can we have both a secure and extendable system? Maybe but none of them exist yet. I'm really worrying that Linux mainstream distros will become like Android or iOS.
You would be surprised how many content creator gets by with a single ipad.
But requiring user permissions for apps to do shady shit is a good thing. Cannot fathom why people are against that.
> But requiring user permissions for apps to do shady shit is a good thing.
Because it’s a mobile OS and every single spent CPU cycle is a detriment to battery life? There is absolutely nothing in the security model that would prevent it from running - but it is essential that processes have a “structured” lifetime.
E.g. compare how much more graceful android is in low-memory situations, asking apps to serialize their state and then stopping the last used one. Linux oomkiller will just reap my whole display manager for some reason.
Nobody is turning Linux into iOS. But iOS DOES have some good ideas. It's good, for example, that for an app to access your photos library they have to ask. I know for a fact you prefer that to the app just opening your photos without your knowledge and doing whatever they want with them.
Similarly, I see no reason why Chrome should be able to read the display output and keyboard inputs of my graphical password manager. It should ask me.
Can you name one professional software developer?
Probably, you can. But I don't want to limit myself to that sub standard environment. I love my iPad for some activities, for others iOS is just impractical.
What use is a safe API when it makes the entire system impossible to use by a significant fraction of the population (ie wayland and the visually disabled)? It's been a decade+ and none of the waylands have managed to support screen readers yet.
Are we to just throw out that whole class of people and tell them, "You don't get to use linux desktop computers anymore when X11 support is dropped". As someone with retinas that are progressively tearing apart, who already uses text to speech for many things, this is incredibly disheartening. I really don't want to have to switch to the Apple ecosystem.
This has absolutely nothing to do with the technology, it’s just that there is no standard protocol for one more thing simply because accessibility experts don’t happen to do some free work that will be de facto accepted by multiple different vendors. Comparatively, apple or google can just declare that this new API is the way to go, and support it natively from the de facto frameworks of the platform (and probably paid for accessibility experts along the way).
Like, your pdf reader is surely not evil, but do you trust every single pdf file you open?
And unless you fully sandbox your PDF reader then an exploit is going to have access to your user directory without any display server involvement anyway. X11 vs. Wayland doesn't even come into the picture.
And they should simply not have access to my home folder, it should be given access to a specific file only it is about to read.
No, that is something you don't want. I and many others, do want this functionality.
I mean, would you like VSCode tracking your mouse movements across the entire desktop and your keypresses and then sending them off to Microsoft? Probably not, so we're all in agreement.
The days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
It's an example, demonstrating that the mentality of "I don't use bad software" doesn't really work.
> giving up their computing freedom for "security" might as well have been the goal of the operation
How could you possibly reasonably argue that you're losing "computing freedom" because now you have the power to deny or allow applications from doing things? You're literally gaining freedom - that's not something you were able to do before. Now, you have the freedom to deny applications accessing something you don't think they need.
I have no opposition to Wayland in theory; my concerns are entirely practical and unignorable.
Then nVidia decided to switch to GBM/EGL way of doing things and it turned out everyone had incorrect assumptions...
The ecosystem slowly moves towards explicit sync now, but Nvidia was supposed to provide a driver already before that was happening and they didn't comply with platform's requirements at date, resulting in user-facing issues. With this one, they just got lucky that the consensus happens to move towards what they had already asserted.
It depends on the type of jank you're talking about. It's wholly disingenuous to characterize Wayland as janky but x11 as not - the jank inherent to x11 is what made the original Xorg developers start making Wayland in the first place. Empirically, if x11 was perfectly fine there would be no motivation to design a successor.
x11 gets the first-mover advantage of a lot of implementations and a straightforward design goal, but that's about it. It's not secure enough to recommend as a serious alternative to Mac and Windows users, and it's too slow and unopinionated to present a friendly face to new users. Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
If you want to keep using x11, it's unlikely someone is going to take it away from you. But it's on track for depreciation and hasn't been actively developed in years. Recommending it as a panacea to new users is a bad idea.
I've been recommending it as a serious alternative for years, and it's always presented a friendly face.
> Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
I have never heard anyone not already a Linux user comment on even one of those as a problem.
....because Windows and Mac users have not had these problems since 2006?
Those specific words are uttered by Linux users, true. However, Linux beginners _do_ notice some X11 issues, it's just that very frequently they only know "something" is off but not why they feel so.
From anecdotal experience, touchpad gestures are actually something my friend complained so YMMV. We ended up making a file in /etc/X11/xorg.conf.d/ to configure the synaptics driver. Another experience had to do with screen tearing, I helped them fix it by installing a compositor.
The security risk of X11 is theoretical, not practical. Yes, X11 programs can maliciously keylog each other, but this just isn't a thing that actually happens. And even if you do start installing random malware from the internet like a classic windows user, Wayland isn't going to prevent you from screwing yourself anyway. To actually be safe while installing and running malicious applications you need extensive sandboxing. Wayland can be one part of that sandboxing but is useless without the rest (to prevent the malware from stealing user files including credentials, using LD_PRELOAD hacks or similar to keylog other applications anyway, etc), and no distro suitable for recommending to Windows/MacOS newbs has the rest of the requisite sandboxing. The sandboxing touted by Wayland advocates is very esoteric and without all that sandboxing, a newb using Wayland has to exercise just as much caution when downloading software as if he were using X11.
A simple search leads me to this: https://github.com/anko/xkbcat
There isn't a real attack using it yet, only because attacking Desktop Linux is a really unprofitable endeavor (considering the marketshare, the ROI must be very low).
> To actually be safe while installing and running malicious applications you need extensive sandboxing
FWIW, X11 is unsandboxable unless you run a second X server on top of your current server [0]. Which is fine, but you need to consider that most, if not all sandboxing solutions on Linux that "newbs" use, like Flatpak, do not employ such technique when running sandboxed X11 applications.
The "security by default" behavior of Wayland limits the possible attack surface a lot, without requiring the end user to understand all the nitty details involved.
[0]: https://wiki.archlinux.org/title/Bubblewrap#Sandboxing_X11
This cannot be more further from the truth. Amongst the newcomers, it is rather popular nowadays for them to use Flatpak-bundled apps, especially with the rise of SteamOS (the Deck essentially) lots of Linux newcomers are in fact first exposed to Flatpak and running untrusted executables in a sandbox.
And the most prominent "untrusted executable" today to those newcomers has to be Bottles, which is a nice GUI wrapper for Wine and is sandboxed (if you enable wine-wayland, of course).
But I don't think a game purchased through steam counts as untrusted.
As compared to running untrusted programs completely naked?
>But I don't think a game purchased through steam counts as untrusted.
Bottles is there for people to run any Win32 program, not just Steam games. And I shouldn't have to tell you how many malicious Win32 programs there are.
Containerization on Linux was never intended to be a security feature for totally untrusted, malicious code. It's isolation for trusted code. If your scenario relies on securely running untrusted executables in a Linux container you are doing stupid things.
You see: If you want absolute security, for sure, go for a full-fledged VM! Or run something like QubesOS. It is a completely reasonable decision.
However, malice certainly has degrees, and the "mildly malicious" programs most likely cannot take advantage of sandbox escaping exploits. If Flatpak can stop 95% of all attacks (relative to running a program completely without sandboxing), that is already a win in my book.
But I will note again that X11 is a big hole (as in, almost a complete free-for-all) for sandbox escaping in Flatpak.
I'm done with this thread, have a nice day.
And besides that, "these threats are off in fantasy land" is an invalid defense in my opinion, considering the (quite sophisticated) XZ Utils backdoor happened not too long ago! Like I said, if such an attack towards X11 hasn't been deployed in the wild, it can only suggest such endeavor is unprofitable, not because the threats are fantastical.
If the attacker decided to backdoor an utility and make use of X11, it is most likely the backdoored utility will listen to keyboard events, read the bitmaps of other X11 clients.
And there's nothing that can stop the backdoor from doing so on X11...
Anyways, if you are saying the Wayland security policies are unneeded because there hasn't been an attack on X11 (this is the fundamental disagreement between us), consider the following: You don't install doors in your premise, because there hasn't been a case of burglary in your neighborhood?
When this is the case and there is a supply chain attack, what you think is a trusted application (and therefore not running under "WaylandX") can very well keylog you or take screenshots of your desktop without your consent.
In a deny-by-default model ala Wayland, applications will have to ask for permissions before they can do something considered to be privileged.
Are you really saying keyloggers do not exist in the wild???
> Wayland can be one part of that sandboxing but is useless without the rest
Yes but that is the point, and if you turn it the other way around X11 usually makes the rest of the sandboxing useless.
And yes, X11 makes that sandboxing useless, but that sandboxing isn't in play anyway because we're talking about noobs from Windows.
In principle I am also vulnerable to something like a RCE zero day in Firefox turning an otherwise trusty program into malware which exploits X11's open nature, but again, this sort of thing actually happening is unheard of.
I'm not superior, I'm just trying to keep a realistic grip on the threats I face. Modern security culture is fixated on what is theoretically possible, I care more about what is actually likely.
I disagree. How do you hijack interprocess communication on a Wayland device? I can tell you in very certain steps how to manipulate an x11 client but outside hijacking /dev/ I can't imagine a similar attack on Wayland.
This is not something you should ever have to do
I think your argument is about specific implementations of WM. While the argument of "I deal with X11-based WMs because it's fine when I don't care about security at all" may be valid in very narrow cases (such as air-gapped systems), the argument more generally is pretty weak.
Its not surpising that x11 based WMs, such as the almighty [awesomeWM](https://github.com/awesomeWM/awesome), have more features implemented than, for instance, [jay](https://github.com/mahkoh/jay) due to the enormous time it has had to develop (though I am _very_ excited to see `jay` develop more fully, and expect it to be well used by the more tech-savy devs).
However, some WMs in the Wayland space are doing quite well on that front. I recently had some substantial problems arise in my system which (surprisingly to me, but perhaps some are getting used to this) would have been prevented by using a memory safety language for my WM, so I have made the switch to (for better or worse) only ever consider Wayland+Rust WMs. In this space, [niri](https://github.com/YaLTeR/niri) is actually quite good, and to the point - it is developing correctly _and very quickly_. So, any issues on some WM not implementing some desired feature are quickly disappearing.
IIRC, all the major 'gateway' linux distros, such as Ubuntu or Fedora, are all on Wayland by default now - so I don't imagine x11 will stay relevant much longer.
Even Windows generally switches down to software rendering when resizing. And on mobile devices, nobody resizes.
No pointer warp, however, is a failure (CAD packages all have significant issues on Wayland). Lack of multilingual support/accessibility is a failure. Lack of screenshots/screencasts is a failure. Lack of support for the BSD lineages is a failure. etc.
People are still bitching up a storm because Wayland still (Pipewire does screen sharing using DBUS(!) for example) hasn't fixed basic things while DeadRat is shoving it down everybody's throat by dropping X11 support.
The Wayland devs aren't wrong about the security implications of this kind of stuff. However, they're also not giving anybody solutions, either.
One big issue is that Wayland devlopment is so slow that the entire space moved forward and destroyed a bunch of assumptions that Wayland is based around.
Not true. FreeBSD and OpenBSD have had various wlroots compositors such as Sway available for some time. e.g [1] Some people have even been experimenting with KDE-wayland on FreeBSD since 2021.
Wayland support on any OS is not binary because there is no single layer like Xorg. It's a matter of individual compositors and their components being ported to the OS, which is a matter of popularity, and the BSDs are always at a disadvantage in that respect, so they lag behind, same as other software. Nevertheless, they are definitely gaining support. Then again this distinction is only technical, for functional support new DEs/WMs would always have needed to be ported regardless of display architecture, the only case it would not is for an Xorg drop-in, which defeats the purpose.
> Lack of screenshots/screencasts is a failure.
... but there are many functional screencapture apps, and even browser support, I use this pretty much every week. I think you might be operating on out of date info. I'd highly recommend giving it another try, the Wayland ecosystem has come quite far over the last decade.
And even then you have to deal with a mix because you have to work through two different unsynced possibly broken in weird ways connections (Wayland + D-Bus).
This results in how last week I couldn't screenshare under Wayland, and had zero chances to figure it out - and to make it worse since I had some important state and no time to play with reboots, I couldn't fix it.
Multiple IPCs make the whole stuff way more complex, there is a reason why we have a big monolith as browsers and not some `curl | htmlRender | jsInterpreter` Rube Goldberg machine — the unix philosophy is not the pan ultimate design, it works in some cases and is utterly broken in others.
And Wayland did not stop requiring multiple IPCs, in fact it mandates them as several features require passing data by secondary channels, not just for Portals et al - some don't even have any described way to reliably pass the information that I have seen (like how to pass around cookie to let one application activate window of another? Or maybe the spec is such a mess that I'm looking at completely wrong place when I tried to fix ability to reliably switch between applications without extending compositor).
And yes, the architecture is more complicated in practice, otherwise it might have reached parity with what X11 did after as many years - like input method support. Unfortunately it's so broken that you have multiple versions of it in practice, it requires extra IPC in practice, and at least in my experience just does not work.
You need to have `xdg-desktop-portal-wlr` or equivalent package installed (should be depended upon by sway package really), and also the gtk one for some other stuff wlroots doesn't implement like notifications I think (don't worry it doesn't drag in all of gnome).
This is technically the only thing you should need to do beyond picking a wayland compatible screen capture program or browser - However it's possible the dbus variables are broken if you wrote your own sway config from scratch (guilty)... Depending on your OS, the default config should do this automatically, on Debian it includes /etc/sway/config.d/50-systemd-user.conf, I can't say for FreeBSD. So point is if your config omits the dbus-update-activation-environment command, you probably need to include it in your config.
TL;DR
apt install xdg-desktop-portal-{wlr,gtk}
exec dbus-update-activation-environment --systemd \
DISPLAY WAYLAND_DISPLAY SWAYSOCK XDG_CURRENT_DESKTOP=sway
Remember to restart sway to test this.
But they are wrong. Their security model assumes an ecosystem of untrusted apps when we already had something far superior: distributions with vetted packages.
Wayland is like bolting down your furniture because if you let in random strangers from the street they might steal your stuff. Instead of adding obstacles that make my own life harder I prefer to keep better company.
It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
I don't want someone who finds a vulnerability in a fart app to be able to escalate that to attack other apps on my computer.
I trust my accountant with a lot of sensitive data but I don't give them the keys to my house. I trust a friend with my house keys to water plants while I'm gone, but I don't give them the password to my bank account.
> It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
Ok, when is the last time you or anyone you know personally sustained any nontrivial damage because of such an event? You can make up hypotheticals to scare people all you want but the simple fact is that no, people are not actually in any more danger on their computer than they are just being alive.
Most legacy X11 apps in active use are actually games which tend to be fullscreen and not resized, so it wouldn't happen that often for many users.
(That said, sure, I can also respect the stance as I tend to place a premium on glitch-freeness too.)
My comment was concerned with making it abundantly clear that this glitch happened in specific scenarios and not just any and all window resizing.