Wayland Protocol Finally Ready for Fractional Scaling
phoronix.com
phoronix.com
-taking screenshots
-auto keyboard/mouse scripting with xdotool
-not have unnecessary mouse/keyboard to monitor delay
-4K fullscreen, windowed fullscreen and windowed gaming with nvidia drivers, for native and steam proton games
-choosing to have vsync enabled or disabled in games
-EDIT: I should another important one for me: xclip, read from and write to the clipboard in the command line
How's the current situation of Wayland for these? Thanks!
Another area missing functionality is remote desktop. Yes, most Wayland compositors support VNC or similar remote desktop, but without the functionality of previous solutions. For example, I used x11vnc extensively: with it, I can share just one window or a whole screen, with various security options, and I can do so programmatically from the command line. Because VNC is now a feature built into the compositor, I no longer have a choice which remote desktop tool to use, and adding features is not a priority for maintainers of the compositors.
So, while, yes technically, you're right that some analogues of screenshots and remote desktop exist on Wayland, they do so without any of the modularity and flexibility that Linux is (allegedly) proud of.
Linux seems to have been infiltrated by people who simply don't believe in this anymore, and so we keep getting "modern" "batteries included" integrated monoliths that suck at everything but supposedly make things easier because this new crop of people don't want to have to know about or compare multiple options :(. And like, because for specific use cases the monoliths can have better performance, that gets used as a way to undermine what used to be the unique selling proposition of Linux :(.
The good news is that this is not unfixable; there is a venue and a governance organization to standardize the protocols and compositor interoperability that is needed. And there has been notable progress.
Specifically when it comes to screen share / remote desktop, standardization of the API to invoke sharing is happening by way of the XDG portal API efforts. There is active collaboration there e.g. between the browser vendors and the compositor vendors to agree on an interface.
Here is a recent progress report: https://jgrulich.cz/2022/11/21/webrtc-chromium-year-end-repo...
I'm not sure if that applies specifically to this case, but I'm not a fan of some standardization of protocols that is going on in Linux desktops, because:
- desktop managers can't properly keep state of your windows anymore on shutting off/on monitors. Claimed reason why this is not fixable: "the standard requires it"
- I dislike MM/DD/YYYY date formats. File managers used to have their own setting to choose your date format. Now they use some common standard so you have to set your Linux locale to something non-US. This then breaks other things. It's much nicer to be able to configure things per-application.
- Kolourpaint used to have a good color picker that showed you the numerical and hex values of colors. Now it has a horrible one that is only some palette where you can edit individual entries. You can't easily see the RGB values of a pixel in an image you open in Kolourpaint anymore. All due to "standardization", using a shared color picker component
- Focus stealing prevention no longer properly works, all due to adhering to some desktop standards
- Desktops like KDE and Cinnamon using horrible non-human editable json/xml/base64-encoded UTF-16 formats for their config
- Some Qt or GDK applications not having icons sometimes, due to them expecting some common icons available somewhere. This issue used to not exist, applications came with their own icons that just worked
I much preferred how unix and the desktop used to be, where you could manually edit textual config files to do whatever you want, and had the power and flexibility to change almost anything in intuitive ways
Standardization and Linux desktops are sometimes mutual conflicting terms.
There's doesn't seem to be much standardization on this front but tribes of people saying "new things should be this way" and other tribes saying "n'ah mate, that sucks, we'll keep using our own better way".
Linux desktops don't have the Apple/Microsoft dictatorship powers to actually enforce any kind of GUI standards so we get this constant hassle and even more fragmentation.
Do Apple and Microsoft actually have them? The GUI experience on Windows is relatively disjointed, and Apple famously did it to itself with e.g. randomly chosen brushed metal windows and other app-specific background textures. It appears far more common now for popular applications to ship their own look & feel and even theming. Let's say Discord, or even first-party, VS Code.
I'd say the accomplishments of standardization and interoperability on the Linux desktop are actually more substantial than on either Windows or MacOS. Consider that e.g. KDE and Gnome applications share a lot of important standards and are generally able to run in either vendor's shells. Windows and MacOS each effectively only have a single GUI frontend, so this kind of collaboration between vendors doesn't need to be accomplished, and it's not happening between the app makers, either.
You're about a decade late with this complaint... and wrong. The brushed metal look was for application windows that resembled "appliances", which would remain open as you loaded/saved different projects. This was to distinguish them from the normal white document windows, which are tied 1-to-1 to a file.
I'm not saying they always followed this rule exactly... but there definitely was a rule. And the Apple of the era would publish detailed Human Interface Guidelines detailing all the thinking and intended practices. Third party mac software of that era was famously extremely consistent too, because of this shared base.
I only have one request from Linux desktops: mac style shortcuts with meta instead of ctrl. I also know it will never happen because everyone took their cues from messy Microsoft instead of the superior Apple practices.
That not true anymore... I use Flameshot
Xorg doesn't support his either. Your screenshotting tool runs the timer and takes the screenshot when the timer expires. You complaint is like saying "the Linux kernel won't render markdown for me". No, it won't, you're looking at the wrong part of the stack.
For example, you can simply run `sleep 3 && shotman --output` to take a screenshot of an output in three seconds. There might be other GUI tools with a pretty timer setting. I've no idea if such a thing exists or not, but it's perfectly possible to write without any changes to the compositor (e.g.: the compositor provides the necessary APIs for thi).
You mentioned shotman, but that isn't even a real screnshotting tool: it's just a screenshot GUI that depends on existing functionality that may or may not be present in your compositor.
If all the compositors actually supported a standard for providing this functionality; and if that standard provided enough flexibility to do what I want, then I wouldn't complain. That's the essence of modularity. But the standards are poor and poorly-supported, and I literally can't do what I want. Your trivial example of using the sleep command to simulate delayed capture doesn't address the actual issue, and fails to recognize that no amount of hacky solutions will allow me to use a VNC screen sharing tool that shares only a specific window, because the Wayland compositors have chosen not to support that feature and failed to provide a standard way for external tools to do so.
Yes it does. Use the screenshot, screencast and remote desktop XDG portals for that. It's understandable you're confused because the X11 way was to try to jam everything into X extensions whether it made sense or not. The overall trend lately is to move APIs into other components (XDG portals, pipewire, DBus, systemd, etc.)
However, their implementation is not meant for native applications and are pretty inefficient. For example, the screenshot portal will save the image into disk and return its path -- so if you wanted to render the image on screen, you now need to read and decode a file. The screenshot is encoded, serialised to disk, serialised FROM disk and then decoded again just to get pixels from the compositor back into the compositor.
Portal really don't make sense if you're writing native wayland clients, since portals just really end up using the wayland protocol under the hood, with two or three services in the middle handing messages over just to expose a "standard" API.
In my use case, I want to screen share a specific window, not the whole screen; this is functionality previously provided by a third-party X11-based tool, which cannot be provided from an XDG portal (even on those compositors that support portal at all).
Regarding the fact that GNOME doesn't implement a lot of functionality: I don't think it's fair to criticise the protocol and architecture because one implementation decides not to implement a given feature. This is like saying "C sucks because some specific compiler doesn't support the feature I need". It's the compiler you've chosen that sucks in that case, not the language.
I'm sure there's some X server implementation out there that's also lacking features you want -- but you wouldn't blame the protocol itself for that implementation's shortcomings.
Xorg also doesn't prevent you from using an application that does have these functions. Wayland does prevent you.
> but it's perfectly possible to write without any changes to the compositor (e.g.: the compositor provides the necessary APIs for thi).
The necessary APIs don't exist, so it isn't possible to write the extra functionality without changing the compositor.
Honestly, I don't understand why anybody defends Wayland at this point. It's been, what, ~~six years? Six years~~ FOURTEEN FUCKING YEARS we've been telling you that we want to be able to use our screenshot and recording applications and that removing functionality that we use day-to-day wasn't acceptable. Are the Firefox devs the same people making Wayland? Is that why they're so unresponsive to user needs?
It's because Wayland is a set of protocols, rather than a single dominant implementation.
In order to do it right, cross-compositor and flexible, you have to get different actors with different interests and bandwidth agree on a standard. This takes time.
In order to get it working fast, developers need to make a compositor-specific implementation first and then put in the extra work of getting it through the standards discussion as well as switching their compositor ecosystem to the new standard.
Until this happens with all parts that you care about, you're going to be annoyed either about interoperability or functionality. Pick your poison, and cue Moxie's "The Ecosystem is Moving" blog post. Also, keep using X11 until Wayland has the features you want. Most DEs/WMs haven't ripped out X11 support yet, and hopefully won't until their support of Wayland protocols is solid enough.
However, the Wayland team is responsible for the set of protocols that they've developed and that they've asked others to implement. The Wayland team has failed to define protocols that are flexible enough to provide basic functionality that users expect, even if they were implemented perfectly.
This should not be a process that depends on individual developers building compositor-specific remote-desktop tools first, then praying that someone likes them enough to put it in the standard. Wayland built the standard, they just built an insufficient one.
If the Linux kernel had worked the same way, we'd never have gotten Linux.
Yes we would. In fact it's exactly how Linux works and has worked for decades. New features get added in drivers and then get rolled up into driver subsystems when enough hardware supports them.
I believe that the only thing missing is capturing individual windows with screencopy. There's actually an issue for that and ongoing discussion on how to implement it. Nobody's decided it's a bad idea, it's simply a matter of nobody having done it so far.
Xorg never "implemented" this feature. It's just that any client can read what other clients are doing. So Skype can screen-scrape your password manager by default. Obviously this was not a good choice, so Wayland has dedicated protocols for this functionality, with the intent of restricting them only to privileged clients.
Finally, while no _standard_ protocol exists to address this, there are compositor specific ways of capturing a single window, so most users have their needs met.
Surely, as much as seatbelts prevent my movement. And literally every wayland compositor has this functionality, that there is not a universally supported standard here sucks, but that’s just the way of life of the bazaar. There is no one company exerting a cathedral approach over the whole thing, and I believe that is what most of the linux users prefer. It just so happens to have some tradeoffs as well.
Except they do exist and do work.
To which I would add:
(1) taking screenshots from the command line (xwd)
(2) moving windows around from the command line (xwit)
(3) keyboard-free copy-paste between different programs: select to copy, middle-click to paste
(4) global indicator of where my cursor is (xeyes)
(5) define global key combinations that launch programs (xbindkeys)
(6) globally configure the keyboard in a particular way, key by key, not according to some pre-configured "language" (setxkbmap, xmodmap, xkbset)
(7) set keyboard repeat/delay to absurdly unusable values (xset)
(8) run graphical programs from a headless virtual machine (ssh -X)
I do not mind whether I'm running x11 or wayland, but I care a lot for all these features that are crucial to my workflow. How likely is it that items (1)--(8) will be supported somehow under wayland?
I'm a bit concerned that wayland gives a lot of attention to things that are meaningless to me (tearing, scaling, etc), and does not care at all about these basic usability features.
None of these discussions ever go anywhere. They will tell you that wlroots is standardizing all these things etc etc but as it stands today every compositor is at best reimplementing a subset of the features you describe. If I had enough money I would pay the Wayland people to work on something else and let the project collapse because they are constantly undermining the reasons I use Linux on the desktop.
There is a way to protect against LD_PRELOAD: use sandboxing. X11 is bad for security because it's an intentional escape around the sandbox.
The "tear it down and build something else argument" is also going nowhere, anything else you build is going to run into the same questions of how to go about implementing these features.
> (4) global indicator of where my cursor is (xeyes)
AFAIK, no tool allows you to do this. The issue is: clients generally can't snoop on input when they're not in foreground (e.g.: to make key-loggers less trivial).
A privileged client could do this using something like layer-shell. It's a hack, but it should work. Somebody needs to write it tho.
> (5) define global key combinations that launch programs (xbindkeys)
Compositors implement this themselves for the above reason.
> (8) run graphical programs from a headless virtual machine (ssh -X)
I guess this should work with XWayland? Keep in mind, most X applications work via XWayland. This means normal applications; xbindkeys or xeyes won't work since they rely on reading input when other applications are in foreground.
Even with root they'll have a harder time; there's a difference between "I can trivially tell the X server to give me details about events and then interact with arbitrary windows in arbitrary ways" vs "by running as root, I can watch raw input events come in from the mouse/keyboard and then try to figure out what they mean".
Still, xeyes is the favorite program of my kid who is starting to learn linux... I'd have a hard time selling wayland to him.
> to make key-loggers less trivial
Really? Is that a serious reason? Key-loggers are still very much possible because any program can read the memory of all other running programs of the same user. This seems to me like a pointless limitation that creates a lot of unnecessary complexity. But what do I know.
That's not true. Reading other processes memory requires root even for processes of the same user.
But it turns out that you are right! Well, at least in modern ubuntu/fedora desktop installs, a few years ago they disabled the ptrace(2) system call. ("The maniacs! They did it!", I'm tempted to cry in anger.) You can still read children processes, but unless you change the yama defaults in the kernel, ptrace(2) no longer works for processes that are not children.
I didn't notice because I surely have changed this default in my old laptop because I use ptrace daily. But you are right that modern linuxes are likely thus crippled.
Still, it seems that an attack model based on defending against a rogue user process is quite doomed. Can't this process change the init files of other programs that will run, even if they aren't its children?
No, they often can't. Things like Flatpak make it so apps have no unnecessary access to the filesystem.
I know there's automation tools like xdottool, never used those though.
I never saw much difference in term of delay, I find playing on wayland better, some games being much smoother even when using xwayland.
I have AMD hardware only so can't say here.
There's a procol being worked on to allow tearing, it's not available yet though https://www.phoronix.com/news/Wayland-Tearing-Control-Proto
The irony, to say the least.
There's plenty of tools for this. I wrote a light and simple one that shows a tiny thumbnail, [`shotman`](https://git.sr.ht/~whynothugo/shotman). But there's plenty others of all sorts.
> auto keyboard/mouse scripting with xdotool
There's a few tools for this: https://github.com/natpen/awesome-wayland#tools
> not have unnecessary mouse/keyboard to monitor delay
There's none that I can notice. Mice cursor are rendered with hardware support when available. Not sure what you reference is here.
> 4K fullscreen, windowed fullscreen and windowed gaming with nvidia drivers, for native and steam proton games
All this works on non-nvidia. I can't speak for nvidia, I'd suggest avoiding nvidia if you want to use Linux (or any other floss OS). Generally, nvidia on Linux isn't very pleasant.
> choosing to have vsync enabled or disabled in games
See this thread? https://www.reddit.com/r/swaywm/comments/ntszm9/how_can_i_ma...
I'm not 100% sure what you're asking.
> EDIT: I should another important one for me: xclip, read from and write to the clipboard in the command line
Some compositors don't use the hw cursor plane. All others that do don't update the cursor position asynchronously on mouse events, just once on the next swap. Those cases add an extra half-frame latency on average. It's worse at lower refresh rates.
Your tool works on wlroots compositors, which is to say it doesn't work on GNOME or KDE.
I don't notice any delay on keyboard or mouse entry across multiple monitors.
Can't answer the rest.
You may not personally notice it, or be used to it, but it's always there. Some people are more bothered by input lag than others, and there are so many compounding factors today that make it worse than ever. It's not only the double buffering in graphics subsystems like Wayland. Monitors are also often adding a frame or two delay to bump up their pixel "response times" and reduce ghosting by looking at future frames.
I swear, using Wayland or MacOS or Windows 10 on the fastest hardware today still feels slower and laggier than 10 year old hardware running plain X11 or Windows 7 (with the compositor turned off).
Given the choice between tearing and input lag, I'll take tearing any day.
https://zamundaaa.github.io/wayland/2021/12/14/about-gaming-...
Plus, the test uses a display at 120Hz, which is a substantial head start -- by making each frame of delay half as long compared to us mere mortals running at 60Hz.
For me it happens in the second monitor (120Hz with HDMI) there's a lag in the cursor movement. When I switch to X11 that lag is gone and the movement feels more fluid. The strange thing is that I don't see that effect on the laptop screen (144Hz). I'm using Plasma btw
- the keyboard and mouse can be controlled programmatically, but unaware if xdotool itself works
- I don't notice any input delay with the monitors at 60Hz
- I'm using the unsupported GPU flag with SwayWM and 4k games and windowed games work for me
- never tried disabling vsync. I'm guessing it won't work
I suspect if you use fractional scaling, the games wont run at 4k, but instead the fraction of 4k
X11 games maybe; SDL's Wayland backend can deal with fractional scales and make it render at native 4k.
Good news: https://www.phoronix.com/news/Wayland-Tearing-Control-Proto
> Introduced now with Wayland Protocols 1.30 though is a new staging protocol to allow screen tearing.
There's wl-clipboard with the commands wl-copy and wl-paste: https://github.com/bugaevc/wl-clipboard
Works flawless in my experience.
I have also been using the wl-clipboard program with no issues like the parent commenter. I use it for my password store, which decrypts a file containing the password with GPG and pipes it into the clipboard!
Among all the other technical issues, Wayland devs have an attitude issue. Users asked for fractional scaling for years and were faced with flat refusals and insulting quips from the wayland developers. The only reason they've added it now is because Valve asked them for it. Wayland devs think users are worms and corporations are gods.
If rounding / scaling artifacts are avoidable, this is fine. On the other hand if this stuff occurs in practice like with macOS (at least at one point, with the 2x render and downscale), this is totally fucked.
Wayland is much more low level than, say, CSS. So I see zero problem if the scaling factor can instead be a black box which applications are familiar to treat however they like. This is how "*.dpi: 120" in a .Xresources file works, for example. Not all applications might be able to support fractional scaling, but those that do can do it perfectly without any risk of scaling artifacts.
This was repeatedly rejected for reasons that I don't understand.
I remember Intel removing their code from their driver saying they won't support them as Wayland was the future, for example.
And it finally gets fractional scaling now?
How is colour management on Wayland?
Is there HDR support?
The system they use gets them something: 1) apps do not have bother with fractional scaling, so they do not get to bother about rounding errors and 2) they get to use output encoder for scaling, which is essentially for free, instead of GPU, which will take its toll on power consumption and GPU load.
This problem doesn't exist on Mac where the OS is kind of responsible for all the scaling. So, between MacOS's approach which may be not pixel-perfect but "just works" and Windows's one which usually works perfectly but still has problems in certain categories of software, I'll probably choose the former.
Firstly, your VST situation will be that the plugin, the host or both are not implementing things correctly: one or the other has claimed DPI-awareness inappropriately. That’s on them.
Secondly, you misunderstand the relative situations of macOS and Windows. The difference is not to do with their approach to scaling, but rather their approach to supporting old software, and the diversity of hardware. Social factors, not technical. Apple could absolutely have done fractional scaling, and it’d have worked just fine for them; the present situation in how well high-DPI stuff works across platforms would be indistinguishable: macOS would still have basically nothing broken, and Windows would still have quite a few widely-used pieces of software getting it wrong. Here’s why:
Apple developers are very probably using a high-DPI screen, so they’ll immediately notice if anything is off. Apple is controlling, and anything published through their store will have any obviously-bad scaling noticed and rejected. Apple doesn’t care about long-term compatibility, so earlier software that didn’t implement scaling properly largely just doesn’t run at all any more.
Windows developers are probably using a low-DPI screen, and are publishing to a largely unmoderated platform that still has software from decades ago run just fine.
Functionally there’s no difference between fractional and integral scaling in how well software will support them. If you try to implement things completely from scratch, fractional will be a little harder because you have more rounding decisions, but in practice you always use a library that takes care of that for you (normally a full widget toolkit, but VST plugins will often use something somewhat lower-level but still above the physical-pixels layer except when you deliberately opt for fine control), so there is literally no difference at all. The difference in how well things work on these different platforms is purely other factors.
They did¹, but it turns out the simpler solution just works, and that pixel-perfect purity doesn't matter because your eyes can't resolve HiDPI pixels any more than they can resolve the grid of subpixels that they're made of.
You drag an app between them and it goes apeshit, very annoying
Otherwise it seems to work for me
It was a good idea, but the implementation clearly isn't working.
My bet: Wayland will be the Windows Vista or the OS/2 of Linux.