Linux Touchpad Like a MacBook update: progress on multitouch
bill.harding.blog
bill.harding.blog
However, I've been using Ubuntu on Wayland for the past 2 years and have thoroughly enjoyed it. Frankly, I'm not willing to move back to X and it seems like a step backwards to me to invest any time, effort, or money in X at this point, so I'm going to discontinue my contribution.
I was hoping my monetary contribution would be a small ($25/Month) way to push Wayland forward, but the approach Bill and Povilas are taking is doing exactly the opposite by making it more comfortable for folks to stick with X. I understand there is disagreement and apprehension among the community with the slow move to Wayland, perhaps not at the same scale of systemd, but I firmly believe it's the best path forward and that X should be left behind.
So yeah, betting on Xorg in 2020 is not a smart move IMO.
1: https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-G...
I think this is incorrect. See the following, X11 backend is listed among supported backends. https://developer.gnome.org/gtk4/stable/extra-configuration-...
Having support in X (even if only in proposal stage) will allow much easier work in the toolkit and application layers. X and Wayland cover essentially al Linux users, so we will not need to estimate how many users will benefit from implementing gesture support in toolkit X or application Y. It will make convincing project maintainers easier.
This is really important, because maintainers of open source projects don't usually care about suggested features if they're not interested in them themselves. A new feature means additional work to them - discussing the design, reviewing PRs and handling of eventual bugs. Often the person who implements a feature disappears after the PR is merged. This makes the maintainers to view all feature proposals with a grain of salt. We need to have a convincing story of how majority end users would benefit in order to make the contributions easier.
Technical skill is not enough when contributing in open source. Politics are often as important.
To be clear, you're saying that implementation in the applications/toolkits ecosystem is the barrier to a better trackpad experience on Linux and that by unifying the feature set of both X and Wayland, this will encourage that implementation among the developers of the applications/toolkits?
If that's correct, that is one angle I had not considered before. It's also somewhat unfortunate that it was not spelled out more clearly in Bill's post, but hindsight is 20-20 of course.
Yes, exactly. AFAIK only Gtk right now supports any kind of touchpad gestures on Linux. Also, most of the users are still on Xorg and it's right now unknown when Wayland will be the default universally. So we would be in a weak position to demand maintainers attention, as there's little immediate benefit.
In the end it is not my money so I cannot decide, but IMO they should focus on bringing Qt and WxWidgets up to snuff with GTK gesture support. That means that as long as people are on Wayland, gestures will work ‘everywhere’.
Also, what really sucks about Qt is there's no kinetic scrolling in classic qt widget apps (e.g. telegram-desktop). Or there is but every app has to enable it or something??
When you start pinching/unpinching fingers apart, the zoom happens in sync with your finger spacing. As soon as you start pinching, the application seems to receive with the event stream all the x and y coordinates so that pan and zoom can be done simultaneously in lockstep with the finger positions and spacing on the touch pad.
Similarly when you pull 2 or 3 fingers from the side to e.g. go back a page, the reveal animation starts, or an arrow appears, and you can change your mind mid gesture by reversing the finger movement and the animation feedback is helpful enough to tell you that indeed the gesture will be canceled.
I honestly hope that the developers interested in improving this in libinput and stack have the same understanding that gestures are not about recognizing the one-off side/pinch movement with a couple of fingers, but about low latency fluid interaction of fingers with visual animation feedback tightly coupled. The other (one-off) type of gestures where you swipe your three fingers to the left and then after a sampling interval the framework or stack receives a ThreeFingersLeft swipe event (with length or time or xy values) is IMO a very underwhelming experience.
I think the one gesture I used more than any other on my MBP was the 4-finger swipe left and right to switch workspaces. I would have a fullscreen iterm/tmux/vim session in each workspace - each tmux is a separate project with multiple panes and windows. I can't really bind a keyboard shortcut to switch workspaces because I have so many shortcuts already bound to tmux/vim/coc and other vim extensions, which is why it's so nice to be able to switch workspaces with a touchpad gesture. I could flip back and forth between workspaces effortlessly, without losing precious key shortcuts for development. It’s a fantastic dev setup.
Fedora 32 seems to support this now! I don’t think that F31 did, but on F32 I can do the 4 finger swipe to switch workspaces, and on my thinkpad it works pretty much as smoothly and as interactively as it did on my MBP - swipe slowly and the screen slides following your fingers, change direction in the middle of the gesture and the screen slides back - the whole ‘stream input events and the screen is an extension of your fingers’ as you describe.
The Linux desktop is getting a lot better. I don’t think I would ever have switched without Wayland and now they’re getting gestures right.
Probably related: right around when the setting was moved deep into accessibility, Finder started having issues with three-finger drag.
I'm in the group that gladly foregoes three finger drag, if it means we get to keep other three finger gestures.
Disabling the tree finger drag is not enough to return everything back, you have to configure that in separate control panel.
To get three-finger drag to work, you need a gesture right? So you report the issue to libinput. Libinput looks at it and says "well, this is more than just a gesture", and says "this should be implemented in the compositor". Now, this is desktop Linux, and now you need to get at least three compositor projects to agree that 1) yes it's their problem, it isn't a libinput problem and 2) to actually fix it in their compositor. Sigh. And what's more - this isn't even the end of the problem! This is just for dragging windows! Now you also need to get this for arbitrary drag and drop actions like moving text or stuff in your file manager, so now you also need fixes in your UI toolkits. There's at least two for that you'll need as well. Pile on top there that most maintainers in this web have never extensively used a Mac, and don't even understand why this is something people want in the first place. This kind of stuff is very hard to fix on the free desktop.
If the same discussion happens for macOS, at worst some manager who is the boss of both of the squabbling teams will get mad, tells them that the UX here is more important than your technical squabbles and orders them to sit down and fix it. To be clear: I completely understand how these layers of abstraction work on the free desktop, and this is much harder to fix. But I sure wish there was a stronger incentive for these abstraction layers to fix these cross cutting concerns.
I have some hopes pinned on recent progress for this [2].
[1]: https://bugs.freedesktop.org/show_bug.cgi?id=89999#c20
[2]: https://gitlab.freedesktop.org/libinput/libinput/-/issues/29...
There are other issues on the horizon - if input preferences are no longer held in X (because that's not libinput's job and it shouldn't be) but are pushed up to gtk/glib/qt/kde, does that mean that whenever I use a kde app in gnome that it uses some random key-shortcut defaults rather than what's in my gsettings? That seems to be the case right now.
Also, I owned a multitude of MBPs through the years, from the first Intel Al-books around 2005 through to my last MBP, a 2015 model. I was heavily invested in the platform because it seemed like that manager you describe had the same sense of taste and the same engineering intuitions as I do. Well, that all ended with the butterfly-switch keyboard. It was a sad, angry, bitter divorce, but I left Apple and will probably never buy another MBP. The butterfly switches are gone, but I know now that that manager lacks the integrity and professionalism to manage the platform. It was more important to fluff Johnny Ive's ego.
That opinionated manager can giveth, and she can taketh away.
That strikes me as rather optimistic. I mean, yes, in theory a company does have enough overarching management that it can force coherent action, but... the "Microsoft" in https://laughingsquid.com/organizational-charts-for-tech-com... is a thing. Maybe Apple is still sufficiently centrally-driven that it works there? Historical accounts certainly make it seem like they've managed their share of infighting, but I suppose by the time they went to market it tended to be dealt with.
As someone who installed Linux on a computer for the first time 22 years ago (Slackware via floppies!), but have never used it full time for more than a few days before I'd jump back to Windows/Mac, I'm amazed more every year how "we're almost there" is a recurring theme.
Yeah, but I think a lot of that was because so much dev effort has been wasted trying to make X11 suck less, and building infrastructure around X11. Between Wayland, libinput, and freedesktop, it feels like we're finally moving forward.
I don't think that the linux desktop will ever be 'mainstream' - there are too many issues with commercial software - but at least it can be a pleasant experience for the technically-inclined.
I forget which is the default.
The closest alternative (for me) on Ubuntu (Gnome) I've come up with is Super+J/K for switching between workspaces and, as a bonus, moving windows between workspaces with Super+Shift+J/K. Feels almost as productive as on MacBook with touchpad
I'm using three fingers swap for switching between workspaces. But I agree with the comment above, the animation is key.
Crystal-clear graphics with Wayland, great new hardware available from the two big linux-friendly laptop makers (Lenovo+Dell), great touchpad software support with libinput... it's a great time to be on desktop linux!
The UI on macOS to me feels frustratingly slow compared to XFCE. I've tried to disable animations, reduce motion and all the tweaks that can make it a little bit faster, but it remains being unacceptably sluggish.
And please abstain from downvoting unless you've at least tried XFCE and macOS enough in order to make an objective comparison.
I think this is the "animations vs no animations" argument, maybe - I like them as they give context to my actions, but I imagine some people don't.
There are many themes inspired on macOS if you are aiming for that kind of look.
http://reddit.com/r/unixporn has a lot of that stuff. You can just go and grab someone else's configuration and apply it to your system.
If you are aiming for window effects with no regards for performance, try Compiz. You can add as many window effects as you want.
I am no Haiku expert but the Haiku's window manager does not support window composition and does not use GPU acceleration, right? It is also not modular and cannot be replaced.
You can disable composition on XFWM and make it run a little faster on lower spec hardware. You can also use a WM like i3, Windowmaker or Enlightment.
IMO, XFWM is the right balance between functionality and performance.
It indeed does not use GPU acceleration; Haiku as a whole does not have 3D drivers either (but there are plans.)
I just tried this in Preview, Safari, and Chrome and none seem to actually do this.
OS X is like that. So many of the gestures are so natural that you just do them without thinking and I only notice when they're gone.
The contents of the screen will slide a few hundred pixels left, but since there is no workspace on the right to take their place, what slides in is just empty black space - which then immediately bounces back out and returns you to your screen.
The existence of this behavior very clearly communicates to the user that their gesture was received correctly but cannot be successful for some reason - far more eloquent than any dialog box with printed words or any sound effect.
1. Latency: If I drag my finger back and forth across the trackpad, I expect the mouse to match exactly where my finger is--not where it was 50+ms ago.
2. Resolution/Acceleration: I expect to be able to navigate my entire screen without lifting my finger and repositioning it. Further, I expect to be able to move the mouse one pixel by making a very small move on the trackpad (or by rolling my finger a little).
After those "table stakes" items are solved, then let's talk about adding gestures.
I really do appreciate the smooth scroll physics. But I really can't figure out why every raves about the gestures. I only ever trigger them by accident. I think I eventually turned them off, but only after screwing up a few times and thinking that I might some day actually want to use them.
Even on Windows, trackpad support with my giant Apple trackpad is pretty good. On Linux (at least with Pi OS and XFCE) I couldn't figure out how to get it to have any good acceleration curve or work at all like on my Mac or Windows laptop.
Scrolling was pretty smooth, but the cursor control was all over the place, so I had to pull out an old mouse for that day.
It's not obvious, but adding touchpad gesture support to X will benefit you too even if you're not using X.
This will make our work of convincing the maintainers of other open source projects much easier. X and Wayland cover essentially al Linux users, so we will not need to estimate how many users will benefit from implementing gesture support in toolkit X or application Y.
This is really important, because maintainers of open source projects don't usually care about suggested features if they're not interested in them themselves. A new feature means additional work to them - discussing the design, reviewing PRs and handling of eventual bugs. Often the person who implements a feature disappears after the PR is merged. This makes the maintainers to view all feature proposals with a grain of salt. We need to have a convincing story of how majority end users would benefit in order to make the contributions easier.
We'll see if that's true, or if we will need another round of sponsorship to achieve the same thing with Wayland.
If you look at the all of the next highest rated "features" they are not features but bugs. The reason that mac touchpads are so nice to use has _nothing_ to do with multitouch and everything to do with palm rejection, filtering, etc. It is a hard problem, analogous to and more difficult than writing multicopter flight control software. Having looked at the trackpad code, and fixing (in a very hacky way) some of the issues I was having, I have zero expectation that Linux laptops will get a mac like experience in the next couple years. Like all problems, it isn't a technical it is political.
Trackpad software and cut and paste are the two largest detractors for desktop Linux.
I was kind of just assuming that the trackpad issue might have been, at least partly, because the hardware macs use for trackpads was special. Are you saying that's not the case?
Also, what about cut and paste??
Could you explain how/why? Is it about specializing in specific hardware/firmware?
In open source often no one really cares about a suggested feature or even a submitted pull requests. It means additional work for the project maintainers - discussing design, reviewing code and handling of eventual bug fixes. Often the person who made a PR disappears leaving all future bugs the problem of the maintainer. So naturally project maintainers are risk averse and often say no to new features. Things end up not being implemented, not because it's technically difficult, or there's nobody willing to do the work, but because the incentives don't align.
The above opinion applies to open source in general as I've witnessed during the past decade. I'm not describing any projects that are related touchpad or input that I've been interacting with lately.
----
Because of sub-system ownership and crossing boundaries between window managers and input handlers. Just getting folks to recognize the problem and agree on how and where it should get fixed is the major hurdle. When software is overly modularized it both prevents many classes of optimizations and it forces certain communication patterns. Conway's Law [1] in reverse, teams will be split across component boundaries and solving Desktop Linux issues will require a level of coordination I am not sure the community is up to.
Writing the if the statements isn't the issue, coordinating and agreeing on the problem is.
That's not to say we can't do better and it's not worth making an effort. However, thinking Linux can match Apple in terms of trackpads is wishful thinking, IMHO.
I end up having to disable the touchpad and use the external mouse just to be able to type without accidental mouse jumping around.
#1 linux usability issue on laptops in my opinion.
Didn't the rest of this paragraph just explain how difficult a technical problem it is?
I've also personally been using multitouch zooming and scrolling in kde for years now.
It will be cool to see a simplified, 'just working', desktop independent multitouch system for linux though.
Ideally the input framework would know if the streamed gesture was consumed real-time, and if not (e.g. no support for such interactice gesture in some program), the one-off event is issued.
This reminds me of current xorg libinput two finger scrolling / wheel event. Xinput2 is the relevant keyword but I am not sure exactly how it all fits in, only what I can observe: - applications that don't know about multi finger scroll/pan listen for and accept classic mouse4 / mouse5 events and interpret them to scroll in steps if relevant. As an example, xev x event testing uility is not xinput2 aware AFAIK nor are classic x or older gtk programs - applications can be xinput2-aware (e.g. eog Eye of Gnome image viewer, but maybe also any non-ancient gtk3 application as well), in which case they can scroll more directly (pixel-smooth) and with appropriate acceleration / smoothing / inertia (gtk-specific ?). In firefox there's an env var like MOZ_USE_XINPUT2 which tells firefox it can do this smoother wheel handling, not sure if it's required or automatic these days. To test received events including xinput2, there is utility xinput --test-xi2
As a closing anecdote, there's an interesting interaction bug I have experienced with xfce where both xfwm will react to Super+scroll (compositor-level full screen zoom), and the application under the mouse pointer will also react to the scroll up/down. I have not deciphered the interactions here but it depends on app under mouse cursor...
Fusuma is not integrated with the display server, so it's limited in what it can do.
I'm not a huge fan of apple, but these kinds of small details are basically impossible to get right unless you've spent decades relentlessly deprecating old apis and consolidating them so there's only one fish in the barrel to shoot.
Perhaps the money that went into supporting this feature allowed them to overcome these challenges with much more ease. Unless someone who is more familiar with the source of MacOS can clarify otherwise I think it would be safe to say that this is not a new challenge, perse. Just new to Linux.
If you read the first two blog updates, they're straight out of FAANG PM-speak land. I can just hear the words echoing from a PM who vaguely understands an amalgamation of customer wants, without having any of the technical context of actual customer needs, architectural limitations, alternatives, etc.
And this is exemplified in the nearly useless survey data. It is so poorly sliced, and presented that it's hard to know what was learned. There's no axis for X/Wayland, which X input driver was used, what DEs were used, etc. All of these significantly matter in the day-to-day feel of the touchpad considering how libinput is configured.
Frankly, and this blog post more or less implies it, Wayland and libinput address nearly all of the original goals, and the ways in which they don't are either bugs in the drivers/libinput quirks that should be filled and fixed in libinput, or are simply the choices of GTK/Qt toolkits or DE designers and are part of a bigger design discussion that could help establish better defaults across toolkits+DEs.
I just don't see why people are putting their hopes or money on this project. I don't see current, useful-to-Wayland problems being identified or worked on.
Finally, a tip, for wayland + gestures, consider `gebaar-libinput`. And a solid net improvement to libinput (and up the stack) would be "stop scroll events". It's the only thing keeping the scrolling experience in Firefox+wayland+webrender from feeling just as good as Safari on a MBP.
There is also no single Wayland compositor they could focus on.
But while most users are currently on Xorg, the work going into improving responsiveness and suchlike is all going into Wayland. This project is really putting lipstick on a pig.
It's all about the toolkits, and some great work is happening in GTK, e.g.:
https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1562 https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/1117
I STILL feel uncomfortable using my thinkpad scrollpad in geany under linux, as it jumps and the scrolling isn't clearly indicating (through some animations i gather?) scrolling direction when I drag 2 fingers up and down...
Multi-touch, great!
...I just feel like basic behavior still appears to be an anti-intuitive, if admittedly equally logically viable (swiping the paper up or down vs swiping the scrollbar, metaphorically speaking)
https://public.amplenote.com/embed/TnE3JJDDy5QDo5aZAYtqyHz4?...
This plan has also has a huge flaw, that can be shortened to one letter: X.
> Add gesture support to the Xorg display server
> A "fun" fact: if all of this work was already done today, it would appear on Ubuntu as of next April's release.
I expect Wayland to be the default on even more systems by then, maybe even KDE could default to it? Though Qt has been weighting them down with the transition, there are less and less rough edges remaining.
So, this is probably going the hard way, for little at the end. It sounds like it caters more to the author's system configuration of choice (well, it's their money, though not really).
As the author said, Wayland has most of the bits in place, we mostly need to tweak toolkits into supporting this (Qt, SDL, wx, fltk, Gtk?, etc). Starting there would lead early results, and the code path might be reusable is X is worth implementing later.
On the other hand, X.org probably nears its final release, so if you want it to be burried with multitouch support, it's probably a good strategy to hurry.
Let's see who is left in the X ecosystem:
* GNOME: moved on to wayland, mostly, though strangely not with Ubuntu
* KDE: moving along, a lot needs to be done in Qt
* i3: sway is mostly compatible and quite nice to use (typing this there)
* XFCE/LXDE: not sure, I think the projects have merged into LxQt, which has wayland support comming along. XFCE itself meanwhile doesn't have a timeline for wayland, but I think it's on the horizon
* Openbox/fluxbox/Windowmaker: you're probably not into wayland. But there's a myriad of Wayland compositors that caters to the same
* firefox: starts to work quite nicely, has hw-accelerated video decoding now under wayland.
* Inkscape: supports it as of the 1.0 release
* Gimp: On the roadmap for 3.0 with the GTK3 port (same situation as Inkscape)
The missing pieces of the puzzle now are probably xdg-desktop-portal+pipewire support in the web browsers, that I often see people complaining about (and GNOME having their own screenshot interface doesn't help). That is coming. And the last piece, unified color correction with HDR support, I don't know that much about it to comment, but I've at least seen some work on HDR support for sway.