Firefox 75 on Wayland now to have full WebGL, working VA-API acceleration | Hacker News Reader

Firefox 75 on Wayland now to have full WebGL, working VA-API acceleration

phoronix.com

328 pointsnirv173 comments

There's a $380 bounty for this [1]. If you fixed this, please collect this. You deserve it. I hope this doesn't get lost just because it got fixed in a different ticket number.

[1] https://www.bountysource.com/issues/55506502-add-va-api-hard...

That's not a bad "tip", but keep in mind the person who worked on this (Martin Stránský) is employed by Red Hat to "work on mozilla packages (Firefox, Seamonkey, Thunderbird) in Fedora and Red Hat."¹

1: https://people.redhat.com/stransky/

Assuming he worked on it on company time, wouldn't that mean Red Hat could collect it?

for linux users, this is the best Firefox news in a loooong long time. having to copy paste links from youtube to smplayer for mpv hardware accelerated video can be a PITA.

Relevant comment for X11:

> (In reply to lilydjwg from comment #30)

>> I'm glad to here that, but what about support for Firefox on X11? Is it given up?

> Hardware decoding is platform independent so it generally works under X11 as it's implemented as Bug 1616185.

> For whole playback chain see Bug 1610199 what's needed to be done fox X11.

> I was closing this one as it refers to HW decode only, not the playback.

> The overall video playback is tracked at Bug 1210726.

https://bugzilla.mozilla.org/show_bug.cgi?id=1210727#c31

As a regular linux user should I switch to wayland from xorg? will I notice performance improvements?

Upsides: more secure (in X.org every client can read every other client's keystrokes, pointer position, and do screen grabs), generally smoother rendering, better support for mixed-DPI setups.

Current downsides: worse support for screen recording/sharing (e.g. not supported in Skype), a bug in the compositor (e.g. GNOME Mutter) can bring your whole session down, not-as-good NVIDIA support (also depending on the compositor).

I have been using Wayland for two years or so an wouldn't want to go back. I do miss wide screen sharing support, because I am working remotely and Skype is the primary means of communication.

Isn’t modern Skype using WebRTC that would be supported at least in beta with xdg-desktop-portal?

Did you try to use WebRTC PipeWire for screen sharing?

Yes, it works with some applications, like Jitsi. No luck with Skype so far.

Haven't switched yet. Under Wayland, can users which are not owning the session display an app on the Wayland session/server? (e.g. under X I can do something like "xhost +" and then have any user display an app on the X server). I'm not asking about the security implications of doing that under X (as you already mentioned, X is less secure and an app can read another app's screen content etc.): just if it's doable or not under Wayland?

Whoever has access to your wayland socket can display applications. Root already has access to all the things so for running apps as root (e.g. wireshark) you just need to configure sudo/doas/whatever-you-use to keep XDG_RUNTIME_DIR and WAYLAND_DISPLAY. e.g. in my doas.conf:

permit setenv { HOME LC_ALL TERM XDG_RUNTIME_DIR WAYLAND_DISPLAY } greg as root

and I can just run 'doas wireshark'.

Very interesting, thanks a lot. I'll definitely give Wayland a try soon!

What about running a GUI from a local or remote ssh session?

Eg previously I would execute DISPLAY=:0 mplayer foo.avi to play remotely or ssh -X root@foo xcalc to display xcalc on my local monitor but execute on rhost

To some extent you're looking for Waypipe, which aims to bring the X-style network transparency model to Wayland apps: https://gitlab.freedesktop.org/mstoeckl/waypipe

X forwarding works just fine with Xwayland.

> Upsides: more secure (in X.org every client can read every other client's keystrokes, pointer position, and do screen grabs)

What if i want this as a feature? Does wayland allow this per app?

The screen-grabbing app would have to interface with the compositor instead of the display server. For example, krfb is adding Wayland screen sharing support with KDE.

Is this better from a security standpoint? Does the compositor verify clients in a more secure fashion? Presumably screenshotting is still a thing on Wayland so this has to be a solved problem.

The point is apps can’t grab other apps input/output. If you run the app in an otherwise secure manner (eg execute in a perfect security chroot/container).

A malicious application on your host can still do anything at the executing privilege level (read .ssh, inject malware into .bashrc) but can’t sniff your gksudo password

Why would you assume it's a solved problem? Basically every compositor has to implement this feature.

The idea is that the compositor can then pop up a dialog saying "App X wants to record your screen. Allow?"

It could also limit recording to certain windows, or heck even redraw the whole screen any way it wants, like pretending sensitive windows aren't there.

Does this lead to a more fragmented application space? If each compostor required its own screen grabbing tool, and each DE has its own compostor?

Whilst backend implementations are going to be different per-compositor, they're exposed as freedesktop apis so tools shouldn't need to be adjusted for each one.

I respect what your saying, but I’ve seen that model tried and failed multiple times

Furthermore I kind of wonder if anybody bringing up this point has ever in their life seen or heard of a malicious X11 client ...

I know this is not how security works (a theoretical hole is often as bad as a practical one) but it's a bit like saying you should have antivirus software, to which I might say: where are the viruses?

If containerisation takes off and ends up being perfect security it’ll make sense. An example would be running Android on X, apps would be able to read / modify windows of other apps via The X socket.

Why would anyone suppose that containers on Linux which are designed to keep applications isolated enough not to interfere with each other ends up being perfectly secure when it's not really designed to be secure at all?

I’m not suggesting it’s sane particularly if your threat model is nation state level actors, but people use containers (chroot, docker, vms) as a security boundary all the time.

Neither chroot or docker is a worthwhile security boundary. People that act as if it is risk appearing incompetent when they get compromised.

I’d argue that anything but bare metal is not a good compromise, but I’m not balancing the books of a large company.

People care about docker escapes so they’re getting more difficult, at least.

Edit: Plus, I’d rather $app_i_dont_fully_trust be sitting in a container rather than on my host. And no, not running apps i don’t fully trust is not something which is at all viable for me.

> a bug in the compositor (e.g. GNOME Mutter) can bring your whole session down,

True for Gnome, no issue for KDE. Their arguably better design decisions led them to proceed slower though.

I've found wayland still has strange input, if there's a lot of CPU usage going on (something compiling or whatever) then the mouse and keyboard input will 'lag' and get stuck, plus it feels worse in games to me but the benefits seem really good, I hope wayland can keep improving so I can switch without it feeling worse for me

I use Wayland on a single-monitor desktop with a modern AMD card and Freesync, and I have had zero problems. I run all kinds of software and games (under Wine or otherwise) and everything just works out of the box.

I hear other configurations have more issues.

There is no screen tearing. That is quite noticeable.

X doesn't have screen tearing. Broken video drivers do.

Are all (Intel, AMD and Nouveau) drivers broken, then? I experience tearing on X with all of them.

X definitely has screen tearing, otherwise we wouldn't have the Present extension.

Core Xlib is entirely ignorant of vsync or anything relating to completed frames. It's an immediate mode rendering model in that everything you draw just gets thrown on-screen by the display server whenever.

Compositors were then bolted on w/redirect and damage extensions and basically randomly snapshot the client window contents before compositing a desktop frame, within that snapshot you can easily have tearing of half-drawn updates.

Further extensions and hacks have tried to fix this like Present and frame clocks from the gtk/gnome folks, but it's all opt-in by X clients and non-uniformly supported by compositing WMs or not at all in classical non-compositing WMs where the X server just draws things directly on-screen as the protocol requests arrive from clients.

These are all driver issues. Xorg and commercial servers on Windows don't have these problems. Intel and AMD drivers don't either unless misconfigured. It is not X's responsibility to handle low level hardware implementation details.

Everything I just described is implemented in userspace at the X protocol level.

You clearly don't know what you're talking about.

Every presentation or explanation I've ever heard by an X.org/Wayland developer (many of which are the same people) says otherwise.

There is nothing in Xlib (that I'm aware of) that has a notion of complete frames - i.e. no buffer swap, no "frame complete" signal. How would the server know when a frame is complete then?

Isn't this the point of the XDB extension that everybody has?

Never heard of that, I don't think it's commonly used.

The XDBE [0] extension attempted to add this, but I don't think anything actually implements it. (see edit)

I tried using it over a decade ago on XOrg and it didn't seem to do anything useful...

[0] https://www.x.org/releases/X11R7.6/doc/libXext/dbelib.html

Edit:

It's definitely there in the Xorg xserver [1]. My memory is fuzzy as it was a long time ago but I distinctly remember wasting a day trying to use this extension and becoming frustrated with its ineffectiveness. Maybe it was just for lack of any vsync mechanism to schedule the swap?

[1] https://gitlab.freedesktop.org/xorg/xserver/-/tree/master/db...

For KDE on X11, enabling the display compositor also seems to resolve screen tearing.

Performance was worse than X11 for a long time[1] and is finally catching up for some singular compositors. However most compositors still have worse performance and every single compositor has different quirks. Latency still seems to be worse across the board. In my test not one Wayland compositor could beat xterm input/output latency on X11 because of enforced global vsync.

1.:https://www.phoronix.com/scan.php?page=article&item=gnome-32...

Yeah, mutter/gnome-shell was pretty awful. Weston and wlroots (Sway/Wayfire/etc) never had the issues Gnome had.

In fact Weston since 2015 has had a clever delay algorithm that allows clients to hit the very next frame if they sync to presentation-time feedback:

https://ppaalanen.blogspot.com/2015/02/weston-repaint-schedu...

(And Sway has a configurable delay; since Sway is so simple to render, the recommended max_frame_time value is 1ms — i.e. the compositor has the very last millisecond before vblank to composite, and everything before that is for the apps.)

Not really fair to compare with non-vsync systems — tearing is completely unacceptable ;)

Why is X11 able to be tear free (if drivers a configured properly) and still allows single applications to disable vsync?

I still don't understand why I need Wayland.

You probably don't need it, but you can use it if you really want. If you're using a fullscreen compositing WM in X11 then single applications can't disable vsync, it's the same situation as with Wayland.

Wayland compositors can have something called direct-scan out, which passes the buffer for the full screen application to KMS, which should be superior to Xorg unredirection (Wayland doesn't "redirect" windows in the first place though).

Well, of all the monitors I have, I think one of them supports >60Hz refresh. Overwhelmingly (including on windows7) I've disabled compositing because it makes the machine seems laggy. A bit of tearing when I'm scrolling fast, or moving a window doesn't bother me. About the only time its annoying is low frame-rate video.

So, if you think about what the tearing is it makes sense, your getting a partially drawn frame because the application didn't provide the full frame in time for the vsync, waiting to display the update for and additional one or two frames simply to assure a complete frame can only slow things down.

Depends on the window manager you are using.

If you are coming from i3 the move to sway is a no-brainer and works great (sway is wayland only). Firefox, thunderbird, Libreoffice all work great and look great, i.e. sharp when you need scaling for example. Sway is to thank for base work they did for other distros like a wayland clipboard protocol all the others are using.

Second comes Gnome. No issues. Gnome laid the gruntwork of porting gtk3 to wayland which works great.

Third maybe KDE. Still two years to come. KDE plasma is suffering from arguably better design decisions like client side decorations vs. Gnome serverside decorations but as many users use GTK based apps those GTK apps do not yet play nicely with the KDE design decisions. As such the brave KDE folks have to implement things themselves while those relying on GTK are done. QT is fine with wayland but not everything is QT.

That's all only true if you are not using NVidia hardware.

FYI GTK apps look native now under plasma 5.18

> If you are coming from i3 the move to sway is a no-brainer and works great

It may work great, but it's not as well packaged by/for upstream as i3.

I recently started running wayland+gnome (rather than i3) on my Ubuntu 18.04 at work - because less bad support for fractional scaling looks a bit better on my two 24" with different dpi.

But there didn't appear to be a "no effort" way to install sway (yet). Especially not with automatic (security) patches.

I'm hopeful it'll be in 20.04?

Compiling sway is literally three commands (not counting cloning the repository).

Sure, the instructions are clear. But since there isn't a build ready for 18.04 - that kind of makes me wonder about the versions/state of dependencies too - that is - the packaged dependencies in 18.04.

They'll generally be frozen at the point 18.04 was frozen - so api and versions from 2017 or so. With 20.04 around the corner, I'd rather wait than faff around.

Sway is in 19.10 already, but 20.04 also adds some nice extras like wofi, wdisplays and mako.

> If you are coming from i3 the move to sway is a no-brainer

I was thinking of waiting another year to jump as I'm having no issues with i3, what are the advantages?

Sway has nicer configuration for multihead display setups than you ever get with xrandr, which is very nice on laptops if you often connect to different external monitors. There's also some clunky support for moving windows around with the mouse, which I've found to be occasionally useful.

Then there are the wayland advantages: very little screen tearing if any, no blanking the screen when connecting to a new display, better security properties.

There's disadvantages as well. There's no xdotool support, and if you want to use any sophisticated keybindings or remapping keys, you'll be forced to rely on Sway's configuration language to do that for you. On i3 I was able to bind ctrl_r to right click, but that is — as far as I know — not possible on sway without rewriting the libinput event stream, which is very painful.

Can you use `bindsym ctrl+r seat - cursor press button2, seat - cursor release button2`?

Sorry, I was unclear. By ctrl_r I meant the right ctrl key, pressed on its own with nothing else. Binding Ctrl+r might be possible. It's also been about a year since I last tried, so there may be a solution now.

  There's no xdotool support

Have you looked at ydotool?

https://github.com/ReimuNotMoe/ydotool

Also, do you know about the "i3 Migration guide"?

https://github.com/swaywm/sway/wiki/i3-Migration-Guide

Given the need to access /dev/uinput as a, user doesn't this require negating the supposed security gains of wayland?

Also not exactly maintained

>Since Jun, 2019, I have little time to maintain this project because I'm striving to start an undertaking (instead of working 996).

You can implement a xdotool's equivalent by using virtual-keyboard and virtual-pointer Wayland protocols.

Sorry, but you seemed to have missed answering the OPs question... Why they should switch to Wayland? If there were any benefits to using Wayland, like performance improvements, etc.

Personally, the one real gain I've read about previously is no tearing when watching videos or scrolling fast and security improvements. Also now these Firefox performance improvements. But that's about it.

Are there any other benefits that a normal user would see?

>Personally, the one real gain I've read about previously is no tearing when watching videos or scrolling fast and security improvements.

Finally, my Ubuntu box has screen tearing all the time. I'm on good hardware with pretty standard 1080p monitors so I've been absolutely baffled that screen tearing is so frequent in standard desktop use.

I think Wayland is supposed to have better support for various DPI settings and mixed DPI settings.

Having a compositor for X11 will help with the tearing some, but not completely. This will be a nice step up once Wayland is ready in other ways (mainly more window managers and a few other tools that don't quite have equivalents to their X counterparts yet).

Any program using X can see the keystrokes entered into any other program using X. This has been bothering people for many years, especially those who have to use untrusted proprietary applications for work reasons. Wayland doesn't have this issue.

If you have an app that is literally attacking you then you are probably compromised. Even with wayland technology to contain hostile apps is grossly insufficient for the use case.

I would be surprised if this weren't as true in 2030 as it is in 2020.

The OS with the best isolation primitives is probably Solaris. It would be hilarious if 2020 Solaris was still better than 2020 Linux.

Defense in depth.

>If you have an app that is literally attacking you then you are probably compromised

Sure, if your adversary is some literal hacker. But you also have to worry about legit companies that use aggressive "analytics". VSCode most likely logs keystrokes. It would suck if it also logs keystrokes that you type into other applications.

VSCodium is great. It has all the MS junk removed.

https://vscodium.com/

Your point still stands, though.

Here are the real, user-visible benefits that I am aware of, at the moment:

  * Fractional scaling of monitor outputs
  * Multi-touch gestures for e.g. switching desktops
  * Security sand-boxing of apps
  * Tear-free video

As the original link shows, though, the architecture has much better abstractions for software engineers to work with so developer quality-of-life is improved, too.

I forgot the most important one: per-output scaling (integer or fractional). This allows you to have your HiDPI laptop display and an external 1080p monitor set to different scale factors.

>> Sorry, but you seemed to have missed answering the OPs question... Why they should switch to Wayland?

Because X is deprecated. The question isn't Why switch to Wayland, but When. You have mentioned some benefits too.

Plasma has too many knobs to tweak and their settings menus are too modular, so it’s hard to turn those knobs (there are like 4 keyboard shortcut panels for example). IMHO, I find it easier to be keyboard driven in gnome than plasma (at least, as of a year ago)

How is this related with Wayland? I prefer having checkboxes then a regedit or gnome gconfig.

About shorcuts, I also like having the GUI there to configure the keyboard layout (like set capslook as Escape) rather then edit xorg configs. Also it makes sense to have a section for Global shortcuts and a section for regular shortcuts. If you do not know what Global shortcuts are maybe you don't need them.

Probably GHOME users complain about Firefox's "about:config" because it has too many options and their brain can't fit that many options. The point is the distro gives you some defaults and if you are competent(you know to read and use google) you can change the distro defaults. if you are not competent you find the distro that looks cool and use it as the developers intended and adapt to it not it to you.

Anything that's shown in main settings has to be QA'd and officially supported, often at a commercial level.

Anything hidden in Tweaks or about:config like settings can be classed as "here be dragons".

That's one reason GNOME has less settings (not the main one). Less Settings = less to test, less to support.

But why are some GNOME fans get irritated and irrationally in this case complain about the fact that some developers are more capable of supporting more settings. Why you as a user are bothered that application X has options for people more capable then you of using them.

I am a developer and I understand the part of more coptions and more code paths means more things to test, but with robust code it is not a big deal. Imagine the wallpaper would be hard coded because the developers are lazzy and won't test with a different wallpaper. As a person with dsabilities I contributed a few patches and it would suck to have them rejected because I would need to bring a proof that there is a majority of users that need those things addressed.

Anyway i was not complaining about GNOME removing options or hidding them but complaining that some insecure person for some reason had to complain about KDE into a topic about Wayland support.

Also worth mentioning: Wayland has no support for color management at the moment. Work has been ongoing on this for about a year now, but I think it hasn't really solidified yet into something you can use.

This is the biggest show-stopper for me. Color management is a minimum for usability.

I'm using i3 right now instead of sway, but even in that case all you've said is that I won't lose too much by the transition. You haven't mentioned anything (except scaling, which I don't need) that I have to gain. I'm open to new info, but what's the benefit of wayland for me?

I am not an X/wayland dev, but:

I don’t think it’s supposed to be an upgrade from the users perspective, just a new display driver. Maybe a few new native things like rotatable windows etc which if exist in X11 are probably emulated.

X11 by modern standards is a shitshow of dirty hacks, I spent the weekend researching ‘how can I not have notifications pop over my lock screen using openbox and any composite manager’ and it ended up in the top hard basket. Also repeat this story for screen tearing, DPI, acceleration, weird bugs, etc etc.

Wayland is maturing which is nice, but you really won’t notice many changes, but youll be much cleaner / less buggy in the long run.

The way I see it, Wayland itself is better in some ways but not great. For one thing, it's so secure that things like menus can't show outside window borders - hence, case-by-case handling of exceptions to the security model. Read: dirty hacks built in from the start. I think clipboard handling also falls under this category.

Edit: to me, it seems like wayland isn't fundamentally better. It's just a "new and improved" version of the same system architecture paradigm X is built with. I'd be more interested in something that follows the 'everything is a file' paradigm that made Unix so great to begin with.

> For one thing, it's so secure that things like menus can't show outside window borders

This isn't true.

Although Sway is great, they may miss out on certain features they'd need like screen sharing, especially for applications like Zoom which is used by many enterprises.

When I left Linux for Mac a few years ago, the Wayland situation was about as you describe it. Cannot say I'm surprised.

I want to, but I use two computers at my desk and run Synergy so I only need one keyboard/mouse. (bought a copy twice) Synergy does not yet support Wayland. I really hope they come around and support it soon.

Another question: I don't take advantage of X network transparency all the day every day, but it is very handy when it's needed. Will there be an equivalent in the Wayland (or, should I say, post-Xorg) world?

https://gitlab.freedesktop.org/mstoeckl/waypipe

I find lack of remoting to be a showstopper for a lot of apps that belong in datacenters, but a web browser is one of the few where it's reasonable to always run on your desktop (attached to the GPU and display, which Wayland can't live without).

https://github.com/udevbe/greenfield

This is kind of a joke answer but the real answer is: This depends on the compositor you're using. So Wayland may not be network transparent, but it's possible to implement equivalent functionality on another layer of the stack.

Do keep in mind that just because you might be using X11 doesn't mean you get any network transparency at all. The network transparency only "works" if the GUI applications you're running actually use X's 2D rendering system. Which many things don't do because it's old and terrible and slow. QT5's scene graph backend, for example, doesn't have a X renderer anymore. So those GUI apps already lost network transparency (and did anyone even notice?)

On the other hand it turns out streaming video is super easy, entirely agnostic of the actual rendering stack in use, and not really that expensive in terms of bandwidth anymore.

that is not really an answer.

regarding the video streaming: meh, i can do X forwarding out of the box with just ssh, mostly out of the box, no other software required.

Does each individual application have to convert to wayland, or does wayland allow you to use old xorg apps without modification?

Yes and no.

- Apps which use toolkits like GTK and QT will “just work” since they did the porting grunt work.

- X11 apps can run unmodified using XWayland but this just runs an X.org server and directs the output.

But the literal answer is “no”. Wayland and X11 are incompatible and apps will need to port.

No more screen tearing is a nice one.

If you use a distro with up to date kde/gnome you may want to. If you use i3 you may want to try sway.

If you use Nvidia gpus don't bother.

If you use a nvidia GPU just sell it and by an AMD one. You will save yourself a mountain of headaches.

I run clamshell mode on a regular dpi monitor and the laptop is hi dpi. Everything rescales when switching between clamshell mode and laptop mode on Wayland (Sway). I tried to get it to work as well with xorg i3 and xrandr but I couldn’t. I would always need to close everything and restart my workflow when switching between modes.

As a regular linux user, no, don't worry about whether you're using wayland or xorg, just use whatever comes with your distro. Let the maintainers worry about those things and just be a user.

What's missing so far is VAAPI support for WebRTC. Apparently, Firefox is using same libwebrtc as Chrome does, and it still doesn't support hardware acceleration.

This results in some ugly behavior of WebEx for instance, which blocks video in its WebRTC calls, since it assumes that "CPU is too weak" to handle it. Cisco never fixed this issue.

The mention of DMA-BUF makes me suspect that it involves rendered video being copied from the GPU to the main RAM and back to the GPU, which wastes energy. Does someone have details about how the integration works?

The linked blog post makes it reasonably clear that this isn’t happening.

https://mastransky.wordpress.com/2020/03/03/webgl-and-fgx-ac...

It's the opposite. The point of using DMA-BUFs is that those are the handles which are passed around, and the data never leaves GPU memory. The handles can be directly accessed from EGL/Vulkan and used as render source/targets.

This is quite unclear to me. Is DMA-BUF so generic that the memory it refers to can be in different types of RAM, including GPU RAM and CPU RAM?

Yes.

That is the whole point, one of the original use cases was passing around buffers between different GPUs on dual-graphics laptops (PRIME)

Do you maybe know, when frames rendered by GPU 1 (e.g. fast GPU) need to be sent to GPU 2 (e.g. slow GPU doing the compositing and outputting to the monitor), how does the data actually get transferred? I can imagine the following possibilities:

1) GPU 1 writes to CPU RAM, GPU 2 reads from CPU RAM

2) GPU 1 writes to GPU 2 RAM via PCI Express (DMA between devices)

3) GPU 2 reads from GPU 1 RAM via PCI Express (DMA between devices)

AFAIK, GPU 1 writes directly to GPU 2 via PCIe.

Since GPU 2 is most often an Intel iGPU, that happens to also be CPU RAM.

It's supposed to refer to an object that is in GPU RAM. You can map it to CPU RAM to access it, but obviously that is slow. The userspace function you normally want to call to do this is "gbm_bo_map".

Does this work on Gnome i.e. with official Nvidia drivers?

At this point Wayland is nothing more than a 10 year old, over-hyped, tech-demo. Why should Nvidia care about sub 1% of a sub 1% market?

I use Wayland with GNOME all day every day and it works fine. There are still minor issues versus legacy X, but it's certainly more than a "tech-demo".

Is it finally possible to share the screen on a video conference call?

Unless it's changed in the last month or so, no.

Exactly nothing changed last month, because it works for quite some time. See https://jgrulich.cz/2018/07/02/screen-sharing-support-in-web... and https://jgrulich.cz/2018/07/04/how-to-enable-and-use-screen-... and https://twitter.com/tomaspopela/status/1105740090784006144. Also we are currently working on some improvements to improve the UX - https://chromium-review.googlesource.com/c/chromium/src/+/19...

I use Sway but what I do is use Google Hangouts in Chromium (Xwayland) instead of Firefox (Wayland) when I want to screen share. I heard Zoom doesn't work yet though.

If the software supports it, yes. Most commercial programs have not been updated to support it yet but the feature is there.

Does VNC or RDP work now? That's what is keeping me back.

Yes, both Gnome and Sway have VNC: https://wiki.gnome.org/Projects/Mutter/RemoteDesktop, https://github.com/any1/wayvnc. Weston has RDP, and there are efforts being made to also add VNC.

I don't need VNC or RDP, so I don't know. I know that sharing other windows or my whole desktop in Google Hangouts doesn't work, but I rarely need that. I figure I'll log back in with an X session if I need that.

Anybody with any sort of graphical expertise on Linux with the exception of Keith Packard is working on Wayland, not X. Wayland is still the future.

Seems like an out of touch opinion, I'm using sway and very satisfied with it.

If you use sway (an i3 clone), how can you say something is out of touch? If anything your use case is absolutely in the minority

Sway is a pretty popular project, and gaining momentum. I also use sway for all of my machines (including workstation and personal laptop) and am quite happy with it.

Someone who uses their GUI as the worlds most over-engineered tmux session should be careful about calling others "out of touch".

This is a strange opinion to express in general, but really strange on this news. Linux hasn't had hardware video acceleration in a browser for over a decade when Windows and macOS did have it, precisely because it was too hard to support this use case within the restrictions of X11. The buffer sharing trickery, combined with having to render web pages and a browser around it, was just too hard to solve for years (within the limited attention X11 based desktops get in general).

Linux finally has a graphics stack on which this isn't as much of a problem, and it's received enough adoption that Firefox is now making changes for it. And on that news you decide to comment that Wayland is a tech demo? This news should tell you it's completely the opposite! Wayland seems to be finally delivering.

Parent is kinda right. Wayland was supposed to be this great next-gen display protocol for Linux, finally doing away with the cruft of X and thereby making development faster and easier while bringing modern features and yadda yadda. Instead it has taken 10 years to kinda-sorta catch up to X and is still behind its equivalents on Win/Mac.

Wayland has a lot of problems, but they're adoption problems.

1) Wayland isn't a replacement for all of X.org. It's a replacement for much of X11, the protocol. "Wayland is just a protocol" - there's no wayland binary on your computer like there is an xorg binary. All the problems/limitations people commonly perceive with Wayland are actually problems/limitations within Wayland implementations like Mutter (Gnome) or KWin (KDE) and lacking application support. There's really not that much that "Wayland" as a project could deliver as it's just a protocol [1]!

2) Replacing something like your entire graphics stack requires a widespread, coordinated effort to unify the various Linux distro's under it. That widespread coordinated effort is not actually possible in the (by "design") extremely fragmented Linux market. The only approach is to slowly let Wayland implementations evolve, and there we have it.

Hot take: Wayland will not fully replace X11 the coming 10 years. There is no fundamental solution to the two problems above. Many popular desktops (Xfce, Cinnamon) have no plan to support Wayland at all. Some don't even want to. In the future, there will be choices for graphics stacks on Linux leading to further fragmentation.

[1]: The projects that should deliver here are: Gnome, KDE, other open source desktops. Although I think using Gnome with Wayland is superior in many ways to X11, it still has certain quite major shortcomings. It's also application toolkits: Qt and Gtk work luckily, but Electron is waiting for some Chrome renovations before supporting Wayland, and then there's stuff like Wine and Java where the refactor/reward balance simply doesn't favor them. And sometimes applications have to be updated to stop relying on using hardcoded X11 stuff, and use agnostic Gtk/Qt/Freedesktop functionality.

There's a lot of history with X11 that we have to move away from, and the incentives and coordination only allow for slow movement.

> Wayland has a lot of problems, but they're adoption problems.

No they are deep philosophical problems. The lack of necessary protocols that allow a unified implementation for things like screen sharing, hotkey daemons, window managers, xdotools or clipboard managers are the problem.

As long as there is no standardized way to do essential tasks that are typical for a desktop use case Wayland will eternally cause problems.

You're not describing a philosophical problem of "what's possible", as all of these things are perfectly possible on a desktop with Wayland and broader Freedesktop standards, and many have working examples currently. Wayland being as small as it is, is not supposed to be a restriction on features of your entire desktop.

But there's no standard interopability with a standard X11 socket anymore. There's interopability with KWin and Mutter and wlroots and a bunch of other things through FreeDesktop standards (Wayland being among them).

You might consider it a philosophical problem that Wayland is not trying to solve all the problems that X11 tried to solve. You might attach a lot of philosophical value to that architecture. But Windows, macOS, ChromeOS and Android seem to be doing extremely well on an architecture that is a lot less like X11 and a lot more like Wayland.

> things like:

> screen sharing:

- Security problem, untrusted access to display, explicitly denied by design.

> hotkey daemons

- Security problem, untrusted input, denied by design.

> window managers

- Messing about with windows without authorization, denied by design.

> xdotools

- Untrusted input.

> clipboard managers

- Untrusted access to clipboard.

Either the compositor application has to do it, or it has to delegate it to a trusted application.

So yeah, deep philosophical problems. If you wanted Wayland to work exactly like X11 it won't because that is explicitly against its security design.

All these security arguments are pure figments of your imagination. They are not based in reality at all.

Wayland is trying to do less than X11. The things you mention are not solved at the display protocol level on any platform but X11. Your security strawman never comes in to play.

> Security problem

Yeah, it's not like anyone needs that functionality to do work or anything. In fact, why not just refuse to boot the computer in the first place? That'll stop a lot of security problems.

Thanks for clarifying more of what Wayland is. I was really hoping for a better remote desktop solution on Linux and had high hopes Wayland would deliver on that. My understanding is it doesn't, but is that something a compliant implementation could add as extended functionality? Or is that all wishful thinking?

Sometimes I boot my Linux native installation as a raw disk VM in Windows 10 with VMware Workstation and then use Windows's remote desktop. That is surprisingly more responsive and a better overall experience than either VNC, X2Go, or X11 forwarding can deliver. I'd love to see something on par for Linux desktops.

This is possible, but it requires application support. As a user it is therefore extremely painful and limited still. Standards and libraries for this have only recently become available, and only the first applications are adapting towards it. The nice thing is that these new standards and libraries should support both X11 and Wayland equally as I understand it.

Why is this taking so long on the """Wayland desktop"""? Screen sharing/remote desktop is a problem that Wayland as a protocol does not try to solve (this problem is also not solved by the display protocols on any other platform for the record). And because of that, it requires central coordination in the Linux world to standardize solve. And central coordination in the Linux world always turns out to be a total mess.

> All the problems/limitations people commonly perceive with Wayland are actually problems/limitations within Wayland implementations like Mutter (Gnome) or KWin (KDE) and lacking application support. There's really not that much that "Wayland" as a project could deliver as it's just a protocol

In practice if all/most implementations are buggy, I think it's fair to lump them all together and say that Wayland (as used by the software actually running on my machine) has problems. You know the meme about how whenever someone tries Desktop Linux and has trouble, someone pops up and says "just use this other distro, with this particular set of software and this exact configuration, and it'll just work!"? Wayland feels like that. Sure, if I want to go all in on GNOME (on Fedora), it'll probably work. Sure, if I use Sway with the right set of compatible programs it'll work. But... I could also just stay on X where all my programs work now with no tinkering, where I can drop in any screenshot program, any clipboard manager, any screen sharing program, any keyboard configuration/emulation tool (I'm really fond of xdotool), and it will work.

> In practice if all/most implementations are buggy, I think it's fair to lump them all together and say that Wayland.

Fair enough. I think it's a bit unfair to label the collective failure of the Linux community/ecosystem to actually make change happen as "Wayland has problems". But it's true that in 2020 you will still lose use cases when using anything Wayland based.

Wayland will never replace the X11 based workflows you describe though, because it isn't X11. Although X11 has a lot of fundamental problems, flexibility isn't one. And some flexibility you will have to give up moving to any Wayland based desktop.

Which is yet another case of fix the problems with the existing stack rather than creating a new one.

Yah, you don't get to be a hero, and product arch, etc for making X work better. I have yet to see a convincing argument that X couldn't have been made more secure by bolting on another opt-in extension. Then once in place, like GL ES, you start deprecating all the features used by basically no-one.

I don't understand your post, nobody mentioned security in this discussion?

One of the big issues is there is a heap of legacy cruft in the existing X Stack which makes trying to fix/enhance X difficult. From what I understand the way it is written not easy to cleanly depreciate certain features like you've suggested.

So I can totally see the appeal for a clean break from having to maintain strict backwards compatibility with a stack that has its roots in the 1980's.

Wayland has been the default on Fedora since Fedora 25, released in 2016. I think replacing X.org in 8 years is pretty impressive, given that many desktop environments and too some extend toolkits have been tightly coupled to X11 in the past.

Did Compiz on X11 not have hardware acceleration? The wobbly windows and other effects on my old Atom powered Acer Aspire One were buttery smooth. If that dinky processor was pulling that off without help from the GPU, that's really impressive.

You're talking about something else. Hardware acceleration of 3D graphics is something completely different from utilizing the specific video decoding hardware in your GPU to decode web videos without hitting your CPU. Firefox has had generic GPU hardware acceleration on Linux for a while (albeit more buggy than on Windows/Mac), but this is about hardware video decoding.

How do applications like Kodi manage to overlay accelerated UI on accelerated video but Firefox can't?

The way I understand it: Dedicated video players can build around hardware acceleration. They display a video and UI they control. Web browsers have to show other people's UIs, which can be dynamically resized, overlaid and otherwise modified.

This makes it really hard to implement in current browser architectures without resorting to a solution that causes you to e.g. have to copy video frames from GPU memory to main memory, causing performance drops and stutter. With Wayland, it's "easier" to actually access the GPU's video frames directly (like on Windows and macOS), enabling this implementation.

Firefox does more than a simple overlay.

Compiz/beryl exists since 2006 which is 14 years ago. And this was really impressive working software, not just a demo. So impressve in fact, that it made many people switch to Linux. Windows Vista with a truly 3D accelerated desktop came after that.

Also this "news" does not tell that Wayland is finally delivering. It tells us that the Wayland implementation of Firefox has reached feature party to X11 for very specific subsystems.

You're completely misunderstanding what this is about. Hardware video acceleration is not about rendering 3D graphics.

This is about being able to play web videos without causing you to lose CPU performance and battery life. This has not really been supported on Linux or X11, whereas it has been supported on Windows and Mac by almost all browsers for over a decade.

The fact that this will be officially supported by a major browser vendor on Wayland first should tell you something.

> Hardware video acceleration

All video players on X11 have VDPAU support since ages. This is a Firefox specific problem which could be solved more easy on X11.

> The fact that this will be officially supported by a major browser vendor on Wayland first should tell you something.

It just tells me that Mozilla/Firefox really doesn't care about Linux because it took them 14 years since VDPAU works reliably on Linux to implement it.

Gnome Web, Midori, Falkon and Konqueror are browsers that only run on the "free" desktop, and don't support VAAPI/VDPAU either. Google built an entire OS based on Gentoo, and the desktop Linux build of Chrome doesn't support video acceleration either.

I think you should reconsider how easy it is to smoothly get an accelerated video buffer from a specific GPU component into an already heavily GPU optimized and dynamic web page. Web browsers have a lot more complicated graphics rendering problems to solve than VLC does.

> This is about being able to play web videos without causing you to lose CPU performance and battery life. This has not really been supported on Linux or X11

This is simply not true. I've regularly used hardware accelerated video on linux+X11 for over a decade. There are multiple ways for video playback to be accelerated, including hardware-specific video acceleration libraries like VDPAU and VAAPI, and hardware-agnostic acceleration using OpenGL (GLX). All of these options accelerate generic video playback steps like colorspace conversion, scaling. Most of these methods also provide direct hardware decoding of specific video codecs (h.264, mpeg1/2, vc1, wmv3).

Also, just like Wayland, hardware accelerated compositing (2D, not 3D!) of windows has been possible with X11 for more than a decade. Even my ancient window manager (Enlightenment DR16!) can manage X11's compositing.

Wayland was never required for any of these features.

Did you notice I said "web videos"? I'm talking about web browsers. I'm aware that mpv and vlc and others can use video acceleration okay, but they don't have complicated browser UI problems to solve.

Yes, I did notice that. There is nothing preventing a web browser from using the same methods and libraries.

> they don't have complicated browser UI problems

While not as complex, they do have a UI that overlays the video. However, the complexity of the UI doesn't affect video acceleration. Using libvdpau as an example, the X11 client provides[1] a separate child window[2]:

>> VDPAU expects to own the entire drawable [...] it is recommended that applications create a dedicated window for the presentation queue target, as a child (grand-child, ...) of their top-level application window.

The video playback happens in its own sandbox. Other non-video part of the UI should use their own child windows, isolated and independent of the video playback. The UI's child windows can be as complex as necessary, including overlaying the video[1]:

>> Applications may also create child-windows of the presentation queue target, which will cover any presented video in the normal fashion. VDPAU implementations will not manipulate such child windows in any fashion.

The child windows can even be alpha blended[3] with the video or other UI layers.

Reading back video data (to implement features like CanvasRenderingContext2D.getImageData()) is easy: read the video window like you would read any X11 Drawable.

What, exactly, was the "complicated UI problem" that supposedly wasn't possible to implement in X11?

[1] https://vdpau.pages.freedesktop.org/libvdpau/group__api__win...

[2] "window" is used in the X sense: a drawing context on an X server: https://www.x.org/releases/X11R7.6/doc/libX11/specs/libX11/l...

[3] X Render Extensison, section 8 https://www.x.org/releases/current/doc/renderproto/renderpro...

If it is all so straightforward, what is your explanation for no web browser on Linux officially support video acceleration? Not Firefox, not Chrome, not Gnome Web, not Falkon, nothing. Despite video players just doing it. Just possibly, could it mean that it's actually not that straightforward in a browser?

By the way, if you would read the actual base of the story here, you would know why it took Firefox's Wayland refactoring to actually pull this off [1].

[1]: https://mastransky.wordpress.com/2020/03/03/webgl-and-fgx-ac...

> Linux hasn't had hardware video acceleration for over a decade when Windows and macOS did have it, precisely because it was too hard to support this use case within the restrictions of X11.

mpv and kodi have no trouble rendering hardware accelerated video on Linux under X11 though

They are a much simpler use-case. They just need to present video. Web browsers need to composite the video within arbitrary documents which may e.g. have elements that overlap the video (opaque or transparent) or may arbitrarily transform the video (CSS applies to videos just like any other element).

> Web browsers need to composite the video within arbitrary documents which may e.g. have elements that overlap the video (opaque or transparent) or may arbitrarily transform the video (CSS applies to videos just like any other element).

both Kodi and MPV have UI overlays that are composited on top of the video. Also, hardware decoding APIs (vdpau, vaapi...) are mostly unrelated to rendering - see for instance https://github.com/NVIDIA/vdpau-hevc-example/blob/master/mai...

Well, the simple answer is: not likely.

In order to run Firefox under Wayland, you have to be able to run Wayland session in your compositor (Clutter in case of Gnome). AFAIK, currently no compositor has working/stable Wayland support with NVidia proprietary drivers due to GBM/EGLStreams thing[1]. At the same time, open-source Nvidia driver Nouveau is in a terribly poor condition. Which, hopefully, may change soon[2][3]. Thus, if system is running off the Nvidia graphics card with proprietary drivers, there's no way to use Gnome Wayland session and Firefox 75 acceleration at the moment [corrected and specified]. Please correct me, if I'm wrong.

A separate case is where the system uses Intel/AMD graphics card and drivers, and NVidia is used for render offloading (Optimus/PRIME/Bumblebee). This may be a laptop with discrete GPU or Virtual Machine setup with GPU passthrough. Since I don't have such a setup, I can't check if it works with Firefox 75 [corrected]. But there're some discussions[4] on the matter and experimental works on implementing NVIDIA VDPAU-VAAPI wrapper[5].

[1] https://blogs.gnome.org/uraeus/2019/04/03/preparing-for-fedo...

[2] https://www.forbes.com/sites/jasonevangelho/2019/12/06/nvidi...

[3] https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-O...

[4] https://old.reddit.com/r/archlinux/comments/a94zzj/state_of_...

[5] https://packages.debian.org/stretch/vdpau-va-driver

Moral of the story: do not buy NVIDIA, except if you need it for e.g. CUDA.

My previous machine had an NVIDIA GPU and despite all the great work, nouveau was almost unworkably buggy. The proprietary NVIDIA drivers did not work with Wayland at all. I bought an AMD card, Wayland worked with open source drivers and everything was butter-smooth.

I now use a NUC with an Intel GPU. Similarly to ADM GPUs, it works great with open source drivers.

Yeah, sorta-kinda. If only AMD would be able to compete timely in the higher end GPU segment. It took AMD 2+ years to release Radeon RX 5700 XT (Jul 2019)† to match (with a very good stretch) Nvidia GTX 1080 TI (Mar 2017).

I support AMD and Intel for their open-source efforts, but still hold hopes for GTC 2020 announces.

† AMD Radeon VII was super overpriced and unavailable in my area, and it still released nearly two years later.

I'm not sure about where you live, but where I am, the 5700 XT is priced similarly to an Nvidia 2060, which it is very competitive with.

I specifically address higher end GPU segment, e.g. current RTX2080/RTX2080S/RTX2080Ti. If you want/need top-tier performance in consumer GPU market, in recent years AMD had generally nothing to offer in two years window. I afraid that this trend may continue with the AMD's "Big Navi" and NVIDIA's new 30-series. I hope Radeon will finally catch-up this year.

AMD Radeon is an amazing choice in low/mid GPU market segment though.

I'm really curious about the kind of workload you're hitting that GPU with. Because from what I gather, you change your GPU every two cycles ?

Do you use it for a specific professional application, or do you simply want the best performance ?

Not intended to be snarky just purely curious.

Ha, good question for a casual self-reflection. To be honest, I certainly do want the reasonably best performance: I find pleasure in the possession of technologies, which allow me to avoid being distracted in my work and leisure activities. But I always justify the practical outcome and amount of money I'm able to invest (e.g. RTX 2080 Ti is a no go). As for types of workload, in fact, nothing too fancy:

- Gaming. I expect playing current AAA-titles on 4K@75-100fps. And, oh boy, I'm an avid Linux gamer (Valve Proton/DXVK)! This is probably the most GPU performance consuming activity in the long term;

- GPGPU/ML/Hashcat/Pyrit/mining. Some ML hobby/educational projects, occasional hash-cracking, mining and other fun activities. From time to time I stumble (on Github or elsewhere) at weird, but interesting experimental CUDA/OpenCL-accelerated solutions for a variety of tasks. Hey, even my terminal emulator is GPU-accelerated nowadays[1];

- Video transcoding. Encoding 4K+ video content in the background, while keeping up with my general activity. Occasionally lending my GPU to a (professional video guy) friend for CUDA-accelerated DaVinci Resolve for A/V post-production.

I certainly don't upgrade GPU every two years, more like 3-5 years for every decently improved generation. As for Nvidia, going from 16nm EV (Pascal) to (supposedly) 7nm EUV[2], might yield in up to +40-60% performance comparing to current gen Turing; so I'm pretty hyped to upgrade from the already aging GTX1080Ti with no ray-tracing. Or perhaps AMD will be finally able to pull out some ground-breaking GPU performance boost with their so-called "Big Navi", based on the new RDNA2 architecture. But I'm afraid that it will be a competitor for the current gen of Nvidia graphic cards, and not future ones. In any case, 2020 promises to be interesting for GPU/APU market. As usual, I will look at the specifications, benchmarks and price tags — and make my consumer choice.

[1] https://github.com/alacritty/alacritty

[2] https://www.sammobile.com/news/win-samsung-nvidia-7nm-proces...

Or you could run X11 instead of Wayland if you have an nVidia card. That configuration also works well.

To be fair firefox 75 is the latest soon and had a ton of Wayland improvements since the version you are referencing.

When you are using wayland best is to bet on a roling distro as of now.

Oh, that was a double typo, thanks.

btw I use archlinux /s

I believe that's tracked in Bug 1210729, which is included in the overall hardware video playback tracking Bug 1210726.

https://bugzilla.mozilla.org/show_bug.cgi?id=1210729

https://bugzilla.mozilla.org/show_bug.cgi?id=1210726

does this mean playing video will not use half of cpu available (at best) constantly on linux now?

Reply on news.ycombinator.com