Hacker News Reader

Comment by elric | Hacker News Reader

> [...] reducing technical debt, improving security practices, etc

Reducing technical debt by rewriting everything from scratch (almost?) never works. It's the baby and the bathwater. It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues. The new, conceptually beautiful system always fails to capture that embedded knowledge, and ends up breaking many things in many ways for many people.

Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does.

View on HN

> It's the sort of thing that seems attractive to junior developers

You make it sound like Wayland has been pushed by some script kiddies, when in fact the lead developers on Wayland are the actual Xorg maintainers.

To me, the migration from X11 to Wayland looks similar to the switch from DOS-based Windows to Windows NT. At the time, lots of programs were broken by the move because they assumed you could meddle in other application's internal memory. But breaking those applications was not malicious intent on the side of the Windows devs. It was a legimitate move towards a more sustainable architecture. I'm not intimately familiar with X11 internals, but my read of the situation is that X11 just exposes too many internals in its interfaces, and therefore it becomes increasingly impossible to adapt it to contemporary needs (in this case, application sandboxing).

> when in fact the lead developers on Wayland are the actual Xorg maintainers.

Daniel Stone[0] incredibly funny and educational talk[1] from 2013 could help many critics who missed the 3 decade long ongoing discussion of why X is bad. We've talked about this already in the mid 90ies (when there were no alternatives) and now 8 years after this popular linux.conf.au talk, people still having troubles understanding why X has always been a terrible tradeoff for both security and performance (well for literally everything).

[0] https://www.fooishbar.org/about/

[1] The Real Story Behind Wayland and X https://www.youtube.com/watch?v=RIctzAQOe44

edit: BUT X IS THE UNIX WAY! << so what is the 1 thing X does, and what is it doing well?

The talk was interesting but it also shows how the speaker is a little deluded that everything turns around technical quality of code and developers. X is used and preferred by many users and application authors because it is the standard and it works.

> so what is the 1 thing X does, and what is it doing well?

It provides functional and well-established(accepted standard) graphics server technology/libraries for any application that uses graphics.

I don't care if it takes 10 liters of blood to compile one version of X or whatever. I'm a user. All I care is that my desktop works as robustly as did it yesterday or 5 years ago. If that's not happening, I'm not interested in macros, classes, variables, declarations, structs, or any other geeky CS technobabble. That's irrelevant. And a product that advertises itself as being created to be convenient for the people developing it is a paradox. Don't develop it, then. Makes things even easier.

-- from https://www.dedoimedo.com/computers/fedora-25-wayland-vs-xor...

Yes and no.

Application sandboxing is not a need I have or one that I want my display manager to solve.

But incompatibility vs the architecture disallows features like screen sharing, automation or accessibility means I will not ever have it installed anywhere.

I can't tell people want to do, but those features seem very "contemporary" to me.

Perhaps it's because I use linux as my work desktop. I suppose if this is the feature I should just migrate to OSX.

Because I need video conferencing. I don't need a sandbox. I already am selective about what I install and most of those things aren't even desktop applications.

> I already am selective about what I install and most of those things aren't even desktop applications.

in X any application can turn into a keylogger, and you don't need a GUI for it, as long as the program can "speak X" it will be able to read the shared memory of all other programs the X server manages.

Yeah. But that's a company risk.

Not being able to participate in video conferencing and desktop sharing is an employment risk.

Im a picky person though. I already use i3, but also have NVidia gpu's. I don't mind spending some money on another real GPU (will it work with AMD).

I understand the frustration of the developers too. But screen sharing or my custom screenshot tool.. are things that will make me recompile if needed for days before I give up and abandon linux for the desktop.

I realize I'm spoiled, but I do wonder if this is the direction things are going who is going to be that hypothetical user that cares?

I run 4 applications: Chromium, Xterm, Gimp and IntelliJ. Am I unsecure?

You don't need application sandboxing from your display manager. I at least, do want my display manager not to break all other application sandboxing.

Exactly! I laugh at this petition because nobody is maintaining X.org server right now or even wants to touch it. Last major release was iin 2018. It is a nightmarish cludge. Nobody is forcing anyone to upgrade their software but the rest of us want a future without X11.

It's not just the internals of X11 that are the problem, it's that the entire concept was baked in a different time. The major innovation of the X11 server was that, well, it's a server. It can be completely decoupled from the client, even physically if you use it over tcp. That was kinda nice, but then came GPUs, and direct rendering. The name of the game is now violating every single abstraction that stands between a userland app drawing directly to the display with only a GPU and a single graphics api in between. It was definitely not junior developers who rewrote the entire kernel graphics stack to move from frame buffers to direct rendering.

Computer graphics are pretty tough if you think about how many bits/s are being transferred to your 4k display at 60hz and how little latency we will tolerate. It's not like network data where we can afford to encapsulate it five times over and abstract it and forget about the contents.

Any criticism of Wayland should be with regard to what is next. Hell I'm not even convinced we need a display server, let's talk about that. I'm not going to take anyone seriously who says X11 is better, and because of a few insignificant issues like taking a damn screen cap (really? and that works fine now btw).

And by the way, that whole "never rewerite software from scratch"... that might be what they teach you in your startup accelerator or your CS classes, but in the real world, hardware changes fast, your clients change fast, and you will do well to start from scratch sometimes when fundamental assumptions change and you no longer want to support legacy hardware or legacy features that slow you down and everything can be drastically simplified with your new assumptions in hand.

Take for example, Mac OS 9 -> 10. Complete rewrite. One of the best decisions in computing history. And especially relevant here because the display server was one of the babies they threw out the window or whatever, and the quartz compositor was born.

I agree that rewriting things needlessly is bad. However I actually think the folks working on this knew exactly what they were doing and actually made a reasonable choice; they’re not rewriting Xorg, they’re making something completely and utterly foreign to what Xorg is. And I think it is justified regardless of whether Wayland ultimately succeeds or fails.

Here is a reasonable overview from 2012: https://lwn.net/Articles/491509/

It feels important to note that in the case of Wayland, it is indeed designed by the very seasoned folks that have worked with Xorg. If someone like keithp who have worked with X since before X11 has opinions on the design, it is likely well informed.

I think very few people involved underestimate the work involved to replace X, and that it is going to be many years before it is somehow done.

> It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues.

Many seasoned Xorg developers work on Wayland. All the decades of lessons form Xorg _were_ carried over to Wayland.

Many of the lessons are being carried over no doubt. Others are embodied in the heads of people not working on Wayland or in the codebase itself as emergent properties from interactions never consciously designed.

There will be some loss of lessons with almost no doubt.

>> Many seasoned Xorg developers work on Wayland.

All of them are either working on Wayland or moved on. There are no Xorg developers.

Who is downvoting this and why? Face it, Xorg is on life support. The only patches it still receives are Xwayland-related fixes.

I don't expect a stable application to receive many patches. How many patches do you think Xorg should receive nowadays and in what areas?

There are a ton of long standing security issues. Which is why they are working on wayland: X11 isn't possible to secure as bad security is part of how it works.

My desktop is not a smartphone where I download random shady apps from an app store. I don't need any of the "security" wayland offers. If I don't trust some software I simply won't apt install it.

You are not the only user.

I am the other user like him. Where are your imaginary users?

X isn't stable

No. That's exactly what I'm saying doesn't happen with a rewrite, ever. You think you take all of the knowledge with you, because after all, you wrote the thing so you know how it works. But it just doesn't work that way. There is too much complexity to carry around in your head.

I'm sure many lessons from Xorg were carried over to Wayland, but "all" is an insane exaggeration. And I'm sure many new mistakes were introduced. It's inevitable with any new system.

too hardline. progress is possible in this world. software is complex but starting with better sorted fundamentals counts for a bulk ass ton of the marbles.

This is not a rewrite. It's a new product that has overlapping functionality with X11.

Rewrites don't work when the following two statements are both true:

- the rewrite should achieve feature parity at a higher generaly quality

- the people who're doing the rewrite don't have most or all the knowledge that people working on the legacy system had (this could be written docs, automated tests, or just actual people hanging around)

There are also cases where the embedded knowledge and workarounds collected over years are a problem and can be solved in better ways by nuking the whole section. It's a 36 years old project after all. But it takes deep understanding of the Xorg system to tell which case it is... which I expect most people on HN do not have.

>It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues.

The Wayland developers are the same people who have been building all these workarounds for xorg for decades. This isn't a new thing coming out of nowhere. When the old guard is telling you that their own work needs to be replaced, I'd listen.

>Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does.

Good thing nobody forces you to use it and it's just one click away on the login screen from switching to xorg. In the meantime people who want to hack on the project can do so and ship their changes to users to test them plus those who do suffer from xorg issues( because yes, xorg doesn't do well on a lot of things that aren't even that modern despite all the workarounds for the "real world" issues it has) can use wayland right away.

Everything you said is correct. But, there are surely cases where even though rewriting everything from scratch almost never works, the status quo doesn't work either.

I'm afraid that's where the Linux desktop is today. Linux as a platform actually sucks for running "untrusted" software. And "untrusted" doesn't just mean proprietary games. It also just means "can I please run the alpha version of this cool open source project without being afraid that it'll fuck up my whole home directory?"

Android does a hacky thing where each program is its own "user". But then, having multiple (human) users is confusing (I don't know how Android does this, actually).

I don't know if Wayland will succeed. In fact, I'm rather pessimistic about it. But I'm also pessimistic about desktop Linux "succeeding" (staying reasonably useful even to nerds) if Wayland doesn't.

Android does a thing where it has its own display stack and very strict sandboxing so X11 isn't really a thing :)

And for the sandboxing it uses SELinux which is available on desktop as well. It's just a royal PITA to configure. This is why many people don't bother with it. But in a highly restricted environment like a smartphone it's a lot easier.

I wouldn't call that a 'hacky' thing, it's just using it in a very different environment.

The Android thing was a bit of a tangent. I did know that it doesn't use X.

The point was intended to be that Linux is becoming more and more lacking as a personal computing platform, relative to the gains elsewhere in the space. The display server is one part of that- the entire user/group permission model is another part.

The part of Android that is "hacky" is using a separate user for every app. That's obviously not what the Linux/Unix guys had in mind for the user mechanism, thus it's a "hack". It's not necessarily a bad thing and it's totally hidden from the UX of your typical Android user, so it's fine. Just interesting.

> And for the sandboxing it uses SELinux which is available on desktop as well

I used to think that, and then I started using Fedora. They have put a lot of effort into their SeLinux policies. I do all sorts of things with my workstation and very rarely have to care. In the last 6 months the only thing I have had to do is add ":z" to a bind mount running a podman container so that labelling happened correctly.

Well, if the protocol is fundamentally broken there's no way to go but an incompatible rewrite.

The protocol is extensible. Nothing would have stopped an approach of having new clients signal they support new behaviour and aggressively deprecate old functionality to iterate the protocol.

The problem was not the protocol, but that the Wayland devs explicitly wanted to throw out a whole lot of functionality people actually depend on without having put thought into replacements.

> Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does.

I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons.

Edit: Thanks for pointing out that Xorg is actually much older. My argument indeed doesn't hold.

Xorg was kind-of forked from a working solution (XFree) whereas wayland started from scratch.

Ahh, 2004 is when Xorg forked off of XFree86, but XFree86 had already been going on since 1991, and already that was an implementation of X which already existed since 1984, and you can trace back even further than that if you want. It’s been a long ride home.

Xorg development largely halted circa 2008 though, since then most of the "Xorg" features have been deleting code in X and moving it to Mesa/libdrm/the kernel.

Wayland's latest release "mostly contains bug fixes and minor protocol updates." [0]. That is ~12 months worth of changes. Pretty similar to 1.18. It isn't like the protocol is a hotbed of massive changes, it looks largely complete according to the release notes.

[0] https://lists.freedesktop.org/archives/wayland-devel/2021-Ja...

> I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons.

Xorg is a fork of Xfree86, which was released in 1991 (https://en.wikipedia.org/wiki/XFree86).

> Xorg is 16 years old

Xorg is a fork of Xfree86 which started in 1991 but feature parity minus the cruft has been reached years ago. Some people disagree about what is cruft however.

For example, I view 90% of the list in the posted articles to be minor softwares relying on undesirable behaviour and unwilling to adapt (the remaining 10% being misrepresentation of actually solved issues listed in bad faith). The original author would obviously disagree.

Then again, I consider X.org to be and always have been completely unusable. How can people accept to use a display manager with constant screen tearing is beside me.

> How can people accept to use a display manager with constant screen tearing is beside me

That's solved by using a compositing window manager such as Compton.

That's also solved by using Wayland with Gnome 3 which comes by default and doesn't require me to tweak anything.

You don't even need a compositor. Just activate Force Full Composition Pipeline if you have an nvidia card.

see: https://imgur.com/a/lDfObqw

It's in the advanced settings.

Silky smooth Xorg web browsing and video at no extra compositor cost.

Or `Option "TearFree" "on"` in your xorg.conf for AMD users. Might even be the default by now.

Thank you so much for this! I've been looking for a solution in picom config for way, way too long.

You're welcome.

Frankly I have no idea why it's not enabled by default.

No, that does not solve the issue. Not using a compositor definitely makes tearing worse, on top of a number of other things, but even with a compositor, you'll get tearing on X, at least using intel graphics.

My notebook is using Intel graphics and there's definitely no screen tearing nowadays on X.org with compositing enabled.

> How can people accept to use a display manager with constant screen tearing is beside me.

I never even noticed it, and I never quite understood what people are on about with this. I had to go to YouTube to watch a demo video to see what it really looks like. And if I try it, then yes, I suppose I see some tearing if I move a window and pay close attention to it. But I never even noticed on my own, and am not bothered by it in the slightest.

Would it be better without it? Sure. But the amount of effort to eliminate what I experience as basically a non-issue is a poor trade-off IMO.

Same here on all points, including Youtube. I always envisioned some kind of critical graphics failure from the way people talk about it. Maybe my standards are too low.

Maybe it's overhyped, but tearing does make me feel like I'm using an amateur OS, especially when I'm using X for some weeks and then hop over to wayland or Mac or Windows, and the picture looks stable.

Tearing can be a gigantic issue when you have full screen videos or render games. However as far as I could find these are caused by the KDE/Gnome/etc. compositor not running in sync with the screen refresh rate. Find a way to bypass the compositor(KDE options to disable it, GNOME with full screen windows, or tell the X window to override the compositor) and it goes away. Its easy to lob all problems on X but whats next? Did it also cause 9/11?

I'm not familiar with the internals enough to know if you're right about tearing being the compositor's fault, though I'd be quite surprised if disabling the compositor resulted in less tearing. But I do know that I experienced tearing using compton with i3 (though much less than i3 with no compositor).

If Gnome, KDE and compton all manage to get compositing wrong on X, and yet every wayland compositor manages to get it right, then maybe it is a problem with X, even if there is some arcane way to fix the issue.

> though I'd be quite surprised if disabling the compositor resulted in less tearing

I have been running OpenGL based applications on KDE for some time and the answer to tearing was always disabling the compositor in the desktop settings or just overriding the compositor related redirect state of the window. OpenGL tends to sync against vblank anyway so I don't find it surprising that rendering directly to the final output buffer instead of an off screen buffer that is at the mercy of the compositor helps.

It's especially noticeable with fast-moving videos or games.

I never noticed it in that either. I played plenty of action games on my Linux machine twenty years, going back all the way to many hours I spent with the Loki port of Unreal Tournament, and I've watched plenty of videos in the last twenty years as well: but it just never registered for me. Maybe I'll notice it now that I know what to look for, but I'm the kind of person who is pretty happy with 1080p (or even 720p) too, so I suppose my brain is just not very good at picking up these kind of visual things, or something.

Yes, tearing is visible in fast-moving videos or games (when run in windowed mode), but video players or games usually run full-screen, where tearing does not happen (as applications can do vsynced doublebuffering), so usually it is not an issue.

> How can people accept to use a display manager with constant screen tearing is beside me.

I'm not saying this isn't true, but I always find it interesting when I see people complain about screen tearing.

How often does it actually happen to you, and under which circumstances?

Personally I've been using computers since 1995, (Windows and Linux) and I practically never encounter screen tearing. And this is on a wide variety of hardware. Spread evenly over intel, nvidia, and AMD.

The rare times I did encounter it, was only on one PC with an ancient nvidia graphics card (running Arch linux, bspwm). And it would only ever (if even) happen right before a restart after updating the nvidia drivers.

>> I'm not saying this isn't true, but I always find it interesting when I see people complain about screen tearing.

>> How often does it actually happen to you, and under which circumstances?

It happens every time I Watch a video on X, which is really annoying. It also happens when dragging large windows on my 4k. Wayland is fantastic in this regard.

> How often does it actually happen to you, and under which circumstances?

Scrolling, moving windows, resizing windows, approximately constantly.

Then you haven't switched on Full Composition Pipeline on your nvidia card (if you're using nvidia that is).

I've been enjoying silky smooth Xorg and video for a very, very long time.

Screen sharing seems a critical feature to enough people to rise well above mere “minor feature” if the goal is to replace Xorg. That was probably true 12 months ago, but definitely true 8 months ago.

If the goal is to have a different display protocol that a significant fraction of the possible user base can’t reasonably use as their root display because it provides yet another way in which “Linux isn’t ready for this to be the year of the desktop”, that’s a perfectly reasonable aim, but it seems like Wayland is targeting replacing X, in which case supporting screen sharing (and recording) is probably not an ignorable “minor feature degradation”.

Wayland definitely supports screen capture.

Can you write an application which uses screen capture on Wayland, and which works equally well regardless of compositor?

If not, then Wayland doesn't support screen capture. Individual compositors might, but asking developers to implement five separate screen capture protocols is a big ask.

You can. The standard is new, but it exists, and now works on Gnome, KDE and wlroots based compositors https://wiki.archlinux.org/index.php/PipeWire (read past the intro paragraph to where it talks about the various backends available)

Screen tearing for me has been a thing of the past for years now - just switch on "Force Full Composition Pipeline" on your Nvidia card and hey presto, no more tearing.

In 25 years I've never noticed any screen tearing on a variety of hardware. I'm sure it's happened but it's never been a noticeable issue

Is this really a problem for people?

Yes. Even scrolling a long page (like this HN thread) in a web browser can easily produce it. Can be very, very annoying.

I consider tearing a feature. Tearing is an unavoidable consequence of writing to the front buffer, and writing to the front buffer is required if you want the minimum latency possible. If you write to the front buffer, then so long as the part of the screen you're updating is below the raster position you'll see it that same display frame. If you write to the back buffer and then wait for vsync to swap buffers and avoid tearing then you won't see it until the next display frame. I value latency more than aesthetics, so I never use a compositor or Xorg's TearFree feature.

The goal of Wayland is "every frame perfect", which means I consider it broken by design. I want imperfect frames right away, not perfect frames later.

In an ideal world that would be an option to toggle. I can see use-cases/occasions for both "fast&nearly_perfect" and "slow&pixel-perfect"

X as a display server is over three decades old, the popular incarnation in the nineties being XFree86, which devs abandoned in 2004 to form Xorg due to licensing issues. Most of the code itself really is ancient.

> Reducing technical debt by rewriting everything from scratch (almost?) never works.

Agreed but this is one of those "almost" exceptions.

1. Is the code built to 40+ year old requirements, many of which are irrelevant today?

2. Is it hard to make it work for today's requirements without widespread fundamental changes?

3. Is the code a huge pile of foo that's damn near impossible to navigate, much less understand, much less debug?

4. Would a hypothetical program that solved today's requirements (with an architecture where the most important backwards compatibility features could be hooked in as needed) be smaller, simpler, and easier to maintain by newcomers?

If the answer to all of the above is "hell yes!", you might want to think about a wholesale rewrite.

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

This is why you need tests, otherwise you're stuck with your legacy forever.

It doesn't ?

Maybe we've worked on different projects, but sometimes the legacy is just that things were done differently in the past, and the APIs available were crap.

Let me tell you an anecdote: A big, old, crusty online retail company wants to bring their web shop up to par with the competitors. Due to many years of technical debt, feature development speed has come to a halt and it is determined that the web shop must be rewritten to remain competitive. Over a period of months, requirements are gathered from user focus groups and all the employees who maintained the content on the previous shop. Development starts and 3 years later, still not having implemented all the requested features, the project is deemed a failure. Another 3 years later, a new consulting company is brought in to attempt to realize the project. But this time, instead of trying to replace the old shop, a completely new shop is built iteratively and independently. Domain experts in the company complain: there is no way to review products, they can no longer put text wherever they want in the shop and users have to call the hotline to unsubscribe from newsletter because there is no settings menu. Still, the new shop is already outperforming the old one, despite having far less features. It turns out that many of those features weren't needed after all.

The lesson here is that writing a direct replacement system rarely succeeds because the new system will hardly be able to implement all the features in a reasonable time. Instead you should take a step back and develop a completely new system that follows actual user requirements and gets rid of all the dead code.

Reply on news.ycombinator.com