Hacker News Reader

I'm sure you know all this, but just to give other people some context on how Arch expects one as a user to handle this:

> - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too.

You accidentally put a plural in "vendors", but in practice this is just Nvidia.

If you are forced by circumstance to deal with an Nvidia card, use the LTS kernel and most of your problems go away, and you can also pin your X.org version and manually update it. The real solution is to use a GPU vendor that has mainline drivers, the only valid reason nowadays not to do that being CUDA or being unable to acquire hardware with mainline support.

> - I've had gedit start crashing in new minor versions of gnome due to a setting being incompatible, needing to track that down and unset

That's a gedit bug. Complain to gedit devs or stop using it.

> - If you have python virtualenvs for development and system python is upgraded to a new major version, all your virtualenvs break

You should not be using system python for non-system tasks. Use asdf, Nix, Docker, pyenv or a similar tool for projects requiring their own non-system environment.

> - If arch upgrades the major version of[...]

Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own.

> - If you do a full system upgrade and have AUR stuff installed

AUR is not Arch, it's up to each AUR maintainer to keep their scripts up to date and you as a user to keep up with those external dependencies. A common approach is to use an AUR helper to handle both system and AUR upgrades.

> - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date.

And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality.

Yes, all of this is definitely true and I'm not actually complaining about it, I have used arch on all my machines for 7 or 8 years and don't plan to change that anytime soon, but these problems are actually unique to using arch and other rolling distros. Ubuntu, Fedora, etc have coordinated releases with some level of testing that things are compatible with each other. On arch, these are problems one must deal with though. I don't mind dealing with it, I live for Linux stuff, but the point is that it does happen to real users and it's not for everyone.

> Partial upgrades are explicitly unsupported by the distro. Pacman allows you to do this, but you're on your own.

Definitely correct, the issue is when you're trying to install a package to get something done, and suddenly your faced with the proposition of either upgrading your whole system while trying to get work done or attempting to upgrade just the required libraries. The latter often carries less risk, but occasionally is very problematic.

> And this is why as an Arch user you will be nudged by experience to use software that gives a crap about quality.

The issue is that this is true of so many very major pieces of linux software. Upstream vendors don't necessarily test their stuff in a ton of contexts, plenty of bugs occur in the kernel/xorg/wayland/gnome/glibc/python/etc. All of these are very mainstream projects that are difficult to avoid.

The problem with rolling release distros like Arch is that I don't actually care about running the latest version of all my software. For most of software on my system, I only care that it works. The few things I do want on the bleeding edge, I'll just build myself.

Arch isn't bleeding edge, its latest stable. If the software you get isn't stable that's on the software authors you've chosen to rely on, not arch, not rolling releases.

I found more problems using e.g. debian stable. Upgrades were totally black box and anything could break spectacularly. This was most commonly caused by mixing up to date software you build yourself and the stable repos. Arch on the other hand updates happen daily, and for me, unattended. If I ever do find issues (rare to never), backups fix it.

> Arch isn't bleeding edge, its latest stable.

For any definition of stable.

That said, the ABS tree and the AUR in arch, combined with the general avoidance of patches, do make arch one of the easiest places to play with bleeding edge software. With the toolchain in place, so many packages are just a one line version number change to get any particular version building. Whereas building packages from source on ubuntu/fedora/etc can be an uphill battle depending how entrenched the distro's patches and inter version dependencies are. (Though most of the time, building things via make and sticking them in some directory is acceptable, arch really does make building packages easier than other distros)

>Definitely correct, the issue is when you're trying to install a package to get something done, and suddenly your faced with the proposition of either upgrading your whole system while trying to get work done or attempting to upgrade just the required libraries. The latter often carries less risk, but occasionally is very problematic.

I may be wrong, as my default is to -Syu any package I install, but doesn't -S install the version of the package compatible with the last time you updated your repos? It's -Sy that can cause problems.

Package versions don't remain in the repo for long so if you don't update the package index you'll find that many packages are not installable. There isn't really a snapshot of compatible versions at a given point in time as far as I know, though this might work for a couple of weeks.

> of your problems go away, and you can also pin your X.org version and manually update

> Partial upgrades are explicitly unsupported by the distro.

The problem is obvious. Sometimes you might have to pin a package version because the newer version does not work with your hardware driver, as you said, but then again Arch calls this unsupported.

I had the same problem in the past, not with Nvidia but with DisplayLink. For that reason I always leave a USB stick with a bootable Arch ISO in my Thinkpad, so that I can arch-chroot and repair the system. It happens rarely, but it happens to me and others as well.

Thanks.

> - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too.

I imagine this is mainly a problem if you have an Nvidia card (which would explain why I haven't had the problem).

> - In general (not specific to arch linux), new versions of software break stuff all the time. You tend to hit way more of this on arch if you keep your system up to date.

More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well.

As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year.

> More often, sure. But for software delivery from the production side, the common viewpoint seems to be that deploying more often leads to less pain in total, because you're making smaller increments so the individual failures are smaller as well. > > As such, aside from integration issues like Xorg and the drivers, I would expect Arch to have fewer major breakages than (e.g.) using LTS releases of some distro and upgrading every second year.

This is mostly true for production infra (at least for your own software in production infra) but I'm not sure this logic flies for personal machines and software you don't directly interact with. I have used arch for at least 7 years and I have had upgrades render my system unbootable (or nearly unbootable, like Xorg/GDM/Gnome failing to start). These issues were mostly not due to my configuration needing to change, they were due to breakages between different packages that would end up being fixed in some later minor version.

As an end-user upgrading distros every 2 years, I really doubt I'd hit many major breakages each time as all of those pieces of software will have been in the wild for a bit and major bugs will have likely been fixed. I think the system-level issues that are resolved by frequent deploys are stuff like "systemd deprecated setting X in service unit files, so I need to update my config" or "library xyz changed some API so I need to update my app", etc. With linux distros that have coordinated releases like fedora/ubuntu/debian/etc my software's interaction with the distro may break, but for the most part the major inter-package relationships within the distro get some amount of coordinated testing which doesn't happen with rolling release distros.

Put another way: deploying more often is great for quickly discovering problems introduced in software that you write. Deploying every single dependency in a linux distro more often is going to cause you to hit every bug destined to be fixed in some minor version of the software in pieces of code that you are very far away from and lack the context to quickly debug. So you will hit a much higher sum total of bugs that would have otherwise ended up getting fixed whether you personally hit them or not. However, if you are developing a linux distribution itself, then yes your CI infra should be constantly upgrading dependencies.

I'm not convinced that an arch-paced rolling release at a distro level will ever reach a point where inter-package dependencies do not cause totally unexpected bugs given just how many inter-library dependencies there are. The entire OSS ecosystem would need to write a ton more tests for this to be a reality, and distros would need to run that full suite of tests any time and dependency is upgraded. And even then, there's still so much possibility for breakages given that shared libraries don't do a fantastic job of versioning and the upstream vendors can't possibly ensure their software works against versions of a library they never ran it against. There's a linus torvalds post about this: https://lwn.net/ml/linux-kernel/CAHk-=whs8QZf3YnifdLv57+FhBi...

Despite all of this, Arch is actually amazingly stable, and the entire community is probably better off due to the existence of arch and arch users. We might be the best integration test there is for OSS software.

> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once, every app that didn't get upgraded will fail to start due to soname mismatches... and that can mean that pacman, sudo, etc are all busted (this has actually happened to me)

Okay I just wanna point something out here, partial upgrades in Arch are broken by design because they simply can't work. Don't do it.

That being said...

> - Arch will release a new version of Xorg before the graphics card vendors release new drivers that are compatible with it. Same for kernel versions too.

... kernel ABI is stable and even ancillary interfaces are usually stable, so usually it's quite A-OK to not immediately upgrade the kernel.

Can you elaborate why has Arch chosen to not support partial upgrades by design? They do work in Debian and it's saved my day a number of times. They let me roll back a broken upgrade without holding back upgrades of the rest of the system, and my computer then continues to be usable until the package in question is fixed.

They choose not to officially support it precisely because of the types of issues that upgrading e.g. glibc or openssl can cause.

That said, the vast majority of the time only upgrading a particular application/package (and its dependencies) will work just fine. It's just that there are no (official) guarantees.

The types of issues upgrading e.g. glibc or openssl can cause are largely predictable, aren't they? That's why there's libtool and sonames and that kind of stuff. The only missing piece is support for this in the package manager…

Pacman can roll back packages, and does let you stop certain packages from being upgraded. As you say, you could probably get away with doing it, if the package in question has "boring" (i.e. ABI stable) dependencies.

Because Debian sort of maintains their own "pinned" versions between upstream and you, you basically get Debian's version possibly with their own patches and fixes; whereas Arch just directly gives you upstream, compiled for your platform.

Debian's way of working is labour intensive, requiring packagers to fork, follow and maintain fixes and security issues in the version of the software they're packaging. They generally do a great job, but this is not a sustainable approach for smaller distributions. Arch Linux on the other hand follows upstream in real time, so you get the latest fixes directly from upstream, but there is no real Arch Linux "version buffer" that allows to freeze the versions of (parts of) your system. You move with the stream, that's sort of the philosophy of rolling distributions like Arch Linux.

I don't think that's it, really.

The real trick is that Debian packages have sonames in their versions, so ABI compatibility is encoded in package dependencies. So when I try to downgrade a library (to a known older version from snapshot.debian.org), it knows precisely which packages depend on the new version and forces them to be downgraded as well.

This is entirely orthogonal to following upstream vs. backporting patches. Sure, Debian (stable) does backport patches, which makes it more likely that single package downgrades don't downgrade half of the system, but it really is a different thing. Debian testing/unstable follow upstream to a larger extent than Debian stable, and upstream fixes are usually preferred to patch backports. Still, partial upgrades and downgrades almost always work without trouble.

Yeah, definitely usually fine to avoid kernel upgrades, so that is a saving grace. The issue is just that pacman -Syu is going to be full of surprises

> - If arch upgrades the major version of glibc during some random package install (rather than a system upgrade), and you don't upgrade the whole system at once

this was what convinced me to switch to Debian back in the 90ies. It was the only distribution that could do this without throwing a tantrum. I was coming from RedHat and SuSE and both had huge problems with this. Debian presented a higher learning curve (at least that was what people said). Debian really only lagged behind back then with their graphical installer and the overall lack of integrated DE back then (compared to others).

I really like the Arch wiki as a Debian user but never used Arch itself. Not going to change because old dog, new tricks ...