Yum is dead, long live DNF
dnf.baseurl.org
dnf.baseurl.org
I wish they'd keep the yum name, since dnf is still based on yum, even if a lot of the innards have been replaced. Also, "dnf" is not at all awesome to say out loud, while "yum" is among the most awesome commands to say out loud.
Indeed, to me, saying "dnf" out loud suggests "did not finish," which is the opposite of what you want from a package repository tool!
His belief in Ansible being a decent idea really kept me going with it after the first 2 months or so, otherwise I might not have continued it as a side project, and over the course of several years, he helped me a ton with various issues and was an amazing sounding board, including sharing a lot of advice when I as being plagued by nuisance CVE reports and wondering what to do about them. We had occasional arguments and he was usually right, and we always resolved things. (We had also previously worked on Func together with Adrian Likins, who wrote most of up2date, which was also a great experience - Seth knocked out most of the fun SSL parts). Back then used to meet around Duke to come up with crazy patent ideas to supplement our low Red Hat salaries (mine was anyway), and ... yeah, so much fun.
At his service, what was super amazing was pretty much everyone said the same thing about how he had helped them, dozens upon dozens of people, and it was really one of the most moving things I've ever seen. One of the few that made you come out saying "whoa" and wanting to be a better person and care about helping other people more than I did. And I think I did. Or I'm trying hard to.
Seth got a lot of hell for yum and shared many of my frustrations with the nature of the OSS community at times, but I still prefer it than a lot of idioms in apt. Things like yumdownloader, createrepo, --enablerepo, --disablerepo, and many things like it, just made it click for me, and it worked so well. And working with people like him at Red Hat made it really feel special to be there and to be able to learn from them.
I think about him every time I run yum, and every time something in apt doesn't work the way I want. Folks should just enjoy the metadata refresh time as a reason to get coffee, and blame PackageKit for your silly yum locks. (ALWAYS. REMOVE. PACKAGEKIT.)
And I agree on keeping the name, and it doesn't clash with "disjunctive normal form" that way too :) .. not that I remember what that is so much.
From the article:
"undocumented API, broken dependency solving algorithm and inability to refactor internal functions. The last mentioned issue is connected with the lack of documentation. "
Well some of them I don't care about as a user of yum some of them I can verify not true (documentation) and some of them I care about and it works for my use-cases (dependency solving algorithm).
There are always things that won't be fun, and those are the things that get kicked to the curb.
If you're going to swap out: why not switch to apt? What does apt lack that DNF is going to provide? This seems like one of those low hanging fruit where standardization across distributions would make sense.
It's a shame, though - apt-rpm worked very well.
Even if dnf and zypper are 1000x faster than apt there, that alone wouldn't convince me to use them.
I would also say that dnf and zypper are better package managers than apt, in as much as they are faster and easier to automate. Compare the code here:
https://github.com/libguestfs/libguestfs/blob/1.29.43/custom...
Here is a relative simple package, done for both RPM and .deb. The RPM spec is 141 lines (excluding the changelog):
http://pkgs.fedoraproject.org/cgit/virt-top.git/tree/virt-to...
The .deb is actually shorter in this case, but split over several files, and uses cdbs which I find infuriating with its lack of documentation and multiple hidden implicit rules. If you have a Debian machine around, try reading the /usr/share/cdbs/1/ files some time. Remember also that for most Debian packages, the files come in a tarball or even a patch, which makes them hard to manipulate without obscure deb-* commands.
https://anonscm.debian.org/cgit/pkg-ocaml-maint/packages/vir...
Packages have sticky-vendors, so you only get updates from the same vendor unless you choose to switch. Even if another the update is in a different package repository. This means you don't get flip-flopping package vendors when the same package is available from two vendors and they release updates one after each other.
Another example is the tooling around package/repository signing - Letting the user decide whom to trust without making them jump through hoops to manually import keys.
There's also a lot of things in the packaging itself designed to help support third party vendors. Dependencies tend to be configured with capabilities rather than package names. e.g. The kernel provides information about the binary api version which third party drivers can depend on. Third party kernel modules can also supplement/enhance (soft dependencies - inverse recommends/suggests) specific PCI ids so the solver will suggest installing them if available and compatible with your hardware.
In my view debian package management "just works" because the set of packages is carefully curated and tested, there's a high standard of package quality, and everything is available in one place. PPAs can present problems.
The rpm distro ecosystem philosophy is somewhat different. It's impractical to expect a single organisation to package all the software you will ever need, so let's provide tools to help software from multiple vendors to get along.
Try upgrading from major versions and seeing what happens.
It's absolutely relevant
Given, I understand some people want "servers like pets" and Linux desktop to be a thing, in industrial applications it's usually not an issue.
In most cases, no-longer-specified items would be removed. BTW ansible sucks for this aspect (I use it extensively at a new place) ;)
My point is that, the modern attitude of "you can't trust what's on a server unless you build it from scratch or apply an image" is not the only valid way to do things, and is somewhat defeatist, like we're no better than windows where you can't guarantee that anything can be cleanly uninstalled / replaced. Is your package manager that bad?
Yum being dog-slow was very annoying. Updating all server lists for every yum command was very annoying. I customized various modules and scripts to use "makecache" and "-C" where appropriate, but that was an annoying task. And yum was still slower than I'm accustomed to a linux package manager being.
Finally having a couple of years experience with yum, I can confidently say that pacman (archlinux) and apt (debian) are worlds better. Maybe zypper and dnf make rpm not suck; I probably won't find out for a few years.
Today's "continuous deployment" world, that just "tear down and put the latest version" once a week is crazy.
A stable foundation is important, and updating it without breaking everything is needed sometimes.
It all depends on what kind of apps you are deploying and supporting. While I wasn't even discussing immutable systems in this capacity, some workflows work better for .com style applications. In a typical bank environment where you have thousands of legacy applications floating around, you are more apt to not be able to control the architecture and need to push out security updates.
In place updates here are fine, however, I still wouldn't want to do an in-place dist-upgrade across all of those systems, and then find out which ones of those thousand applications had problems. In this case, it's better to redeploy those applications if they need a new OS and the OS is no longer recieving security updates - and try to shift some of that burden onto those who maintain the application.
If you are just deploying a .com app though, you need a good backup/DR strategy, and it helps to be able to redeploy everything and take steps to not get attached to state on that machine.
This isn't the case. It updates information when the cache expires, so this will only happen about every 30 minutes or slow. It should also only take (usually) way less than a minute.
> BTW ansible sucks for this aspect
Yep, it's not meant to understand how to remove resources it doesn't know about. That being said, it feels like I've created PHP at times, and I don't mind people not liking it. Many of the ways we have to automate Linux systems (due to lack of structure and API) are kludgy at times, but removal of packages not present in a master manifest seems dangerous to me for various reasons (group installs, tools self-bootstrapping, work happening out of band). Fair enough.
> is not the only valid way to do things
This was not the argument for immutable systems (though I like them), but rather that you need a good disaster recovery strategy and this is likely the best way to handle a major distribution version upgrade.
Minor versions? Continue to do what you do. Major upgrades between EL versions are full of all sorts of fun.
I've typically seem them done with upgrade kickstarts and the like, and you don't get good error reporting there at all when things go wrong. There were some advances in pre-downloading and then doing upgrades but... yeah... not a fan of doing them in automated context.
> " I can confidently say "
Trying to drive it programatically, yum seems much better engineered to me than apt. One example recently (and maybe there's a way in apt to do this) was to be able to select just one repo to use during an update to grab packages from, and I was missing --disablerepo=* --enablerepo=X. But there are a lot of things like that. Config files also seem a lot more capable. My opinion there too of course.
It doesn't do full downloads of files, but (as of RHEL5) it still makes HEAD requests or something, re-processes package lists (I guess apt does this too), I dunno. "-C" makes a significant difference. I want it to take less than a second to make a decision or spit out information; other package managers can do that.
EDIT: also want to say, I appreciate ansible overall, and I appreciate your attitude towards it. I've been close to a project that was good for a couple of uses, got kinda popular, and then people wanted it to be good for all purposes...
When I do an apt-based dist-upgrade, the long part of the upgrade is not the single-or-low-double-digit seconds it takes to solve the deps, but the download and installation of the new packages.
Some of the benefit also derives from the packaging format itself. The ability to unpack DEBs using nothing but shell tools (busybox within a Debian system's pre-boot ramfs is sufficient and has been successfully used by me -- Red Hat loses in this instance by using a ramfs shell that's both 1) larger than Debian's dash and 2) doesn't offer interactive use -- it's a scripting-only shell, FML.
Joey Hess at one time had a detailed comparison of various packaging systems. He's pulled it apparently due to political / fanboi bickering, which is a shame. He's author of 'alien', a tool for converting between packaging formats.
But Policy (and fucking enforcing the fucking hell out of it) trumps.
Source: 18 years' use of both distros and many other Linuxen. 30 years' experience on further Unixen.
I feel ashamed that for the last ten years I could not find a kind word to the portage developers. For what is worth, they don't care.
Paludis does correct dependency resolution while Portage just pretends to.
All three of those things have gotten better over the years, and I have little doubt that given the attention and effort that RedHat and Debian package managers get, portage could be a clear winner. But the portage we have now has too many pitfalls to be the best all-around choice.
Portage is not used.
The broadness of the system isn't quite as vast as many distributions but running a desktop / dev workstation I have never encountered a package not available that I needed.
Exherbo may not be for you. It values users who are willing to be developers as well, and augment the system with the packages they need. You want others to do the work for you, that's not what Exherbo is about.
Besides, the original discussion concerned what package management system was best, not if it had tool X, Y or Z that you claim is very often needed.
If you can't be honest about its shortcomings, you won't be able to convince anyone to try out your pet project. It doesn't matter how reliable and trouble-free it is at managing the core of the system if it immediately degrades to "build it yourself" anytime you want to use something that's not popular enough to make the cut for a live CD.
I like to think my comments were honest: I admitted that the system while technically superior does not have the breadth that larger distributions do, but that for my purposes it was sufficient. You ignored that and found some packages not currently packaged in an attempt to disprove my experiences. Furthermore I admitted that the project may not be for you since you expect different things from a distribution than many of us do. What is dishonest about any of this? I have been incredibly frank.
Besides, one of the nice things about Exherbo is that it handles the nonexistence of a package rather seamlessly. You can compile it by hand, install to a tempdir, and then have the package manager merge it directly while giving you the ability to specify information about the package (metadata, dependencies, etc.). And then of course the package manager can uninstall it when you no longer want it. This makes the problem of "build it yourself" kinda moot.
I'm not going to bother responding to the "make the cut for a live CD" remark since obviously the there are far more packages than would fit on a liveCD or liveDVD.
It has many of the strengths of Portage, and even go beyond (it builds from source, and users may configure package dependencies, like USE flags on steroids -- its declarative language used to write packages is also used to configure them, so you can do more than just passing flags to a package). But it offers a substantial advantage, because builds are deterministic. The set of installed software (with all needed configuration) is determined from a config file (or many), and from this it's always possible build the same system.
This means that upgrading always end up in the same state as installing from scratch. This also mean that common packages can be cached as binaries, without risk of breakage - it only downloads a binary package when building from source would build exactly the same binary.
Nix also feature atomic upgrades and rollbacks: it only touches the running environment as the very last step of the upgrade (setting up a symlink), and stores the previous versions of packages until garbage collected, so an upgrade stopped in the middle can't break your system (the exception here being kernel upgrades). Indeed, if you interrupt an upgrade or install at any stage, just issue the command again. (also, this architecture makes it incredibly concurrent)
NixOS is a distro that uses Nix. It can provide a GRUB menu to boot previous versions of the system. When you upgrade, you can have it affect running system or only upgrade after a reboot. Either way, when you reboot GRUB will give the option to also boot the system you was using before the upgrade. On a technical level, Nix works a bit like a git repository: each package is addressed by the hash of its derivation (that says how to build it, and all its dependencies), and if more than one system version uses the same package it gets stored only once.
Coupling NixOS with Nixops, a deployment tool; and Disnix, that does service-oriented deployments (like Docker), they can help build more repeatable systems for production servers too.
Some links:
http://lethalman.blogspot.com/2014/07/nix-pill-1-why-you-sho...
It's a series that walks you through setting up and using nix. It found it really simple and satisfying - I now use nix on Ubuntu.
Naturally no one fully obeys it, because LSB is a mediocre standard in general.
It was not mentioned in the post that dnf is based on the openSUSE dependency SAT solver (libsolv) that was created by Michael Schroeder years ago and that powers libzypp and zypper since openSUSE 11.0.
The dnf developers built a thin layer on top of it, called hawkey, and then build dnf in python on top of hawkey.
One of the biggest innovations of libsolv is not only the SAT based solver but also the .solv format which allows to store package metadata for big amounts of packages in an efficient way and operate the solver directly on it.
Seems more like `dnf.baseurl.org` is dead (it's down right now).
Google cache: http://webcache.googleusercontent.com/search?q=cache:PiRy8Wd...
It wouldn't surprise me to see it show up in centos-extras for CentOS 7 if someone sees a benefit to it.
Yum/dnf haven't been merged into systemd yet, please merge
This of course isn't necessarily a flaw in the AUR and is really more of an issue with the packagers doing a bad job.
And yeah I know that isn't basically a longform description field on the AUR page, but comments can do that job too. I don't see it as some crippling issue, and honestly I would probably not even change it - the way the AUR is laid out now makes it easy to pull down pkgbuilds and display package information agnostic to the environment easily. Adding paragraphs of detail into the pkgbuild or on the website limits that.
And once you've built a deb, you can use alien to convert to RPM (the only gotcha is that RPM runs preinst/postinst scripts with an empty environment and dpkg doesn't).
(and I say this as a diehard Arch user who came from Gentoo)
Zypper is actually fairly good. Still way more verbose than it needs to be, but it seems like a fair compromise between "apt-get stupidly-oversized-function-name" and "why the fuck does -Sy mean grab updated package lists from the repo".
When I have to install a font on my machine that isn't in AUR I'll just roll up a PKGBUILD, install it and dust my hands off. Makes it incredibly easy to remove the packages later if I no longer need them.
The only real missing piece of pacman is that there is no distinction between a feature and security / bugfix update. But then you should be more stringent with upstream to beta test their feature releases better.
Really if you wanted to automate pacman nothing stops you from creating your own custom repo as a gateway to some workstation or server cluster you have and setting it up to use your custom repo exclusively for all packages. It has delta support, does checksums, uses the best compression format available, and is a lot easier to use for custom applications than the OBS or alternative packaging tools.
Would you take debian, with its three branches, and move it to pacman? No, because the lack of pinning and differentiation between security upgrates and normal upgrades would wreak havoc ("get a better upstream" is a nice suggestion, but an unpractical one). The same could be said for Fedora (actually, for Red Hat, but Fedora is Red Hat testbed after all).
I could set up my own repository, clearly, it may very well be a good solution, but to me that is not automation anymore. Similarly, there is nothing preventing one from setting up one's own repo of old packages, and reinstalling those. Still, I see the value in having the rollback features inside DNF (call it laziness, if you wish). I guess my comment came out as random pissing on Arch, when what I wanted to point out was that other distros simply have different needs.
And that kind of operation could be automated - its just a pacman -U on the old version and append into pacman.conf on the IgnorePkg line.
And nothing really stops you from having, with pacman, repos the way Ubuntu does - because really, its not that security and feature updates are hugely segregated - they usually are just a boolean in the package description. What happens is they have repositories of software they will not update with feature releases but instead only ship bugfix and security patches for, and they just call them jesse / wheezy / vivid / wiley etc. You could use pacman for the same end, making a repository of software you don't push feature changes to but just push bugfixes in, and again replace Archs repos.
The point I'm trying to make is there is a distinction between Archlinux the repository and Pacman the package manager. You can get around a lot of the unfavorable aspects of how Arch does packaging by doing it yourself. Of course it makes no sense to actually do that when Debian, CentOS, and Ubuntu LTS exist to do that exact same job without all the work, but it isn't because pacman is crippled in one aspect of package management to such a degree its unusable for that purpose.
As far as pinning goes, however, I disagree: if you mark a package as IgnorePkg it does not get updated. You could use either naming conventions or splitting the repos up to track different repos for different packages, but it starts looking like an antipattern to me (i.e. the way you would have firefox track jessie while you are on wheezy would be by setting up your own repo only for firefox, a bit of an hassle). It's fine if you are the packager, it's a bit cumbersome if you want a stable debian box with a fresher version of django and nginx.
After all the road map for pacman 5.0 proposes hooks and a better handling of optdepends. Both can be already automated via scripts, but both would be nice to have out of the box.