OpenBSD ports are more than just a Makefile
homing-on-code.blogspot.com
homing-on-code.blogspot.com
I also always found ports and OpenBSD to be explainable, non-surprising and simple as opposed to much of what I saw with rpm and apt.
However, all seem to be stuck in the past. I wonder why the ideas pioneered by http://www.gobolinux.org/ are not more widely implemented, leaving nix aside.
Lest anyone be misled into believing ports is comparable to yum or apt...it is not.
With ports it is not only possible, but likely, that you will end up with upgrades to some software that can break other pieces of your system, because the dependencies often do not account for things, like compile flags, which can be changed by the user at install time. This leads to a very fragile system a couple years down the road, after a bunch of upgrades and new software installation has occurred. This isn't theoretical; I've seen it happen more often than not in my very limited use of the ports system...it is avoidable, but it is not obvious or well-documented how to avoid it, and adds not insignificant costs to maintaining your systems.
With ports it is not possible to verify that the files installed by the port match what was originally installed. This can be useful for security purposes (i.e. has the ps command been tampered with?) or for troubleshooting purposes (i.e. did someone install a program from a source tarball over the original package, or did permissions/ownership change?). Edit: I was wrong on this one. pkg_check verifies package contents against checksums, which is comparable to rpm -V or debsums.
With ports, it is not possible to reliably roll-back to a prior version in-place (i.e. without uninstalling everything, and then reinstalling it).
With ports, it is likely you will, over time, end up with multiple versions of some things (libraries, languages, etc.) installed. This is common on Linux, as well, but the ability to determine what is built against which versions, and to be able to keep up with whether, say, an insecure version can be removed without breaking things on the system, is much weaker (non-existent, as far as I can tell).
OpenBSD has many fine qualities and very smart people working on it, and I don't think it's the dumbest idea ever to use it. But, ports is simply not a sufficient answer to the package management problem. RPM/yum and dpkg/apt are not perfect, but they are vastly superior to ports in every dimension. FreeBSD now has pkgng which is very much a move in the right direction, and it would be definite progress for OpenBSD to adopt it.
Note that providing binaries or not is not on my list of complaints about ports, above. Somehow, in every conversation about ports failings as a package manager, there will always be people talking about binaries, and how you can totally install binaries with ports and so there's no reason to not like ports. I find it annoyingly time-consuming to build everything from scratch, but it's not a deal-breaker. Providing binary packages is not why apt and yum are vastly superior to ports.
Users are only discouraged from building ports because binary packages have been provided for all available port flavours, there is no opportunity for users to customize ports or tweak compile flags without them venturing into the realm of already unsupported tasks.
The ownership and permissions of files are tracked, and the package tools will notice if files are missing or have been modified. Dependency tracking, resolution, individual file check-summing and package signing are all performed. And OpenBSD handles the versioning of shared libraries separately for the whole system, even ports.
FreeBSD's pkgng was undoubtedly a good move for FreeBSD, but OpenBSD already received it's very own next-generation package system a few years prior.
I greatly prefer rpm-based package management, buy I greatly prefer Debian as it allows me to avoid interacting with the egomaniacs that offer 'support' on freenode and the forums.
Can you link me to the documentation that covers this? I googled before making my comment, because I have limited (and mostly old) experience with OpenBSD ports, and wanted to be sure I wasn't making incorrect assertions. I couldn't find anything about verifying packages.
Anyway, if what you say is true, and OpenBSD now has effective package management, I'm going to spend some time with it, and possibly add support in our installer for it. I was under the (seemingly) mistaken impression that it would have the same faults as ports on FreeBSD.
You could also check the pkg_add(8), signify(1) and ports(7)/packages(7) man pages.
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/...
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/...
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man7/...
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man7/...
* packages vs. ports[0]
* porter's handbook[1]
Regarding rolling back/forward, the ports system allows you to create binary packages from your build. So, if you're in a production environment, the old version of the package still is there, ready to be reinstalled with all your compile flags intact.
Verification is also something that ports does. Because ports work with the pkg system, installing a port installs it in a package, with hashes and everything. Auditing's a no brainer there.
Furthermore, the ports system allows you to build everything within jails dedicated to port building, so every time you update, you get fresh packages with no legacy dependencies.
Oh, and one more thing. That's just the steps FreeBSD has made. OpenBSD does not have a rolling ports tree, the ports tree is tied to a specific version, which creates binary packages by default, without installing the port to your filesystem. So, again, you get everything apt and rpm can do, plus you get a level of flexibility that they can't match.
This doesn't sound equivalent to:
yum install httpd-<specific-older-version>
To me. What does that look like, exactly? What are the steps to roll back to a prior version?
"Verification is also something that ports does. Because ports work with the pkg system, installing a port installs it in a package, with hashes and everything. Auditing's a no brainer there."
I now see this is true; apologies for misleading statements above. The pkg_check command seems to do what is necessary here. That's great!
"Furthermore, the ports system allows you to build everything within jails dedicated to port building, so every time you update, you get fresh packages with no legacy dependencies."
Again, what does that look like? What are the commands? How long does it take?
"So, again, you get everything apt and rpm can do, plus you get a level of flexibility that they can't match."
I think we may both be speaking from positions of ignorance. I'm gonna go install an OpenBSD VM and fix my ignorance.
I am tracking the -stable branch in ports, which means I periodically sync my local ports tree with upstream and then build / upgrade any updated packages. Inside my /usr/ports/packages/amd64/all/ directory I have the following packages:
firefox-esr-31.4.0.tgz
firefox-esr-31.5.0.tgz
firefox-esr-31.5.3.tgz
firefox-esr-31.6.0.tgz
pkg_delete firefox-esr-31.6.0 && pkg_add firefox-esr-31.5.3
And that's it.
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/...
I agree that the package management could be better, but as far as I can tell it isn't as bad as you suggest as long as you stick to the official prebuilt packages.
or is it different when following -current ? I usually stick to releases and only patch major bugs.
Apt, for example, knows how to consume variant packages from alternate sources and then upgrade back into mainline—but also allows the user to specify (with a "pin") that they prefer to continue on the alternative even if it's older.
Apt also has an "install from source" system where the source package gets baked into a binary package within a chroot container on the installing system, and then that package gets installed. That package is then like any other package on the system, and can be cleanly removed, or upgraded from into a newer upstream release, etc.
These are important features. They're not important for j-random-user, but they're important to distribution maintainers, and to sysops who have to, for example, apply temporary hotfixes to specific packages on their production machines until the distribution ships those same fixes, at which point they want to revert to upstream.
The OpenBSD ports system makes binary packages (and then installs them). Those binary packages are managed by the pkg_* tools, which allow users to install, verify, upgrade, delete, etc., packages.
The binary packages are available on the mirrors if you don't want to compile them yourself. You just 'pkg_add <package>' and you're good to go. This is actually the recommended procedure, and users are discouraged from using ports unless they really need to.
> Apt knows how to consume variant packages from alternate sources and then upgrade back into mainline...
If someone else builds a package (mtier, for example), then you can install that package from them, and then upgrade later to a newer / different version of the same package from someone else later. You can have as many package sources as you want just by adding them into the PKG_PATH environment variable.
If you want to build a custom version of a package - in the event that you somehow need one - then you can do that. This is equivalent to modifying the port, which you would do if you were personally updating the port to a new version (before the maintainer did it for you), or adding a new variant, or patching the port in some way. The resulting custom package can generally substitute for the 'official' package, so long as you haven't somehow broken it. If the mainline port is then updated, you can upgrade to it without issue.
If you don't want to upgrade a particular package, then just don't upgrade it - the system will not complain at you unless some other package eventually requires the newer version.
> Apt has an "install from source" system
Building from ports is literally building a package from source. The resulting package is the same as every other package, and will even be signed with your own package signing key (if you set that up). It can be cleanly removed, upgraded, etc., just like everything else.
The OpenBSD packages system is pretty great, and I've used apt. I personally find the pkg_* tools easier to use and more transparent than apt-*/dpkg, but I am sure this just boils down to familiarity. YMMV.
This is great! I didn't know this was possible, and it's been one of my biggest complaints about ports on FreeBSD (I don't know if FreeBSD supports something similar now, but it didn't as recently as a few years ago).
Apologies for starting the thread off with some misinformation and seemingly incorrect negativity. I have had many, many, very bad experiences with ports, mostly on FreeBSD. I made some assumptions that OpenBSD ports was equivalent to FreeBSD ports. I also based my comments on out-dated information.
On the other hand, AFAIK, my "misinformation" is also the "common wisdom" in the Linux sphere on the subject. So I guess I have (unintentionally) used the "Linux can't do X" (or in this case, BSD can't do X) rhetorical trick to prod you into providing the hard facts that were needed in this conversation.
I'm not sure whether I feel bad about doing that. I guess I could have couched my assertions about apt in "so this is what I believe"s. Basically, the reverse of a "this is what I hear you saying, am I understanding you correctly" statement—instead, something more of a "this is what 'my' side is taking as axiomatic assumptions, I think; I'll list mine, and you can list yours, and then we can have a discussion without implicit knowledge." Feels very much like http://lesswrong.com/lw/np/disputing_definitions/, when I put it that way.
I don't think I completely understand the second statement. Sounds like a system like poudriere and similar to me. Also you can do the same on per-port basis. I do that, for example when I want to install a package that got released a minute ago I simply update the like that has the package version with the new one and install it. This works out great. That package is then like any other package on the system, and can be cleanly removed, or upgraded from into a newer upstream release, etc.
Especially on edge cases ports and packages seem to work way better than yum/apt, cause the complexity and the number of things that break is lower. Also how packages work is easily comprehensible (something I always enjoyed about Arch Linux too). This isn't exactly true for .deb or .rpm.
Hotfixes are another great example where ports are superior. As described above the formats in which ports are described are incredibly easy compared to how things work in the RPM/DEB world. We don't have many sysop people at our company, so time for that is precious. Using FreeBSD and its ports system actually reduced the overhead on package-related stuff a lot.
OK, name three of each.
After about ten years of experience with ports and apt (and some minor experience with zypper and yum) I have to say that this as well as most other statements you made are wrong.
I broke things with yum and apt, never with ports. Regarding knowing what port installs what. That's indeed possible, the statement that it isn't is wrong. On the port level itself you have PLISTs on the package level you have even more.
> With ports, it is likely you will, over time, end up with multiple versions of some things (libraries, languages, etc.) installed. This is common on Linux, as well, but the ability to determine what is built against which versions, and to be able to keep up with whether, say, an insecure version can be removed without breaking things on the system, is much weaker (non-existent, as far as I can tell).
Wait, what? Have you ever used ports/the BSDs? Actually this particular problem, being able to cleanly decide which version of a library or piece of software I want to use is one of the main reasons for me to use BSD in favor of Debian or CentOS. Actually that was the reason to switch from a Debian/Ubuntu setup to FreeBSD and we never regretted it. The worries about everything related to packages just went completely away with that change. It really is a main benefit of the BSDs in general.
Did you ever have to deal with multi repositories on a Debian based system, maybe using backports, maybe even requiring unstable branches. It can be a horror. One is constantly between software that's too old to compete or unstable or when upgrading sometimes too new. It is an enormous pain and you can keep a team of sysadmins or devops busy trying to work around all the problems that appear when you do that using apt.
Even things like security updates often make their own problems. Especially when these things try to be smart about which services to restart. Either your server restarts while under enormous load and your infrastructure has a giant hickup that may lead to downtime or you update OpenSSL and things remain as they are.
Another problem is how packages often get modified, split, etc. and behave completely different everywhere.
One more benefit of the ports system is configurability. I set up a system like FreeBSD's poudriere, have a simple, global configuration, specifying whether I want documentation with my packages, whether I want Xorg-Dependencies for my command line tools, things like which databases I want to use, but also and that's probably one of the greatest things, whether I want my packages to be built for Postgres, 8.4, 9.2, 9.3, 9.4 or whether I want to use PolarSSL, GnuTLS, OpenSSL, LibreSSL, just with one single option.
The port infrastructure prevents a lot of duplication of work. Many features added to it simply become available for every port there is. You don't need some maintainer to have exactly the same setup and (compile time) configuration as you. That is a huge plus.
Actually many systems like Arch ABS to build packages are basically a ports collection themselves. Many ideas were taken by projects, like the RPM-based OpenPKG. None of them are as powerful as what the BSDs offer, though Gentoo's Portage comes close.
Speaking about reliably rolling back versions. That actually is yet another reason for me preferring ports. It tends to work way easier than trying to do that in apt, at least when you want to do that in a sane/safe way. But to be honest, it is not the thing you need to do on a daily basis.
Another feature is to be able to hack into it, if it really necessary. You have way more, easily accessible possibilities to quickly change something. Try to do that on apt or yum and you will easily end up with a broken system. That's what I consider fragile.
Also your statement about checks on packages is simply wrong. Why wouldn't you be able to? Other than that I wouldn't rely my security on that. Package Managers are no intrusion detection systems. If you use them like that you may want to overthink that approach.
Because you couldn't until quite recently, as far as I know. As far as I can tell signify, which seems to be what enables signed packages and file signatures, shipped with OpenBSD 5.5. My knowledge is out of date...but if you've been using ports for 10 years, you should remember the time when it was not possible. You should also remember a time when most of my complaints were true, even if they are not, now.
It does appear that my knowledge on the subject of OpenBSD ports is out of date, and I'm sorry for spreading misinformation on several points above. It wasn't my intention to simply trash talk ports...I've been hoping someone would fix it for decades, but, it's only recently (in the last ~5 years) actually started to happen. I apparently just wasn't paying attention when it did.
"Package Managers are no intrusion detection systems. If you use them like that you may want to overthink that approach."
I've done a lot of post-intrusion detective work. On a system that does not have an IDS, it's awesome to be able to compare the installed packages to a known-good RPM or deb. It's more complicated than simply "rpm -V", but not much more, and it can be done without having an IDS configured before the intrusion happened. I can see how if your packages were built custom for the system, this would not be useful (you couldn't verify against a known good version from the web, as it would be unique to your system).
But, it's also useful for non-security purposes. Dumb mistakes, like permission and ownership.
I'm glad pkg_check adds these features to OpenBSD ports.
You don't describe your experiences in detail but it sounds like if you have snowflake requirements where you need to track up-upstream closer, that you're better off building the source debs.
The part where you write off verification checks as also sounds like it is based on a misunderstanding. It is something you do for audit requirements, not something you "rely security on". It is something most professionals would have come into contact with at least once.