Thoughts on macOS Package Managers
saagarjha.com
saagarjha.com
It's totally fine as my code was python3 compatible, but I was surprised that they'd apply a breaking change like this silently. Unlike OSes which might do this kind of thing with a major version update, I was surprised, and it took me a little while to diagnose what happened.
When I checked the GitHub discussions about the change, I found two things: 1) The maintainers claimed they had given a soft deprecation message a few months prior -- but brew doesn't stop for user input, it just posts walls and walls of messages, so I had naturally tuned it out. 2) The maintainers were getting ballistic mad at people who didn't think this was handled well.
Of course, open source is open source, and maintainers owe us nothing, but I was surprised that such a critical piece of infrastructure was being managed by people who imho handled a potentially non-trivial transition like this and who clearly didn't have the diplomatic tact to handle their userbase.
Part of the problem is technical--IIRC they don't have any CI infrastructure to test upgrade processes--but mostly they just don't care. They'd rather have the cleanest, most elegant code today, even if it means breaking everyone who installed it yesterday. That philosophy makes sense for a lot of projects (like in SaaS, where you often deploy new versions by creating new virtual machines and installing everything from scratch), but I'm not sure it's the most productive way to run a package manager.
Does that ever happen with OSX.....
At this point it seems like Homebrew is the standard -- I frequently see open source projects recommending it in their installation instructions without mentioning Macports.
I also recommend Homebrew to friends and coworkers, for the reasons I already mentioned, and also because it is significantly easier to install. Go to their website and the first thing you see is an installation script. Go to the MacPorts website and you see a link to an installation page partway down. That installation page is very long and lists a number of prerequisites needed for the installation to work. Homebrew does the installation bit way better, thus making the barrier to entry far lower.
MacPorts would need to demonstrate significant advantages over Homebrew to get me to switch back.
Getting started: https://medium.com/scientific-breakthrough-of-the-afternoon/...
How much effort was put into making the whole isolation: https://www.youtube.com/watch?v=73mnPBLL_20
There is even a declarative way to setup darwin process management: https://github.com/LnL7/nix-darwin
MacPorts' philosophy seems better in theory, but in practice it has two flaws that turned out to be very significant for me. The first is that you end up needing a huge number of ports, partly because you're duplicating the tools already on the system, but partly because in many cases you need all of those ports' dependencies as well. So upgrading anything was a huge processing sink. I wanted fifteen or twenty ports on my system, but to get those MacPorts had to install several hundred dependencies. If you ever decide you don't want one of those ports, the dependencies are still installed, and pruning the ports tree was decidedly nontrivial.
The second was dependency hell. You end up with at least two versions of everything. Which one random third party apps pick up depends on search paths. Different third party apps need different versions, and I ended up twiddling with search paths a lot. Tryin to install ROS as a standalone third-party package was a nightmare.
Homebrew has its issues, but it doesn't have these issues.
Ultimately the real problem here is that macOS package managers are bolt-on additions and not a core part of the OS. when that's the case there is not going to be any perfect solution. But for a single-user machine where the main goal is to be able to maintain a minimal set of installs, and install new ones quickly so you can get real work done, Homebrew just worked better for me.
Your last point brings up an interesting question in my mind regarding Apple's intentions for the CLI. On the one hand, they seem to care enough to add features here and there to Terminal, but then when you consider that the tools have basically zero chance of upgrades being incorporated for licensing reasons, I wonder if they plan to even have a stance on package managers, or if they are completely fine with that being purely in user land. The entire CLI experience on macOS concerns me in as much as the OOTB set of tools will continue to work, but how much more fragmented will the experience get? There's already a good chance at running across some shell code online that requires bash 4 or relies on a newer gnu version of a tool. Maybe this long tail of developers is too niche for Apple, but I imagine it's still important to many of their developers in some way.
Homebrew is certainly "opinionated" and, as the article says, "abrasive".
I recently had one of those upgrade chains: new iOS needs new Xcode, new Xcode needs new OS (Mojave), new Xcode and OS need new Boost. All fine up to this point.
I'd installed Boost via Homebrew. So, fine, brew upgrade boost.
A bunch of stuff scrolls past... holy ---- it's upgrading Postgres?
Oh great. Now a whole bunch of work I have stored in a Postgres database is inaccessible. No problem, Homebrew apparently has a Postgres upgrade script now.
Except it wouldn't work, complaining "no upgrade path" from PostGIS 2.4 to 2.5.2; and Homebrew wouldn't install both at the same time. Eventually, after much experimentation, I found Postgres.app came with all the right versions, allowing me to pg_dump it all out.
But my word... while trying to figure this out I spent hours looking through the issue tracker, and the state of it. Endless snark about "we don't support old versions". Fun like this: https://github.com/Homebrew/brew/issues/5675#issuecomment-46...
It's a good tool, just don't expect any support with it, ever. I kind of feel that Homebrew's slogan should be "Homebrew locked as resolved and limited conversation to collaborators".
Also github has a lot of use outside of public repositories. If I’m not the admin of my own issue tracker, I’d switch to one where I am.
I don't like this. A maintainer can fake another users' comments for some very nefarious purposes.
AFAIK the reason maintainers can completely remove revisions of comments, is because occasionally a well intentioned user pastes sensitive info into a comment and doesn't realise.
While pointing it out to the user so they can remove it themselves does work, that's an unbounded time frame (perhaps never). So just nuking it directly is "safer" in that instance.
Doesn't seem like there's any really perfect solution.
As a user I'm not a big fan of the order of their CLI switches. For example to restart PHP-FPM:
brew services restart php@7.1
In Ubuntu the same thing is
sudo service php7.0-fpm restart
Should restarting a service require sudo? I think it should.
Should the service name be first and then the operation? I think it should as well.
Good luck convincing brew maintainers of this. It's like talking to a brick wall.
sudo systemctl restart php7.0-fpm
I tend to prefer this syntax because I can perform an action on multiple services with a single command.It just always seems like between "just get it kinda working" and "work how I expect" it often picks the former.
Then there's fink mentioned in the article which was popular long ago which literally was a port of Debian stuff.
At some point I was using pkgsrc from NetBSD.
So a more Unix person's type of tool exists.
Its a bit of work to switch of course, since I use a decent amount of brew packages.
But we’ve also said we’d improve [1] and I feel that the effects already show [2] [3]. We’ve been communicating changes earlier and more clearly lately. We’ve learned a lot from user feedback, and we’re nowhere near the end.
Sad to see you go though! Happy to welcome you back anytime.
[1]: https://github.com/Homebrew/brew/issues/142#issuecomment-214...
[2]: https://twitter.com/MacHomebrew/status/1119288776830926850
[3]: https://twitter.com/MacHomebrew/status/1119502934000029696
Everyone who interacts with the Homebrew community must be treated with respect, no exceptions, and if we maintainers suck at that then we need to be better.
As soon as I saw that homebrew wanted to use /usr/local instead of /opt, I avoided it. These days, /usr is part of the system, even though GNU still uses /usr/local as the default.
/opt matches where the LSB says "3rd party packages" should be installed, MacPorts is (or was) partly sponsored (and thus "blessed") by Apple and it matches the "port" packager that is used for FreeBSD.
Yes you have to do some PATH and LDPATH tricks to make sure that /opt/local/* is at the top of your paths, but other than that, MacPorts has been painless.
There are some steps to follow when a new version of Xcode (and effectively a major release of MacOS) comes out, but that's more about getting xcode to accept you've accepted Apple's license.
> MacPorts almost always builds packages from source
That’s a misconception. As of 2019 a properly-configured MacPorts will download almost exclusively binaries.
From the beginning it was as if no package managers had ever existed and they were solving packaging problems for the first time ever and ignoring not just the other Mac package managers but other package managers in general.
Then there’s the smattering of cutesy names for internal concepts: bottles and casks and tapping.
Popularity of Homebrew skyrocketed, though.
Meanwhile, MacPorts is better than ever.
Glad to hear MacPorts works well for you! Should you ever decide to come back to Homebrew, we’ll be there for you.
In software, I’ve always preferred the more serious tone. No idea why; I love silliness and monkeying around in general.
For example, Homebrew used to support compiling on both 32-bit and 64-bit systems. They provided a "is 64 bit" function that packages could use to check if they were being compiled on a 64-bit system. It worked great!
Then, Homebrew decided they no longer wanted to support 32-bit systems. Instead of doing what any reasonable project with a stable interface would do and modify the "is 64 bit" function to always return "true", they just deleted it! I found out when one of my users emailed me to let me know that my packages crashed when you tried to install them.
Changes like these mean that I spend 20% of my package-maintainer time on real work and 80% dealing with Homebrew's nonsense. Moreover, they show that Homebrew doesn't care at all about the people who maintain packages outside homebrew/core. And as someone who maintains packages outside homebrew/core, that makes me not want to continue.
I had a project that has Ruby set at version 2.5, one day Homebrew turned it into 'ruby@2.5' and now 'ruby' was 2.6 and whenever @2.5 also upgrades, since there is no global symlink, a minor version change breaks my IntelliJ setting as the path has slightly changed.
I was using Ansible from my local Mac but dropped it thinking this might just break one day. Linux distribution vendors do far better jobs at keeping things 'as is' for years.
The only use for Homebrew for me is to install individual tools that don't have wider impact if it breaks (like ripgrep).
If you mess with a program’s internal bits in an unsupported way (like the author of the linked blog post purportedly did), you can be happy that your changes will only get reverted on the next update and not blow up in your face.
https://www.patreon.com/homebrew
Wonder if they'll use it for paying maintainers or something? It's a useful amount for planning things out with. :)
As a minor data point, the creator of Homebrew was Adam Vandenberg, not Mike McQuaid:
> MacPorts almost always builds packages from source
Many packages in MacPorts (including LLVM) have pre-built binaries available from https://packages.macports.org/ and its mirrors for macOS releases ranging from Mac OS X 10.5 to macOS 10.14.
But if a port is considered not distributable because of its restrictive license or license conflicts with dependencies, binaries will not be uploaded to our official mirror so users have to build it from source. We don't provide official binaries for non-default variants or alternative prefix (the default is /opt/local) either, but organizations using MacPorts can set up private mirrors with custom ports and binaries.
port_binary_distributable.tcl[1] describes the license conflicts. e.g. "git" is not distributable because its license "gpl" conflicts with license "OpenSSL" of dependency "openssl".
> MacPorts seems to be a bit lacking in manpower (which makes things takes slightly longer than I would have expected)
Yes, we need more contributors and committers to maintain the large collection of ports we have. Our ports are more often outdated than Homebrew because of this and our maintainer policy. We are also seeking for ways to improve our developer experience, e.g. https://github.com/macports/macports-base/pull/120 which automates updating checksums in the Portfile.
> The lack of color, as well as somewhat more cluttered and less relevant output, makes it a bit less pleasant to work with.
We talked about this in a meeting last year, but it hasn't been implemented yet. https://trac.macports.org/wiki/Meetings/MacPortsMeeting2018/...
Development of the port command is relatively inactive (severe lack of manpower). https://github.com/macports/macports-base/commits/master
> MacPorts will set up its own tools in isolation from those provided by the system
Its own versions of libraries, to be exact. See https://trac.macports.org/wiki/FAQ#syslibs for more background on this.
> builds run in “sandboxes” under the macports user, where attempts to access files outside of the build directory–which includes system tools–are intercepted and blocked
It will block writing files outside of directories specified in portsandbox.tcl[2], but using (reading and executing) system tools is allowed. For example, most ports are built with the compiler from Xcode.
> The upside is that this approach is significantly more contained, which makes it easier to manage and more likely to continue working as macOS changes under it.
Yes, but we recommend users upgrading to a new major release of macOS reinstall MacPorts and all ports because things could still break. We have a migration guide here: https://trac.macports.org/wiki/Migration.
[1] https://github.com/macports/macports-infrastructure/blob/mas...
[2] https://github.com/macports/macports-base/blob/master/src/po...
> Many packages in MacPorts (including LLVM) have pre-built binaries available from https://packages.macports.org/ and its mirrors for macOS releases ranging from Mac OS X 10.5 to macOS 10.14.
Most of my packages seem to build from source. Perhaps this is because I run on the bleeding edge (macOS Developer Seeds, macports-base/macports-ports built from ToT master)? Maybe I've triggered this by virtue of some configuration option I've set?
> Yes, we need more contributors and committers to maintain the large collection of ports we have.
I've been trying my best to help by updating whatever "port livecheck installed" gives me, provided the maintainer policy allows it :) I am curious about new ports, though: what is the policy on these? Should we aim to "mirror" packages that homebrew/core adds? Does MacPorts have any equivalent for Homebrew's Caskroom?
> Development of the port command is relatively inactive (severe lack of manpower).
I "tested the waters" with a bug for a (IMO) usability improvement, https://trac.macports.org/ticket/57950, but didn't get much of response. I have ideas for ways to improve the ergonomics of the port command (and a willingness to write code to back it), but I'm not sure what the process is to get started on this.
> Its own versions of libraries, to be exact.
Yup, this is what I meant. Poor word choice here.
> It will block writing files outside of directories specified in portsandbox.tcl[2], but using (reading and executing) system tools is allowed. For example, most ports are built with the compiler from Xcode.
Again, poor word choice. As far as I understand, MacPorts uses the compiler toolchain from Xcode (or the CLT if you have it installed?) but will block access to random "things" that the user has installed to /usr/local.
> Yes, but we recommend users upgrading to a new major release of macOS reinstall MacPorts and all ports because things could still break.
I have not used MacPorts long enough to have had to do this, so I was mostly talking about MacPorts being immune to Homebrew's perennial issues with Apple changing permissions on some folder that Homebrew thought they "owned" or changing some framework/command/header that they depended on. Tangentially related, has there been any work done on making this migration between OS versions less jarring? What kinds of issues crop up?
Oh, and finally, is there any easy way to get macOS's graphical applications to find MacPorts's binaries? I've been trying a bunch of random stuff related to launchd's environment variables but they seem to be really finicky and a lot of "methods" I found online don't seem to work anymore. And official suggestions?
Maybe the port is not distributable, I've listed several situations in the parent comment where ports would be built from source, if none applies in your case you can ask on the macports-dev mailing list posting the output of port -v install [port].
> provided the maintainer policy allows it
We welcome any pull requests, the maintainer policy is for committers merging PRs.
> I am curious about new ports, though: what is the policy on these? Should we aim to "mirror" packages that homebrew/core adds?
Generally, any open-source software can be added to MacPorts. Feel free to add software Homebrew provides, but don't limit yourself to that :)
> Does MacPorts have any equivalent for Homebrew's Caskroom?
No, but you can use a custom ports tree (https://guide.macports.org/#development.local-repositories). In some cases (e.g. OpenJDK) we would repack binaries, but we prefer building ports from source (on our Buildbot or during install if binaries are not distributable).
We could discuss this on macports-dev. Creating a repository on GitHub and writing (un)install instructions is all we need to get started.
> https://trac.macports.org/ticket/57950
Our Trac tickets are somewhat neglected, please post on macports-dev if no one responds. If the change is simple, you can also make a pull request which is easier to review and merge.
> has there been any work done on making this migration between OS versions less jarring?
Yes, but the PR is stalled for now (lack of manpower again). https://github.com/macports/macports-base/pull/56
I haven't experienced these issues myself because I would always reinstall MacPorts after a major macOS upgrade.
Why does this license conflict prevent from distributing the tool? The tool is under the GPL, which requires making the source code available, which it is, and it also allows linking against OpenSSL, which has its own rules, but none seem to disallow distributing binaries that come with it.
This article convinced me about the conflict: https://people.gnome.org/~markmc/openssl-and-the-gpl.html.
The OpenSSL license imposes certain restrictions on people wishing to distribute their program using OpenSSL, while the GPL-2.0 license says you may not impose any further restrictions when you redistribute the program. There is an exception for OS components, but MacPorts is not part of macOS.
kinda irritating
Care to elaborate a bit on the permission problems?
big fail imo
But I just want to say thank you very much to The Homebrew team, if you're reading this, I really appreciate your efforts.
No sale.
MacPorts is old, but I think that's a good thing: it's robust and well designed–at least compared to Homebrew, I feel. It's lacking a bit of color and a few emojis, but usability isn't that bad (in fact, I would say it has fewer surprises, because commands usually do exactly what they expect you to and nothing else behind your back).
> I don't pipe bash into shell, so no sudo for you either, mister package manager.
Sorry, what? I don't know that the first part means, and the second is standard across almost every system package manager.
However, OS X and macOS definitely were.
$ wc -l /etc/passwd
108 /etc/passwdOS X already provides enough UNIX to my taste, no need to look beyond it.
Everything else I need is part of the SDK.