Homebrew: OS X’s Missing Package Manager
engineyard.com
engineyard.com
Also, installing software on OS X is really difficult. Going from saying "I want VLC" to actually having VLC is an annoying multi-step process on OS X. Google for VLC's website, find the download link, download, wait, mount dmg, drag-and-drop app to app folder, unmount dmg, remove files, empty trash, launch app. On Linux, I just say "apt-get install vlc", go get a cup of tea, and type "vlc" to run VLC. Much easier.
# Take ownership of /usr/local so you don't have to sudo
sudo chown -R `whoami` /usr/local
Hell no, bad idea, do not want.If your single-user system is compromised to the point where it matters what files you can and can't write, you've already lost. And the "low hanging fruit" in this attack scenario was the files you actually cared about.
(Also, this is a dupe of the top-rated comment. Maybe read one comment before replying?)
Am I missing something in the grand-parent that suggests that his is a single-user system?
Even if it is the case, then, as you pointed out elsewhere (http://news.ycombinator.com/item?id=1096327), the solution's still sub-optimal: better `~/bin` than `/usr/local` (right?).
(Also: this doesn't address the security concerns, but by default on a clean install of SL there is no /usr/local, so you're really not hurting anything.)
MacPort's self-sufficiency is its strength. Apple's system is managed by Apple, via Software Update. Depending on and modifying that system is foolhardy. What happens when the next point upgrade of OS X updates a library to a non-compatible version? Or what if you need ruby 1.8.7, but Leopard's ruby is at 1.8.6 (1.8.7 was a large release)? Your package system doesn't talk to Apple. Apple's update either breaks you, or your updates break OS X.
In fact, MacPorts once linked to OS X's libraries, but they found out how brittle this was, and switched to the current system.
OS X is not a Linux distribution, and it is not organized and defined by its package management system. If you want the flexibility of an open source Unix package system in OS X, you have to build one separately from Apple's Emerald City. The thing is, you can, and it works pretty well.
The problem I see most people having with the MacPorts / OS X duality actually stems from improper management of their shell's PATH. One way to manage namespace collisions is this: place MacPorts bin directories below the system PATH:
export PATH="/usr/local/bin:$PATH:/opt/local/bin:/opt/local/sbin"
and then create symlinks in /usr/local/bin to specific MacPorts executables that you would like to have on the top of the search stack. That way, you'll use Apple's binaries by default, and then you can cherry pick the MacPorts binaries that you would like to use in place of OS X's defaults. The inverse of this works quite well too.> Dependency resolution and updates are basic or not working yet.
I'm sure this is in the works, but my feeling is that managing dependencies and updates are the raison d'être of a package management system. Without this, you just have an installation management system; a shortcut around `make' and `ln'.
Ouch!
But… don't sudo!
Homebrew can be used with or without sudo, but, OS X was designed to minimise sudo use, you only need it occasionally. For example, as long as your user is in the admin group, this just works:
cpan -i MP3::Info
Using sudo all the time is annoying, but far worse — it conditions you to type in your root password without thinking about it. Homebrew compliments OS X so you are unlikely to install anything that really needs to be chown:root. Let this be your last sudo for some time:
sudo chown -R `whoami` /usr/local
NOTE: If you already installed, eg. MySQL into /usr/local then the recursive chown may break it. Fixing MySQL should be as simple as:
sudo chown -R mysql:mysql /usr/local/mysql
Also 'inane blogspam' seems a bit much, no?
You don't have to sudo
It's up to you
I didn't say that the README insists on not using sudo. I was responding to a claim that the Engine Yard blogger made up the method of reassigning /usr/local to a regular user. That's simply not true.
Say for instance you have a dev server, so security isn't as big a deal* used by a team of people, all of whom would want access to the same basic packages. It seems that /usr/local/ or some other world-visible place would be the location for this stuff.
That you don't have to be root is the same way that you don't necessarily have to be root to install programs in the usual MacOS way of running a GUI installer. If you are part of admin, you're already privileged to sudo anyway, so the protection you get is primarily psychological; you type 'sudo', and it triggers the part of your brain that pays much closer attention to what you put on the command-line.
But as I mentioned to rbanffy, it is very possible I am missing a key risk that makes this a very bad thing to do. I am open to enlightenment. ;-)
Then again, making /usr/local writable by admins goes along with the OS X convention of giving admins write permission to /Applications but I'm not a fan of that either.
(For non-average users, the hacked binaries will secretly steal your personal information, so that's bad. But this only happens in real life... well, never.)
This becomes a much larger problem if something runs things from /usr/local with increased privileges -- something from launchd run as root, for instance.
I was originally thinking just from the perspective of the active user, but a lot of things get run without the logged in user's knowledge, and with a varying level of privilege.
Ok, yeah, I agree with the original poster of this thread: yikes, time to start chmodding things.
Anyway, back to the dangers of using sudo with Homebrew or Macports. When you build software by hand, without a package manager, do you ever do it as root? I sure don't. But that's exactly what Macports does in its default configuration, and that's what Homebrew would require if you used it with a non-writeable /usr/local. It doesn't have to be a malicious attack, either: just a simple bug in the build system would suffice. For example, if a "clean" rule in a Makefile has a step that looks something like, god forbid, "rm -rf $(BUILD_ROOT)/" and you're running the make as root, you'd better hope that BUILD_ROOT is not defined to be "".
Ideally, all of these package managers would separate the step of building from the step of installing so that you would only need root privileges for the latter, a la Debian's sbuild and friends. Then using a traditional privileged /usr/local directory would be a no-brainer, for the theoretically smaller attack surface it would provide. But none of the Mac packaging systems work like that, to my knowledge. They go for the convenience of a single build+install command. Given that, I'll stick with my user-writable directory and risk the hijacking scenario.
running a fetch as root or blindly changing the permission structure of /usr/local should scare everybody.
I see from the Macports 1.8 release notes that privilege dropping is supported. That's good news, but is it the default? The release notes indicate that you need to pass command-line options to get this behavior. I assume that most people just "sudo port install foopkg".
> blindly changing the permission structure of /usr/local should scare everybody.
On Mac OS X (as of Snow Leopard, anyway), there are no permissions to change, since /usr/local doesn't even exist until you create it. Nothing in the OS itself depends on it or anything that lives there. There's nothing special or privileged, neither by design nor by convention, about /usr/local on Mac OS X. Making it user-writable and using it with a package manager is no different than creating a 2nd home directory for that purpose. If you're running that rare bird that is a multi-user Mac and co-managing packages with other trusted users, then you can simply make it group-writable by those users. It's safer than giving them any sudo privileges.
$ port build wget
$ sudo port install wget
> "is no different than creating a 2nd home directory for that purpose"
except that a million build scripts and make files out there assume /usr/local/ as a default location. setting up something like '/home/shared/' is not a problem, just don't mess with paths that are defaults
> except that a million build scripts and make files out there assume /usr/local/ as a default location. setting up something like '/home/shared/' is not a problem, just don't mess with paths that are defaults
Yet Macports puts everything in /opt/local and that doesn't cause any significant problems. I haven't run across a free software project in years that didn't have support for an install prefix.
Anyway, I should have been more specific: on a Mac, making /usr/local user-writable and using it with a package manager is no different than creating a 2nd home directory for that purpose, from a security standpoint. I agree that putting things in /usr/local is the most logical place to put compiled software (mainly for consistency with one's own scripts that run on multiple platforms), and that's one of the reasons I use /usr/local with Homebrew instead of something in my home directory.
I'm actually pretty excited to try out homebrew and install everything in ~/ with my new MBP which I'll purchase as soon as the next refresh comes. Currently I'm still on 10.5 with a system that has been forward ported via Time Machine backups and account copy-on-install going back to a 2004 PowerBook. The prospect of having a completely vanilla system with all customizations isolated to my home directory is pretty exciting.
I suspect that there are security implications regarding this when done on more of a server-style environment. Perhaps that is what you were saying 'Ouch!' about?
Or maybe you were thinking of another pain point that isn't occurring to me right now?
However the suggested step is suggested because:
1. On OS X /usr/local isn't even there on a fresh install, so nothing that should run as root is there by default.
2. Mostly people use Homebrew to install a few tools that are missing, these tools don't need to run as root, so they are as vulnerable as TextMate is (because you dragged it from the DMG to /Applications and it installed as your user).
Now you can argue that you should thus install to ~. And you can of course. However I've found over the last eight months that a great deal of build systems have bugs and don't find dependencies if you install the deps outside of /usr or /usr/local. So for a trouble free life we suggest /usr/local.
Finally sudo is dangerous. You should only use it with tools you trust. Do you really trust a half baked Makefile made by a developer who doesn't really know how to write Makefiles?
All in all I think depriviledging /usr/local is safe and the least hassle for Homebrew on OS X.
Rudix features a world class collection of pre-compiled and ready to use Unix compatible software which are not available from a fresh installation of Mac OS X but are popular among other Unix environments. Here you can find utilities, programming languages, libraries and tools delivered as standard Mac OS X packages.
The packages are pre-compiled dmgs, easily downloaded and installed, or you can download the whole suite if you prefer.
The nice thing here is that if you only want coreutils and you don't want to install Xcode to get gcc, you don't have to.
http://news.ycombinator.com/item?id=872072
It's worth a quick read, imo.
I still like the git-recipe idea. Perhaps MacPorts should have a Git repository of 'rouge' community ports they can periodically review for merging and then inclusion.
From my own experience: beware with moving that directory after installing some formulae somewhere else. It caused quite some trouble for me and i wound up recompiling everything in the new directory.
Finally! At last! I hope this takes off, because I've been manually ∂ownloading and compiling everything in order to avoid conflicting versions, redundancy, et. al. that come with Fink and MacPorts.
Every other non-language-specific package manager blithely assumes that it's the only one on the system, so they assume that nothing is already present, and don't even think about the possibility of independent unprivileged use.
1. Where do you want the software?
2. Are elevated privileges required to put the software where you want?
If you want to load all your software into $HOME/bin, then more power to you and no sudo is required. However, /usr/local serves an important and valid purpose (to provide packages system-wide), and it should not be available without elevated privileges. (By the way, 'not mixed in with system software' is misleading, since OSX ships with an empty /usr/local. Nothing you put in /usr/local mixes with anything.)Homebrew's official README suggests changing the ownership of /usr/local, and I think that's a terrible idea for multi-user machines. (Having said that, I suppose that many people have single-user machines, and in that case the issue is largely moot.)
Finally, a simple way to put software into your $HOME without ever needing sudo is available in most cases:
./configure --prefix=$HOME/usr/local does start off empty on OS X, but it's where every random .pkg you download likes to put stuff. You'll get mixing unless every non-app you install comes from a package manager. On FreeBSD every package that didn't ship in the default install goes there. There's a reason that MacPorts uses /opt/local by default.
1. You're adding to OS X. It's very important that stuff that gets built definitely didn't touch any files in /usr or /System. Build systems for some projects are big pieces of half arsed software. It's quite a risk — and an unnecessary one at that — to trust them to not break the bit of the system that only Apple fully understands and controls.
2. Homebrew is for developers. I dev software. Mostly I am installing x library and will want to upgrade and fiddle with it readily. Sudo is basically unnecessary for most of Homebrew's packages. Although I agree some stuff should probably not be installed without a sudo. But with Homebrew at least you can decide which to do on a per package basis.
Also, this isn't rationale, but it's still worth pointing out: most people install software on Mac without a sudo anyway (eg. almost everything in /Applications), so it's not such a big step to suggest you do the same for the handful of tools you need that OS X doesn't provide.
(Had a lot of pain trying to install GTK through homebrew and finally gave up. If someone succeeds, please ping me :)
The wiki page has some interesting notes on integrating it somewhat with scripting language modules (cpan/easy_install/gem).
Explain to me a pain point of Macports that homebrew solves.
I dunno - there's probably more, but that's a quick list.
What really drove me to move was that I was having trouble with Snow Leopard not being supported very quickly by MacPorts. Even once everything I needed was working, I still had problems with CouchDB and couchdb-lucene. I'm not sure what the problem was, but I tried a few things that were suggested, and nothing worked.
So, I did the whole Scorched Earth thing and started fresh with Homebrew. Everything worked right away, and quickly. I found formulas were added very quickly. I found some that weren't in the main repo (I think couchdb-lucene was one of them) and they were all understandable and worked well.
All in all, I just had a much nicer experience with Homebrew than with MacPorts. I'm not hating on MacPorts, I'm just saying Homebrew hasn't caused me one moment of grief since I started using it. Well... maybe one, because I had to find and install the OS X package for Git to get started with installing everything else ;)
It doesn't require you to install a second, shadow Perl, Python or Ruby in order to install other packages. Instead Homebrew uses the Perl, Python and Ruby that OSX already provides. (This may be a con, depending on what you think of Apple's treatment of those packages. I grant that.)
I found the Python issue to be a particular pain in the ass, since MacPorts's Python build requires tk, which requires some GUI libraries, which ends up building more or less an entire X11 sub-system on OSX that I will never ever use.
I ran into this when trying to install Getmail. My solution then was simply to download Getmail and install it the old-fashioned way. I was a little surprised that Getmail built so easily using the system's built-in Python. My first thought was, "If I can do it that easily, using what I already have, why does MacPorts force me to download and build all this other crap?"
What annoys me about MacPorts (which I love in many ways, don't get me wrong) is that it is not easy to customize builds. You can edit the portfile easily enough, but every update undoes that.
There are certain things that just don't work right when using the Apple provided interpreters for particular languages, even if the versions are right. Though I don't recall anymore what the specific problem was, I recall needing to install a custom build of PHP to get a particular library I depended on to work. I wish I could remember more of the details. The point is that I can understand a modest amount of distrust for the built-ins.
There's also the matter that the internal versions of the interpreters run the risk of becoming dated, staying that way, and perhaps becoming so dated that the needed code can't run without later version of the interpreter than the OS provides. Discovering whether it is "too dated" sounds like a fragile process, you'd have to either parse the interpreter version or make assumptions based on OS version.
The solutions I see available are: to leave it up to the user to figure out if a newer interpreter is needed, or just assume the internal interpreter won't work and build your own that you're much more certain will work. This comes down to a matter of philosophy, where MacPorts says "we'll figure it out", and Homebrew says "we trust you know what you are doing."
I found the Python issue to be a particular pain in the ass, since MacPorts's Python build requires tk, which requires some GUI libraries, which ends up building more or less an entire X11 sub-system on OSX that I will never ever use.
This seems like more of a criticism for the python port maintainer than the entire build system. It might be worth posting a request for Tk support to be made a variant of each of the python ports.
For the record, I use MacPorts over Homebrew for many of the reasons on this thread. I've been responding to people saying "What's the point?" because I see some of the benefits of Homebrew, even if I'm not fully convinced. (I suppose that I'm also frustrated at the common interwebs reaction of "But I use some other thing, so this new thing must suck." Homebrew may not be the answer to everything, but it's very worth looking at. There are some very good ideas there to some common problems.)
Having said that, my complaint about Python was explicitly specific to that package. That's why I said "the Python issue to be a particular pain..."
The Tk & Python issue has been brought up repeatedly on MacPorts lists and posts about MacPorts. They seem to prefer it that way, and for all I know, they have good overall reasons. Follow-up: Well, I haven't checked in a while, and I'm dead wrong. The getmail package currently depends on Python 2.6, which now does have a no-tkinter variant. Shows what I know.
Edit: By the way, the PHP on OSX is notoriously light on modules. There have also been a few pretty ugly issues with Perl and core modules. So I'm not denying that there are real reasons to worry about the built-ins. I said as much in my post above. There are pros and cons to each approach.
There is so much less friction than using macports or fink that I dropped them immediately. Definitely check it out.
Why is it better for me than MacPorts and why should I use it?
Here's why you may prefer Homebrew to the alternatives:
Zero configuration installation
Copy the contents of this directory to /usr/local. Homebrew is now ready for use.
Or… install anywhere!
You can actually stick this directory anywhere. Like ~/.local or /opt or /lol if you like. You can even move this directory somewhere else later. Homebrew never changes any files outside of its prefix.
The GoboLinux approach
Packages are installed into their own prefix (eg. /usr/local/Cellar/wget) and then symlinked into the Homebrew prefix (eg. /usr/local).
This way the filesystem is the package database and packages can be managed with existing command line tools. For example, you can uninstall with rm -rf, list with find, query with du. It also means you can install multiple versions of software or libraries and switch on demand.
Of course, you don't have to do anything by hand, we also provide a convenient and fully-featured four-letter tool called brew.
You don't have to sudo
It's up to you.
Create new package descriptions in seconds
Package descriptions (formula) are simple Ruby scripts. Generate a template with:
brew create http://foo.com/tarball-0.8.9.tgz
Or edit an existing formula: brew edit foo
Package descriptions not requiredMacPorts doesn't support the beta version? Need an older version? Need custom compile flags? The Homebrew tool-chain is carefully segregated so you can build stuff by hand but still end up with package management.
Just install to the Cellar and then call brew link to symlink that installation into your PATH, eg.
./configure --prefix=/usr/local/Cellar/wget/1.10
make install
brew ln wget
Or Homebrew can figure out the prefix: ./configure `brew diy`
cmake . `brew diy`
OptimizationWe optimize for (Snow) Leopard Intel, binaries are stripped, compile flags are tuned to your exact Mac model. Slow software sucks.
Making the most of OS X
A touch of RubyCocoa, a cheeky sysctl query or two and a smattering of FSEvent monitoring. In these manic days of cross-platform development, it's can be a welcome relief to use something that is better because it isn't too generalized.
No duplication
MacPorts is an autarky -- you get a duplicate copy of zlib, OpenSSL, Python, etc. Homebrew uses what is already there, and consequently, most stuff has zero dependencies and builds faster.
We resist packaging stuff that is already packaged. So we have a wiki page that describes how best to use RubyGems, Pip (or easy_install) and CPAN with OS X and Homebrew.
Fork with Git
The formula are all on git, so just fork to add new packages, or add extra remotes to get packages from more exotic maintainers.
Surfing the cutting edge
If the package provides a git://, svn://, cvs:// or hg:// url you can choose to install that instead and then update as often as you like.
MacPorts made the decision to package and version their own versions of everything for a reason. Conflicts can start to arise when you start compiling against Apple's default shipped libraries for X11, Python, etc. Especially if the default shipped versions don't provide what you need. E.g. the PHP install that shipped with Snow Leopard is missing several notable extensions.
It might be safe to ignore these issues now but as the app grows they will need to be addressed. (Or of course they can ignore it and just point to MacPorts as an alternative for some subset of packages).
Ubuntu (Debian) seems to do this very well in my opinion.
sudo aptitude update; sudo aptitude upgrade
And with 9.10, to add a repository you can just do ppa:$repository
To add Wine for example, ppa:ubuntu-wine/ppaEDIT: Grammar
For example:
$ brew remove git Error: git has multiple installed versions
I've looked at the (very sparse) 'brew help' and the wiki and still haven't found a solution other than manually deleting the extra version directories.
Am I missing something obvious?
$ brew cleanup git
Uninstalling /usr/local/Cellar/git/1.6.5.4...
Uninstalling /usr/local/Cellar/git/1.6.5.7...
Uninstalling /usr/local/Cellar/git/1.6.6...
Or just "brew cleanup" to clean everything. (That doesn't appear to work with projects built from version control repos, though, like emacs with Cocoa support.)edit: Maybe you meant that you want to remove the whole thing, i.e., all versions, not just the unused ones. Looks like you have to "cleanup" first, then "remove". That seems like a bug to me.