How to Distribute Binaries for OS X Using Homebrew
octavore.com
octavore.com
Absolutely. I adore Homebrew and it's ecosystem. Cask, in particular, is quite satisfying; gotta love installing a swath of neccessary fonts from the terminal. And, fwiw, the project is about to be on Homebrew, with no tap required: https://github.com/Homebrew/homebrew/pull/49040 Though getting it up was not quite as pain-free than our initial publishes to NPM.
The thing is: Homebrew solves the problem for just a portion of your audience. In diff-so-fancy's case, we've supported Linux and Windows from the get-go, the only hard part is getting it installed. NPM solves that problem really well, and I can't imagine any other installation technique being our recommendation for Windows/Linux going forward.
> Can install software to a home directory and so does not require sudo
> Install software not packaged by the native distribution
> Install up-to-date versions of software when the native distribution is old
> Use the same package manager to manage both your Mac and Linux machines
It could also likely be used so make a "safe" Linux distro, where the base system is mounted read only, with a writable home partition.
NPM does a great job of distribiting development dependencies and Node.js applications.
It's not really intended for non-js binary builds and platform-specific dependencies.
You can always use NPM to install directly from specific repo release but you'll still have to have to include a native compile-to-platform step somewhere.
In my experience, cask is too fragile for its original use of installing binaries. Issues include
* applications expecting to be installed to /Applications
* applications with weird installation requirements/dependencies
* self-updater conflicting with cask's installation settings
* problems with detecting an installation (and uninstalling it) because the updated version's recipe installs differently or something.
For these reasons I now prefer to install applications by their official/expected/supported installation method.
On the other hand, I find brew very useful.
The brew command for users would be:
brew tap octavore/delta https://github.com/octavore/delta.git
brew install delta
---
Example above of course doesn't currently work because the formula is not there
(I successfully use it in my repo: https://github.com/stepanhruda/ios-simulator-app-installer)
brew tap octavore/delta https://github.com/octavore/delta.git brew install delta
This is probably asking for a PR to hombrew, the ideal command would be
brew install delta https://github.com/octavore/delta.git
(1) You're pushing down bits onto users' machines that may not be necessary (like developer documentation, or development versions of your source code if you're developing on master), and
(2) 'brew update' will need to download (via 'git pull') all changes to your repository any time you update, even when the Homebrew formula doesn't change.
Keeping formulas in a separate repository allows Homebrew to work like it's supposed to (and like it does for built-in formulas): an update to your tap repository means there's updates for the formulas that it contains, and you're not downloading any source code except what's actually going to be used, at the time it's going to be used (during 'brew install').
> when a post on better diffs showed up on Hacker News
> last week ... people below are asking why a bash script
> (that depends on a perl script) is being recommended to
> install via NPM? [cont. about using Homebrew]
In the mean time, you can tap it:
% brew tap OJFord/formulae
% brew install diff-so-fancy
well.. yeah.. Every OS has an easy way to distribute binaries, the hard part is having something that work on each of them.
* (the window manager sucks, but that's not the topic. I had to get it out)
* it needs a good package manager
For us ol' bearded folk (at least spiritually), OS X comes with a slew of unix tools we love. It already includes both vim AND emacs! However, these tools are hopelessly outdated: Despite OS X 10.11 "El Capitan" coming out last fall, Vim is still 7.3 (that's from August 2010. 2010!), and Emacs 22 (2007. That's right.)
So some enterprising folk come to the rescue: Fink, Homebrew, Macports...
Homebrew installs over /usr, so come the next OS update, all that gets squashed. Macports installs into /opt/local, but somehow doesn't survive an OS update. Also, when I tried installing SciPy it went and recompiled GCC4 from source. Guys, the 90s called, and it's been shown that recompiling from source for each user isn't worth the effort.
Apple gave MacPorts some support in the form of equipment, so there's some hints of their preferred method. But the core WTF is this: why doesn't OSX integrate a package manager for its unixy side?
Do I wish the 3rd party ones could be better integrated with the OS? Sure. But I'm definitely thankful it isn't Apple calling the shots (or more likely, releasing it and then refusing to call any shots at all for 8 years).
And that's way more up to date than it used to be.
> Apple gave MacPorts some support in the form of equipment, so there's some hints of their preferred method. But the core WTF is this: why doesn't OSX integrate a package manager for its unixy side?
Yeah who hasn't dreamt of a package manager handled like MAS.
I'd be happy if Apple helped package managers more, contributed (financially) to them and better took them in account or discussed stuff with them wrt major releases (see: Yosemite homebew woes), but I'm really quite glad the community is completely in charge of macports or homebrew or nix.
I don't see how it'd solve the primary MAS woes of horrible contributor/developer experience, uncaring maintainership and inflexible sandboxing.
I was imagining a package manager with the standard kind of "add whatever sources to your sources list you like" model, as most Linux distros have. If the MAS was rebuilt in terms of this, then presumably adding a custom package source would make custom apps (which, after all, are just packages) show up in the MAS app for selection as well, just as they'd show up in e.g. Ubuntu Software Center. Apple would only control their app/package source.
The interesting question is whether someone would be able to list an app in their third-party source as costing money—and, if so, how that would work, and where payment would go. Maybe each "source" could come with a payment processor token for the MAS client to interact with, additional APIs to let the MAS app grab payment receipts from your source server, etc. In other words, you'd be hosting less of a "package management repo" and more of a "store server with built-in package CDN"—but that's not too big of a change.
Now, I wouldn't expect Apple to actually go this far, but it'd be in line with their current philosophy on enterprise app distribution. Apple really really dislike third-party "app store" apps, third-party "updater" daemon apps, etc.; they would much rather build infrastructure to obviate your enterprise's horrible custom solution than try to have their CSRs understand and support it for you. I could see them building this thing if Microsoft, Adobe, etc. all bugged them to for long enough. And then you as an indie developer could take advantage of it by setting up a PPA-alike (and someone could create the equivalent of Launchpad to host yours and everyone else's.)
OSX actually has softwareupdate(8), which does work like this—and which at least the OSX system updates (managed now through the MAS app) execute in terms of—which could theoretically be fed custom catalogs (sources) in addition to Apple's, to get most of this part of the benefit. Never heard of anyone doing that, though.
That's not entirely true. A relatively widespread issue during 10.10 (Yosemite) updates was the installer seemingly locking up and actually moving all the files in /usr/local one by one to then from a backup location veeeery slowly, potentially taking >12h to complete the update rather than a pair of.
Besides, even if the installer did do that, the resulting updated OS still had all the same contents in /usr/local.
As far as I'm aware, the only change OS updates have ever made to the /usr/local folder is resetting permissions back to root:wheel.
because that's exactly what the logs showed it doing: https://discussions.apple.com/message/26856483#26856483 and because the preemptive fix (if you hadn't started the update yet) was moving /usr/local's content elsewhere: https://jimlindley.com/blog/yosemite-upgrade-homebrew-tips/
And it was for 10.10. I'm sure people who'd been bitten (or had avoided it through the procedure above) repeated it for 10.11 reflexively or just in case, but the issue was the 10.10 upgrade.
Incidentally, I had no idea about ⌘L to show logs during OS update.
In any case, even if the OS update takes a long time, the end result is you still have the same /usr/local you did before.
Also, FWIW, the discussion you linked to references TeXLive, as does the blog post you linked. I wonder if the issue here is not in fact having anything in /usr/local, but rather having used a package installer to install something into /usr/local (which I assume TeXLive does). It doesn't make much sense to me for the OS updater to move everything out of /usr/local and then back in one-by-one under normal conditions. But if you used a package installer to install into /usr/local, then it would make more sense for it to do something like that (which is to say, it may have special behavior around things covered by package receipts).
Edit: Well, under normal conditions, it might still move the folder away and back, but I'd expect it to just move all the top-level items from the folder back, instead of manually copying all of the nested files).
However, if everything you're installing that depends on Homebrew-provided software is also installed through Homebrew, then it should work regardless of location (I say "should" because I suspect that not every single Homebrew formula has actually been tested with a non-/usr/local install path).
Personally, I stick with /usr/local both because I've never had a problem with it, and because that makes it easier when working with software that uses a default PATH (since /usr/local is usually put in these default PATHs). But if you have any concerns whatsoever, then by all means use a different install prefix.
Sadly, you are correct about "not every single Homebrew formula has actually been tested with a non-/usr/local install path". In fact, while there are many Homebrew formulas of great quality, my impression is that there are also m many of rather poor quality, with other problems in them. E.g. just yesterday, I discovered that the Homebreak lmdb formula does not build properly versioned shared libraries -- because upstream didn't, and the packager either didn't notice or didn't care to fix it
I rarely (if ever) this kind of issue with MacPort and Fink, which try hard to avoid such problems (at the cost of a higher entry barrier for packagers -- for better or worse). Mind you, I am not everything is golden with them, either.
Disclaimer: I've been involved with the Fink project since OS X 10.1 or so.
> the core WTF is this: why doesn't OSX integrate a
> package manager for its unixy side?
> Ah, you just need to update your packages; install foo:
> macman -Syu && macman -S foo
It's more likely that they just consider package management out of scope for their part of development-- something that benefits too few people to be worth the cost. Seems to have paid off, too-- package management has been well-handled on OSX for a long time, both through semi-sanctioned projects like MacPorts (which is community-developed but is hosted by Apple) and through community projects like Fink and Homebrew.
> there are lots of other "power user" things that are
> built in
I thought someone might bring up AppleScript - but I think that's a relic of every user being a 'power user', and personally I really doubt it'd be created today if it didn't already exist; it probably doesn't take much to keep it around now that it is.
If you like tiling window managers Amethyst is worth checking out. It's a little unpredictable sometimes but otherwise it works well.
MacPorts is very, umm, opinionated. And doesn't seem to trust anything outside of itself. And has some weird opinions, too.
Example: For some reason, svn is a dependency for nmap. Forgot those gravity waves, the next nobel prize belongs to whoever can figure out why a network scanner depends on an obsolete version control system. So macports installs a shiny new version of svn. Which doesn't play nice with the svn that Xcode uses. Overtime I run (macport) svn from the command line it tells me to upgrade my repository, which breaks the Xcode svn.
I use ctrl-left and ctrl-right to switch virtual desktops. Can't remember if that's the default or if I used custom keybinding. There are a bunch of other window management keybindings that can be found in the OSX System Preferences.
The worst offender for MacPorts was TeX. I had an official install the latest version from TeX Live and MacPorts wanted to install a second version of TeX. I think they fixed it so you can use a TeX Live version, but it was definitely an issue.
Agree. I really like Spectacle (https://www.spectacleapp.com/) for window management on OS X. As a bonus, it's open source. (https://github.com/eczarny/spectacle)
Macports handles this usecase far, far better. Give your users sudo rights and be done. Only thing it desperately lacks is allowing third-party additional repositories like Debian's apt does.
indeed, but couple of things
specific to Mac OS X: there are homebrew, macports, fink and probably more
distributing with homebrew to someone who already going with macports is problematic for example as it is advised to not have both installed.
One alternative is to distribute a pkg and so use PackageMaker to build one.
A pkg has the pros to be native to Apple, but the cons to not work for other systems like Windows and Linux.
So here another alternative: Debian packages (deb).
Wether with macports, homebrew or fink you can install the command line utility dpkg, and using dpkg-deb you can build deb supporting the darwin-amd64 arch.
For Windows, you can use the utility wpkg, http://windowspackager.org/ and with it you can build deb packages for win32 and win64 architectures.
Ultimately if you distribute command-line tools (not GUI app), you can use deb packages for Windows, Mac OS X and Linux Debian (Ubuntu etc.)
Here an example (one of my tools that I need to distrib/install on servers): https://github.com/Corsaair/as3shebang/releases/tag/1.0.0
Debian packages for everyone :)
Linux and Mac OS X install: dpkg -i as3shebang_1.0.0_amd64.deb
Windows install: wpkg -i as3shebang_1.0.0_win64.deb
You can see the build scripts at the root of the repo, they are pretty similar.
Also, once you have deb packages you can also convert them to rpm and other nix-like packages usinga tool like alien, see https://en.wikipedia.org/wiki/Alien_(software).
Anyhow, it's nice and simple approach that works.
1. Jump to separate repository
2. Create a .tar.gz with sensitive filename
3. "upload somewhere"
4. Manually update formula, sha, version
4. Push
Versus:
1. npm bump && npm publish
I ended up automating the formula updating after getting tired of doing that every other day, for a private tap.
Linux got it right: https://en.wikipedia.org/wiki/Package_manager
It drives me crazy that Apple removed one the most amazing features of *nix systems: a unified software repository. Brew/brew cask doesn't come close to AUR.
It's not a feature of unices, it's a feature of _Linux_ (Ian Murdoch called it "the single biggest advancement Linux has brought to the industry"[0]).
The vast majority of unices never had a "unified software repository", and BSDs (probably the least indirect ancestor of OSX) have port trees, which at the time OSX was born (ignoring the NeXTSTEP ancestry which predates "modern" BSDs and builds directly from 4.3BSD) were a very different beast and not something suitable proprietary unices (because Apple wasn't going to give the OSX commit bit to randos).
And hell, both apt and rpm were busy _being born_, the premier package managers at the time were raw dpkg and pms(1). And pkgtool[1] I guess?
[0] http://ianmurdock.com/solaris/how-package-management-changed...
In this popular diagram, https://upload.wikimedia.org/wikipedia/commons/7/77/Unix_his..., NeXTSTEP was originally forked in 1988, before Linux really had package management put together. Darwin is from 2001.
Is 14 years not enough time to establish a proprietor supported unified package management system?
My expectations may just be set too high.