Homebrew 1.9.0
brew.sh
brew.sh
Since I don't install 'mission critical' application through brew, I've always had the following command run daily though a cronjob:
brew update && brew upgrade && brew cleanupWhat I mean: "Unlike cron which skips job invocations when the computer is asleep, launchd will start the job the next time the computer wakes up. If multiple intervals transpire before the computer is woken, those events will be coalesced into one event upon wake from sleep."[1]
==> This operation has freed approximately 8.9GB of disk space.Overall I hate how slow brew is, especially `update` should be 10x faster, it should maybe warn about source builds that could take forever, and it should maybe give a summary like `apt` whether there are packages to be `autoremove`d.
Is that documented anywhere? I guess it makes sense for most people, but I often do `brew update && brew fetch $(brew outdated)`, then later rely on `brew upgrade` working without hitting the network. I'd hate for that to change.
[1] https://www.postgresql.org/docs/11/pgupgrade.html
[2] https://github.com/Homebrew/homebrew-core/blob/master/Formul...
==> This operation has freed approximately 41.3GB of disk space.
Automatic cleanup would have saved me a lot of confusion.
==> This operation has freed approximately 343.7MB of disk space.
Wait, that's not trivial.Wow, thanks
There are several updates in here that make me worry that this is somehow going to break my environment or otherwise bite me in the ass when I do something innocent like install Ghostscript and find that Emacs no longer works and to figure out how to fix it's going to be hours of reading github issues. Which is an experience I've had before, repeatedly.
"brew update no longer migrates legacy keg symlinks, tap names, repository locations, cache locations or cache entries."
Is this going to break something I installed years ago? Maybe the homebrew maintainers know, but average tool users don't.
"Homebrew/homebrew-core requires all formulae to be open source by the OSI definition."
Does this mean that something I used homebrew to install in a previous version will no longer be getting updates? Will things that depend on it still be getting updates? Does whether or not updates happen depend on some random committer's legalistic interpretation of an open source definition? Why exactly is this kind of purist decision being imposed on users?
[in 2.0] "brew cleanup will be run periodically and for trigger individual formula cleanup on reinstall, install or upgrade. You can enable this behaviour now on 1.9.0 by setting the HOMEBREW_INSTALL_CLEANUP variable."
So you're going to periodically make maintenance changes to my system without my permission. Goodie.
Someone really needs to write a guide to migrating away from homebrew for people in the unfortunate situation of having used this tool for a long time, and find it taking more and more liberties with one's computer, but also having numerous essential packages now managed by it and not sure how to move to alternatives without weeks of breakage.
This is not true. `man brew` documents how to disable auto-updates.
> Why exactly is this kind of purist decision being imposed on users?
Because there's a lot of current debate about whether things being relicensed under the Commons Clause is "open source" or not and we'd rather lean on an established, well-documented authority on what that definition is. Additionally, part of our agreed membership of the Software Freedom Conservancy (who help us with a lot) involves ensuring our core formulae are open source software.
> So you're going to periodically make maintenance changes to my system without my permission. Goodie.
Again, `brew man` documents how to disable this.
---
The complaint behind the individual complaints is "why doesn't Homebrew work the way I want it to". That's a reasonable complaint and there may be better tools for you to use rather than Homebrew but we're going to continue to build Homebrew to work in a way that works best for the majority of our users and avoids our (entirely voluntary) maintainers having to deal with the same issues again and again.
Otherwise it just looks like a bunch of scary behavior changes w/o a clear upside.
I do think, however, that users greatly underestimate how much time it takes to write up a good changelog. Even one that is as simple as this.
Cheers, and thanks for your work on homebrew. It really is a great tool.
For what it's worth, I really do value this stuff which is why we have release notes at all, a Discourse forum, etc. most of which I've setup and run :)
Hopefully these changes will allow you more time to focus on adding value to the vast majority of users. If a small subset of folks need the marginal functionality, it should be up to them to step up to the plate and make it happen.
> Hopefully these changes will allow you more time to focus on adding value to the vast majority of users. If a small subset of folks need the marginal functionality, it should be up to them to step up to the plate and make it happen.
100% agree.
For those who haven't had a chance to dig through, prompted by this, I searched through the man page [1] on GitHub and apparently the environment variable to set is:
HOMEBREW_NO_AUTO_UPDATE
https://github.com/Homebrew/brew/blob/master/docs/Manpage.md
I suspect this is partly because the cache can consume a huge amount of disk space over time and many users might never run `brew cleanup` or doesn't even know about it. And then wonder where all their disk space has gone.
The biggest thing that has kept me on MacPorts and away from Homebrew is that I don't understand half of the terms that they use.
What is a keg? What is a tap? From the article you linked, what is a cask? What is the Caskroom? Elsewhere I've heard talk of "bottles" - what is a bottle, in this context? What are formulae?
Keg: a dedicated subdirectory for each installed package.
Cellar: this is where the kegs are. Usually `/usr/local/Cellar`.
Formula: a file that tells Homebrew how to build and install a specific CLI package. (Similar to a portfile, I guess?)
Tap: repository of formulae, e. g. `homebrew-core` (the repository where Homebrew keeps all the core formulae)
Bottle: pre-built binary package. Homebrew usually builds and hosts them for you. At install time, Homebrew downloads and uses them whenever it can so you won’t need to wait through unnecessary source builds.
Cask: Represents a package that can’t be built from source. In practice, a Cask usually means something that ends with `.app`. Technically, Homebrew Cask is very much a separate package manager in itself. (You can tell by looking at the individual `brew cask` commands.) That’s because it caters to somewhat different use cases, and because it used to be a separate project before it went on to merge with Homebrew.
Caskroom: directory hierarchy for Cask-related bookkeeping. Caskroom is also the name of the former organization in charge of Casks; but that’s history now given all things Cask have officially become part of Homebrew.
It’s cool that MacPorts works well for you but hey, maybe try Homebrew some time to find out how you like it? (Disclosure: am one of the core maintainers.)
I am a little bit worried for the removal of all options that is planned for the 2.0.0 version: I have some formulae installed from source (with additional options) and I like the flexibility that they give. Hopefully the migration to an "options-free" Homebrew will be smooth and some alternative when options are needed (not requiring a lot of third-party taps) will emerge.
Homebrew has gone out of its way to make this more difficult lately, to the point where 'brew doctor' complains when HOMEBREW_BUILD_FROM_SOURCE is set, and now it's deprecated.
Homebrew's bottle infrastructure is a tempting target if you want to get malware deployed to Macs, and as best I can tell, there is scant documentation out there about what they do to secure it. For all I know, the bottles are built on some random bozo's Mac.
The snarky side of me thinks that if build from source is discouraged then the program should rename itself to Macrobrew (as I'm sure Budwiser would sue their pants off if they used that name).
We have private operations documentation and have had private security reviews of our infrastructure. We are actively moving away from self-hosted infrastructure this year.
Note: I'm not denying that you have a stellar track record, but it would be nice to have more evidence that this is not down to luck or obscurity but is instead due to process, best practice, and understanding of risk.
It is frustrating that you are being so opaque and dismissive. You still haven't answered my earlier question if the Homebrew project has published a summary of the results of these security reviews (I understand not posting the entire review publicly). A quick Google search did not turn up anything, which is why I am asking.
For contrast: I can find out the build process for a Debian package from their website[1]. While they do have some private operation documentation, they also publish the process by which packages get pulled into their system, built, and pushed to the mirrors[2][3]. They have documentation for how to replicate their build environment and build packages on my own[4]. This documentation is open, and I can verify packages with it as they move toward reproducible builds[5]. I understand that Debian is a much larger operation with a much longer history. I understand that it takes time to develop these things. This is not an attack on the Homebrew project or the work they do.
[1] https://www.debian.org/devel/buildd/
[2] https://www.debian.org/devel/buildd/operation
[3] https://wiki.debian.org/Teams/FTPMaster
I have looked at the Homebrew docs page. There is a document linked that describes how bottles are built[1], but it's not clear who has access to it and what safeguards are in place to prevent a malicious maintainer from spreading malware through it (such as who reviews the commits) and it doesn't list the precautions taken by bintray to prevent and detect tampering with packages (and a user has to trust that they as in place, sufficient, and trust bintray to not tamper with them). Another page says that most formula pull requests need to be reviewer but does not go over what this entails[2].
This alarming text, however, does appear in your maintainer guidelines [3]:
>Verify the formula works if possible. If you can’t tell (e.g. if it’s a library) trust the original contributor, it worked for them, so chances are it is fine. If you aren’t an expert in the tool in question, you can’t really gauge if the formula installed the program correctly. _At some point an expert will come along, cry blue murder that it doesn’t work, and fix it. This is how open source works._ Ideally, request a test do block to test that functionality is consistently available.
Is there a set of maintainers who handle security sensitive formula, like openssl, gnupg, and tor?
[1] https://docs.brew.sh/Brew-Test-Bot
[2] https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request
Here you go:
> Do you have published security reviews of your infrastructure?
Beyond https://hackerone.com/homebrew/hacktivity, https://brew.sh/2018/08/05/security-incident-disclosure/: no.
> Otherwise you're asking us to just trust you.
Yes.
> it's not clear who has access to it
A subset of the documented Homebrew maintainers who manage our ops/infrastructure.
> such as who reviews the commits
The maintainers and additional trusted users.
> the precautions taken by bintray to prevent and detect tampering with packages
packages require a manual publishing step, are checksummed and frozen on their CDN a while after publishing.
> most formula pull requests need to be reviewer but does not go over what this entails
This is part of the on boarding and training of new maintainers.
> Is there a set of maintainers who handle security sensitive formula, like openssl, gnupg, and tor?
Not a specific set but they receive extra time, review and attention to their source URLs.
---
I expect some of that will not be good enough for you in which case: please either get involved to help us make it better (we are unpaid volunteers) or use another tool. This is how open source works.
I had an experience years ago where I discovered the lack of quality control: a typo (presumably) caused a package to regress back a couple of years. My issue used wording like "Are you serious?" which was taken as offensive and I was banned, unable to even apologize or defend myself from resulting attacks.
brew is convenient and I still use it, but I have a much higher degree of trust in packages shipping with OS distributions.
You can put your money where your mouth is and consider submitting one to our HackerOne. Otherwise: perhaps it's not "quite easy" after all.
for i in `brew ls`; do brew info $i | grep -e 'Built.*with:' >/dev/null && brew info $i; done
Out of 144 formulas installed, I've only got one with an option (mpv --with-bundle). I hope the Homebrew folks will use that option as default, but I suspect they won't, and will recommend the cask instead. Which isn't really what I want, but ok.No need to use the Cask version if you don’t want to. If you feel stuck with the tap thing, you’re welcome to get in touch on discourse.brew.sh and we’re glad to help!
If that’s not what you want, you’re free to dig up the formula with the option, and continue to maintain it on your own tap. There’s good documentation for that [1], and there are always people in our forums willing to help if you get stuck [2].
[1]: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap [2]: https://discourse.brew.sh
One of those annoying things that is valid from a "lets keep things backwards compatible" point of view that ends you up with the wrong versions of everything you care about as a developer. I fixed it by uninstalling node and then installing nvm and then doing a nvm install 10. Of course having per tool custom hacks around "your distribution comes with the wrong version of everything you care about, 100% guaranteed" gets old pretty quickly. Brew fixes this on Mac.
On most servers on the other hand, you should not be installing developer tools or runtimes at all and use something like docker instead. Installing stuff like node.js, python, ruby, etc. via the default package manager is almost always the wrong thing to do on both development machines and production machines.
Speaking of docker, having something like brew on a lightweight image could be a pretty awesome way of installing what you need without relying on downloading tar balls and other madness that you find in a typical Dockerfile. Why reinvent that wheel?
I haven't come across linux brew. Sounds like a good idea. Merging the two ecosystems sounds like a better one. Having that on windows as well; awesome. Simplifies a lot of stuff developers have to do on different platforms to end up with the same tools.
But distributions often say that it's the only way to keep the whole "ecosystem" of a distribution stable.
So? What I think is that Linux "distributions" are growing less and less useful nowadays. On the desktop, no Linux was able to gain significant traction, and many developers are just using Macs and Windows with WSL.
On the server side, ways to "build your own distro" for creating a tuned stack for your purposes ( read: docker images ) exist and seem more useful than standard distro components. Nixos is another take.
Yes, "just apt install foo" is nice, until it isn't. I think we're at a tipping point between the old and new approach.
In practice, it doesn't work. I did a Debian Stretch `apt-get upgrade` and it pulled in a new version of firefox-esr that did not run on my Rockchip Chromebook. Debian was forced to do it because it was end-of-life for the older firefox-esr, and Debian's security backports policy doesn't scale for projects like Chromium and Firefox.
Of course, Nix also works without root and has many other benefits ;).
- You can create the nix store in some directory you own. But this entails recompiling every package, because the precompiled binaries have dependencies on the store being in /nix.
- If user namespaces are enabled, you can bind mount /nix to some directory that you own:
https://nixos.wiki/wiki/Nix_Installation_Guide
Of course, it is more convenient if you can convince a sysadmin to install multi-user Nix. From there on, every user can install their own packages (in a non-conflicting manner).
And all you can do is hopefully find a trustworthy PPA that gets timely updates or you end up downloading it yourself and get them updated manaually.
I wanted to use Linuxbrew but this seems still not too mature.
This move along with the python2 vs python3 change that was made previously has left a seriously sour taste in my mouth about Homebrew. I'm actively looking at other options.
I really encourage you to create a tap for the formulas you like to customize. Taps are a powerful, much overlooked feature baked into Homebrew.
Where does this constraint come from? Homebrew is written in Ruby.
To cut down on compilation from source issues? Or Homebrew uses language features from Ruby versions greater than what Apple shipped in their latest 32-bit supported OSX versions (as homebrew insists on using the system default binaries)?
In other words: at bottle creation time (server-side), Homebrew tells the compiler to output code that runs on older CPUs. The downside of this is that end users who install bottles never get to enjoy the benefits of latest CPU instruction sets, such as SSE 4.1/4.2.
From time to time, Homebrew raises the bar to keep the balance between performance and compatibility. From Homebrew 1.9.0, bottles are built to use the features of the Penryn/Core 2 microarchitecture or higher, or in other words: Homebrew no longer supports CPU microarchitectures older than Penryn/Core 2.
This is exactly what the bottle bot does, building the same formulas you have on your machine. Why should this no longer work?
While I agree in that building from source is somewhat at Homebrew’s core (as that’s what happens at bottle creation time), the intention is to deliver core packages to end users via bottle as often as possible.
> Linuxbrew does not currently support 32-bit x86 platforms. It would be possible for Linuxbrew to work on 32-bit x86 platforms with some effort. An interested and dedicated person could maintain a fork of Homebrew to develop support for 32-bit x86.
Chocolatey is... ok. It works, mostly, but it's a common occurrence for upgrades to fail because the package maintainer forgot to update the hash, or the program's built-in autoupdater already ran and moved things around, or a phantom .install dependency is missing, or some other inscrutable problem. Competition will be good for it (and the developer sells a premium version, so I do expect proper competition). All the other package management tools on windows have fairly tiny repositories and are, as far as I can tell, optimized for the workflow of sysadmins building images.
I wonder if Brew has any plans to support that package management framework Microsoft is apparently still working on? (https://github.com/OneGet/oneget)
An explanation for this change would be useful.
One major problem is that system and Homebrew versions of library may have incompatible ABIs.
It was my understanding that if you wanted in your PATH you just did `brew link --force php@7.1`. That's not possible anymore? (Since MacOS provides a php binary) What's the recommended way for easily getting a specific version of php in your PATH?
> This would have avoided Homebrew/homebrew-core#34754
which links to: https://github.com/Homebrew/homebrew-core/issues/34754
tl;dr One user had binutils force-linked, causing cmake to pick up the user-installed flavor of binutils, altering cmake’s behavior and causing issues.
Linuxbrew has existed since a few years now (https://github.com/Linuxbrew). We just merged the linux part of the code into the main brew repo.
https://github.com/Linuxbrew/homebrew-core, where the formula reside, is still a separate project. But we all work closely together. The 4-5 linuxbrew maintainers joined the homebrew team and we are now a single team.
This does not add too much work for the initial homebrew maintainers.
The scope has grown a little bit, but nothing to be worried about. And the windows part is "just" Linux running on Windows, so actually there is not much difference there. That part is not so well tested though, because we do not have so many users on Windows, but we hope to be able to improve this part if needed.
On the other side some decisions have been made lately: less from source builds, less options ... this will reduce our workload too.
Currently, about 25 % of Homebrew’s active maintainers work on the Linux/WSL part. I don’t believe that ratio is going to change much during 2019.
If Linux and Windows were to become part of Homebrew’s mission statement, it’d be all too easy for Linux/WSL users to interpret that as Homebrew committing to the same level of support to which macOS users are accustomed at this time.
I feel there’s no way we’d be able to deliver on that commitment and not disappoint users. We want happy users.
I recall that some commands where much faster. It takes on average 2.5 seconds to do brew search.
$ time brew search mpv ==> Formulae mpv
==> Casks mpv brew search mpv 2.34s user 0.59s system 67% cpu 4.348 total
This is not autoupdate — this is with set HOMEBREW_AUTO_UPDATE_SECS to 14400.
https://discourse.brew.sh/t/solved-brew-search-is-too-slow-s...