Goodbye Mac OS Forge, hello GitHub
lists.macosforge.org
lists.macosforge.org
This is the sad truth. It doesn't matter anymore if Bitbucket/Gitlab/etc. are better than GitHub: The social pressure for open source projects to move to GitHub is so high that the alternatives don't even stand a chance.
Maybe a meta-search engine for open source projects could help the competition, but if all projects are on GitHub, who would use such a search engine?
P.S.: I actually like GitHub, I'm just sad that all cool changes that GitLab has made recently will end up being mostly ignored by the OSS community. Even if they fixed their speed problem (which I think is their biggest downside compared to GitHub), no project would move there because "everybody is on GitHub".
If they had a solid sync service between GitLab and GitHub; PRs, issues, code, everything then things could get interesting and some people might consider the switch.
You can also sync any remote repository (two-ways) [1] in GitLab EE (which runs on GitLab.com).
I made an issue for a continuous sync in GitLab [2]. Would love to get your feedback.
[0]: http://docs.gitlab.com/ee/workflow/importing/import_projects... your project from GitHub to GitLab
[1]: http://docs.gitlab.com/ee/workflow/repository_mirroring.html
We'll never learn.
1. Fix the speed problem
2. Build yourself such a meta search engine for OSS projects
What other things do you think we should do?
1. We're working on the speed. It has gotten a lot better with the last few releases. The release of tomorrow is another big step, especially for diffs. We have a number of issues for this, I'm linking one of the meta issue here, but be sure to check the release of tomorrow for graphs and whatnot [0]
2. It's definitely an interesting idea. I made an issue [1]. Love it if you would provide some feedback. I wonder whether we'd be allowed to do such a thing, but it seems to be in the interest of the end-user, which is a good thing.
You can learn more about my search technology at https://gitsense.com/blog
However, it's worth noting that GitSense is not optimized for searching across 1000+ repositories at once and to be honest, this is never a good thing in my opinion. When it comes to code searches, I believe it should be curated, where some domain expert can say if you are working on release x, for open source project y, these are the branches/repos you will probably be interested in and so forth.
Until we have artificial intelligence, like we see in movies, nobody is going to produce relevant code search results, by searching tens of thousands of repos. Software development is completely context driven and what mattered for one release, can become completely irrelevant in the next.
What GitSense is optimized for, is searching across logical groupings of code at the branch level. This can be 1000 branches from 1000 repos or 1000 branches from the same repo/forks.
If GitLab is serious about being a central point for all open source code searches and are okay with providing logical searches at the branch level, let me know. You can contact me via my email in my HN profile. And it's probably worth noting, since the GitSense indexing engine/search engine are completely separate from the frontend, it'll be easy enough for GitLab add their own interface.
Definitions can change from one release to another. Function behavior can change from one release to another and so forth.
Searching across millions of repos is not difficult, if you are willing to accept the matches returned may or may not be relevant. What is extremely difficult, is building a search solution that can take into consideration all the changes, on every branch, from the millions of repos. And this is the problem nobody is going to solve anytime soon.
Personally, I like hosting a repository on Github plus Gitlab for availability reasons, but if Github would finally allow disabling pull requests as well, one could easily make Github a read-only mirror.
In light of non-existence of project patch, ticket, release, wiki interoperability protocols, we will either adopt more of FossilSCM's features into git addons (like git-appraise) or someone will write adapters.
The most concerning aspect of all of this is that projects that have access to hosted FOSS servers which are fast and under (semi-)direct control give in to peer pressure and move to a closed platform like Github.
If, say, Freedesktop.org would run a Gitlab instance, it would prevent the move to Github. I cannot be the first one to think of that.
This would be an improvement of the distributed and highly available characteristics of a DVCS.
for remote in $(git remote); do
git pull $remote master && break
doneCan you explain what you mean by "layered snapshots for backups"? The convention is to push to several remotes because that's part of git's dna, so what's the improvement you're suggesting?
> I still remember all the "free press" github when they nuked all the repos.
What "free press" incident are you referring to?
Git isn't immutable, it is possible to destroy a remote repo. So if one really wants verifiable backups, one needs to take an immutable snapshot of some sort.
> free press
Like the pentium bug, corporate IT departments everywhere suddenly realized that they used github due to the site wide reset. It was a minor kerfuffle on the net that week.
Speaking of disabling, if gitlab allowed disabling pull requests, wiki, issues, etc. and act like a pure git repo, it would solve a problem github refuses to address, namely a project's users wanting to use github/gitlab but the maintainer hosting the primary repo somewhere else. It would also play well with the D in DVCS, which github sort of breaks by leading users to centralize for convenience. I mean, one day github is down, another day gitlab, and so on, it's just how our networks operate.
In GitLab you can already disable merge requests, wiki, and issues per project. See "Edit project".
Today's open source alternatives will be followed by endless more experiments in building a better github, hard to imagine someone "never" nailing it let alone "never" being handed it on a platter because the first website in history annoyed too many of their users at once at just the right time.
Going open source wouldn't have saved digg after everyone decided to break their old routine. They had nearly identical open source competitors with variations of their own interface all along, even direct clones.
None of their customers ever needed it or benefited from it being a proprietary service either. It's probably a lot more important to Gitlab than Github.
In fact, it's a good thing Gitlab is not a pure open source project, because there's leadership involved and business interests with real life problems to address. So, the chances of "endless experiments" is low. Sure, if Gitlab was fully open while keeping project leadership and all, it would be even better.
I think the problem is we conflate selling the only access to our code with selling access to it. If you need online hosting for your project the alternatives are almost immutable - somebody's server running project management software.
No matter what Github does developers are trending towards using open source tools to the extent Microsoft's porting their company to Linux right in front of our faces...
The geeks and nerds have become the bullies. It's sad.
[0] Like automatically running `brew update` with every brew command run. I was literally wondering if my computer had frozen up again when I saw that it had done a full update (which involves not only patching the brew code, but updating its source repos) before installing a pre-compiled version of tmux.
$ env | grep HOMEBREW
HOMEBREW_BUILD_FROM_SOURCE=1
HOMEBREW_NO_AUTO_UPDATE=1
HOMEBREW_NO_EMOJI=1Each time I use a mac I have to relearn what a tap represents vs cask vs brew vs celler etc. But hey, aren't they clever with words!
Chef does the same; it has chef, knife, kitchen, berks, recipes and more. Getting started with it is a freaking nightmare.
Everything has cute logos and names and good looking README.md files to lure you into using it then you're trapped into hell.
There wasn't a problem with MacPorts / Fink, they are built on top of proven mature tech (FreeBSD ports and Debian's apt), but why not reinvent the wheel? All the cool node kids are doing it.
Homebrew has all the markers of a project I should hate, but it's so rarely inconvenienced me in practice that I can't help but like it.
I'm not sure what you were doing, but this is scary. I've been using it since 2012 or so and never had as much as a hickup.
2012 may have been around when I stopped using it, can't recall for sure. Maybe it improved after I dropped it.
HOMEBREW_NO_ANALYTICS=1
[PS: that's not a serious question: it's completely facetious.]
HOMEBREW_NO_ANALYTICS=1In the end it seems that this will get better if it can attract enough OS X users so that patches start rolling in but right now it's just a few people fighting that battle.
[0] https://github.com/Homebrew/brew/blob/master/share/doc/homeb...
This is not accurate. We run a minimal version of `brew update` at most once per minute when you run `brew install` or `brew upgrade`: https://github.com/Homebrew/brew/blob/5504e2c1320d52f8a92a33...
You can set this interval with `HOMEBREW_AUTO_UPDATE_SECS` and disable this functionality with `HOMEBREW_NO_AUTO_UPDATE`.
Well, that's a relief. I run an average of 4-5 brew commands every minute, so at least there's that. /s
> when you run `brew install` or `brew upgrade`
So, only the two most used `brew` commands (for me, at least) will trigger a series of git fetches and merges over the network and an in-place update of the code.
> You can [...] disable this functionality with `HOMEBREW_NO_AUTO_UPDATE`
Why was it enabled automatically in the first place? Why wasn't I notified when homebrew updated that this was the new default behavior. Why must I always dive into homebrew on github to identify (1) why it's not behaving the way I would expect a package manager to and (2) how to get back sane package manager behavior?
If there's been no updates there will be no `git fetch` run. We use a newish GitHub API to check for updates as its quicker.
> Why was it enabled automatically in the first place?
A significant numbers of our created issues are due to people having failures when they don't run `brew update`. This solves that.
> Why must I always dive into homebrew on github to identify (1) why it's not behaving the way I would expect a package manager to and (2) how to get back sane package manager behavior?
We're an open-source project run by volunteers for free in our spare time. We're underfunded and understaffed so our communication is lacking. The best current place to find out about such things is to follow our issue tracker. We aim to improve this in future.
Is it for legacy systems? Or unusual requirements? Habit?
Edit for reasoning: I'm curious because who knows-- maybe I'm missing something really cool or jumped ship to Homebrew years ago too hastily?
Edit #2: Just wanted to say thanks to the folks who have replied below! I'm definitely a bit more curious about macports again.
I wish we could get Octave to our users without any CLI shenanigans, but it's proven very difficult to build an app bundle. We finally have one, but it's not relocatable.
http://wiki.octave.org/Octave_for_MacOS_X#Installing_a_Mac_O...
I'm sure Homebrew could probably be set up this way, but the defaults are a powerful thing.
You said you could probably configure Homebrew in a similar way, but... can you now-a-days configure macports to install without root for user local packages?*
* Pardon me for the potential ignorance of this question being unreasonable or lazy.
A random Google finds this. https://coderwall.com/p/njpsxg/building-macports-without-roo...
My main concrete reason to prefer macports is that it has separate packages for every version of python so I can have them all installed at once for testing. I also like a lot of macports' technical decisions, like the fact that my regular user doesn't have write access to the installation directory. And macports has binary distributions now so the awful rebuild-the-world updates aren't as much of a problem anymore.
That's really awesome.
> ... has separate packages for every version of python so I can have them all installed at once for testing
Are the package versions themselves configured to be installed side by side? Or are you using multiple copies of macports[1]?
[1] https://guide.macports.org/#installing.macports.source.multi...
The un-suffixed version ("python") refers to a default that can be switched with a single command affecting all related tools. It's great.
This is not python specific either, it also works with perl, ruby, etc
time to get familiar with https://github.com/yyuu/pyenv
Maybe it's just that Homebrew turned me off right away. Back in the bad old days, when the Python 2/3 split was more serious than it is now, I had a heck of a time trying to get pandas and scipy running with Homebrew. I haven't given it a try lately so perhaps things are better now.
(I am probably also the kind of person where all the hectoring the Homebrew docs would do about installing to /usr/local eventually got on my nerves ... )
That being said, I don't use MacPorts versions of things like TeX or Ruby (I prefer MacTeX and plain old rvm).
Has this changed ?
I am basing this on my experience of both Snow Leopard and Mavericks ... and in those cases, I had to download and install XCode. Further, this is (I suppose) more a criticism of OSX and how it does not include basic tools ...
http://osxdaily.com/2014/02/12/install-command-line-tools-ma...
Neither do I, but you can get away with just the command line tools. It's even on the install instructions.
The maintainers of packages are supposed to port their software to the relevant platform, that's their job, and yes sometimes this takes some work, whereas Homebrew seems to assume that ./configure; make; make install should flawlessly work out of the box for pretty much anything (this is not true). The fact that they actively discourage even trying to install outside of /usr/local is simply laziness (and not in a good way). This is a foolish and naive approach, and pursuing this idealistic, reductive approach for "simplicitly" breaks down very, very quickly. MacPorts is a well behaved, self-contained system, and a well designed one at that. Homebrew just dumps everything into a system directory that it really shouldn't have exclusive rights to in the first place (/usr/local).
Moreover, there is nothing wrong with installing extra compilers, libraries, etc, under /opt/local and having them run alongside the system libraries. Lots of systems have provisions for doing something like this.
Also, using version control systems as distribution and deployment systems is a horrible anti-pattern and I really, really wish people would stop doing this.
Homebrew's documentation is terrible. Its terminology is facile and stupid in the way it analogizes everything with beer.
I downvoted you. You assert a massive bunch of things with no evidence. A reasonable person could disagree with every single one of them (I do). While I could imagine your reasons for believing some (most?) of these, I would simply be arguing against the strawman I constructed. By providing actual evidence ("why do you believe this to be the case"), you would allow us have a constructive discussion on your points. Instead, you give us bile.
brew leaves:
Show installed formulae that are not dependencies of another installed formula.
this. I'm using homebrew and installing every package as my own user is a security risk. One security mistake in a package can interfere with my main user account's files / other packages / etc.
Other package managers (apt, yum, macports, etc.) install each package as separate users (e.g. Mysql as user 'mysql') to prevent bad packages from 'leaking'.
Next time you go to a meetup, try to do an nmap on your surroundings.
And, it just seems completely idiotic to compile standard packages for the second most popular developer ABI on the planet. How many rainforests have Homebrew users burned down to get the same binaries as the guy sitting next to them?
I've been using pkgsrc lately. Binary packages that seem to work out of box and come with launchd configs. It goes into /opt/pkg/, and /opt was always better than /usr/local because of the unix philosophy that its shorter to type.
Anyway, if you like Homebrew's approach to the filesystem, there was a OS you would really like, it was called Microsoft DOS. (It didn't work out in the long run.)
I find 90% of the brew things I install are binaries (they call them "bottles"), and I don't even use brew in /usr/local (where even more binaries are available). Have you used brew recently or is your experience from some time ago?
Remember leftpad!
I sincerely thought everyone heard of this.
- There are several popular, well maintained, hosted VCS providers, such as GitHub, who have good bandwidth, and host open-source projects for free. Nice as most open-source projects are unfunded. - A VCS gives you traceability in what's changing in your package repository 'for free' - nice when you open it up to contributors. - A git-based package repository can be kept locally in relatively little space, allowing for the case where the package repo host goes down - I've had issues with being unable to install from PyPI in the past because it's down, I have not had that issue with git-based repositories, for example (not a perfect comparison). - A VCS based repository can aid in creating a system where a given state of the repo is consistent, and works together, and can be upgraded atomically. Stackage is doing something like this, providing a known working set of packages at any given release (although I'm not sure if they are backing that concept with a VCS).
Apart from that, version control systems are optimized for controlling versions, not delivering static content.
Can you go into this? I have to write an update framework at my job and I want to know as much as possible about it before I start.
> It is a best practice to assert ones opinion by declaring the alternative as an anti-pattern.
It is recommended, but if you want to run in `/opt/brew` or `~/brew` you'll get a warning some things may not work but that's it. There's no hard block, and the vast majority of core formulae can pour their bottles anywhere.
The only problem I had was that some stuff requiring compilation _outside_ of macports have builds that won't work naturally (e.g. phashion gem).
This is usually solveable and rare, but annoying. OTOH I've seen similar issues with brew too (i.e. need to change openssl from system version to compiled version and recompile stuff)
To turn your question around, who uses Homebrew instead of Macports?
Long answer: When I got myself a Mac (October 2013), I looked for a way to install software, and I was given the advice "use MacPorts or Homebrew, either is fine". So I briefly looked at both and for some reason stuck with MacPorts.
I have not had any trouble with it, so I never had a reason so reconsider.
I think the main reason I preferred MacPorts back then was that it creates its own folder hierarchy for installed software (/opt/local), while Homebrew defaulted to /usr/local. I sometimes will install stuff straight from source, which typically ends up in /usr/local, and not having anything else live there made it easier for me to keep track.
As a bit of history, when homebrew first landed, it existed because the author wanted to pass optimization flags to his builds [0], and found the macports build system too complex for this purpose. Homebrew's real strength is the relative transparency of the build system (it is more obvious what will happen when a build launches), which is really a reflection of this initial complaint about macports. At the time, I didn't care about optimization flags and didn't otherwise get a good vibe from the rest of the project, so stuck with macports for the reasons I listed above (less the binary installs, which didn't exist yet). Since then, I've been a bit surprised at how much Homebrew has taken off, but macports has continued to improve as well and I am still very happy with it.
I have a super easy way to evaluate package managers, so I don't waste my time: can it install useable up to date versions of GNU Emacs (NS variant), TexLive, and Xorg Server, without hassle.
Fink is dead, so we can skip it. Emacs was never up to date on Fink anyway, so I quite using it early on.
Homebrew does not package texlive and has a pathetic error it prints out when you try 'brew install texlive' — this is why I refer to it as a toy package manager.
Nix, failed all of the packages and was crazy slow. Xorg installed, but crashed on launch.
Pkgsrc is the only one that comes close. I couldn't figure out how to install the NS variant on GNU Emacs, but everything else installed (including CLI emacs) and worked. The caveat is that all the packages had really strange names. MacPorts wins here because the packages are name as you would expect: emacs-app, texlive, xorg-server.
MacPorts could make it easier to package software. Hopefully they're working on this.
I would jump ship to Pkgsrc if it had better organizations and included macOS variants of packages.
Sane defaults, respect for system security settings, and a general lack of bullshit. As a *BSD refugee on Mac OS X, it feels like a slice of home every time I need to install software.
Because of the cross-platform focus of the NetBSD developers, I can replicate my package installation workflow on any number (20+ at last count) of operating systems with the same command set. This saves me from the mental overhead of having to pick up Mac-isms, which is nice.
Also, binary installs!
They say they will stick with Trac for dealing with issues. I understand that for managing the ports collection, but are they planning on keeping "base" issues (i.e. the application code) off the github as well?
I don't know how well different repos and issue trackers interop, but it seems like an open-source solution like gitlab would offer the (potential for) customization/integration more readily than github.
I'm sure they just desire a turn-key solution and don't want to manage a whole new project, but maybe, one day, they'll want to unify the experience again.
Do people still buy into this? I feel this phrase was just a marketing ploy by GitHub.
I'll leave the question of "is it a social coding experience?" to the experts.
I just had a look again and found that ffmpeg is one minor version behind behind stable, its binary is annoyingly called "ffmpeg3", and it was compiled somewhere with LLVM 6 / clang 600, versus LLVM 8 / clang 800 on my macOS 10.12 system. I tried to compile it manually (after a shallow clone of their massive, 250,000+ commit git repo), but failed.
For me Homebrew does the right thing here — they basically turn my Mac into a rolling release Linux desktop like Arch Linux, which is great for the userland type packages I use: openssh, zsh, tmux, postgresql, ffmpeg, git, vim, rsync, etc.
That said, there are tens of thousands of packages that build on OSX, certainly many more than enough to be a very useful platform for package management, especially when coupled with all of the advantages that nix brings. The downsides are a higher learning curve and occasional missing and/or broken packages.
I hope to use Nix for go and c programs the same way you use Boot/Lein, Maven, Bundle and Npm for their respective tasks - make a manifest local to a project directory. No more figuring out whatever that library is called in my package manager...
Quite a lot of packages were broken (I fixed 3), but then I found the default nix script (which you are supposed to run every login) broke mercurial, by installing it's own certificate store, and I had to stop using it.
1. Ports define which compiler to use. Sometimes I had to install a small tool and first it would another version of gcc. 2. Ports were outdated a lot. I developed a project using the Grails-framework and it was a couple of versions behind the current stable release. Just checked: Grails port is at version 2.2 https://trac.macports.org/browser/trunk/dports/devel/grails/... while the current version being 3.1.10
With Homebrew the adoption rate seems much higher. Also you don't depend on a single maintainer.
Copyright © 2011 Apple Inc. All rights reserved
The dawn of OS X coincided with the tail end of what was a rather insane open source hype bubble that existed in the industry in the late-'90s. This was the same era where lots of people were convinced that everyone was going to be running desktop Linux in just a couple more years (this is the origin of "this is the year of Linux on the desktop" jokes), and ESR hadn't squandered his credibility quite yet. It was a very different time.
Anyway, point being, MacOSForge is in part a historical artifact of a time when there was an open source bandwagon that a lot of SV companies had jumped on and were encouraging their employees to jump on as well. After a few years the industry restabilized and many companies weren't as publicly gung ho about open source, but for a while at least there were some surprising initiatives from unexpected companies.
I don't miss the messianic/apocalyptic/mystical/tribal/just-plain-silly overtones that surrounded open source hype in the media and in the community in those days, but I do deeply miss the surge of otherwise opaque corporations publicly committing to open source and actually following through on it for a while. It's amazing what can happen when the entire tech press is drunk on excitement about open source, leading to board meetings all over SV in which every CEO was getting pestered to have an articulated open source strategy for how they will remain relevant in this brave new world that was supposedly just around the corner.
So that's the era that MacOSForge was born into, and at the time it probably seemed bursting with potential. This was also the era of bootable Darwin ISO images (and OpenDarwin after that). There are many reasons a lot of that stuff faded away over the years. My theory is that many of these initiatives solved problems that people didn't have. Why put money into putting stuff out there that nobody cares about, and why would anybody care about it if the company wasn't pouring money into it? It's a chicken and egg problem unfortunately, I don't think there's anyone to blame for it. Some things just aren't going to be interesting to enough people to merit throwing money at them. I'm just glad that much of macOS continues to be open source, even if most of it is in the form of a bunch of tarballs that get put up on opensource.apple.com after every release.
TL;DR as others have pointed out, MacOSForge itself is (was?) Apple. MacPorts is an independent project and will live on, but benefitted from being born in an era of unusually high industry enthusiasm for open source.
But now that we have to leave Mac OS Forge anyway, it makes sense
to convert to git and take the opportunity to do some much needed
and overdue restructuring[...]No. They are turning coding into a monopolistic monoculture. Not everybody loves pull request-based workflows, you know.
Bitbucket's ability to lock down branches is pretty nice.
> Github Issues may suck, but it's so darn convenient and so well integrated. It's far worse when a project uses another ticket tracking system
I don't feel this is true, having worked on projects that used Jira, Unfuddle, Pivotal Tracker, Assembla, Trello, etc. Of course, none of those systems have the killer feature of turning a ticket into a giant scrollfest of "+1", emojis, and animated gifs.