Ruby's Bundler: As Simple as What You Did Before
yehudakatz.com
yehudakatz.com
Once I finally gave up trying to use Bundler to manage my forked Rails 2.x and just left vendor/rails as a submodule, all the rest of the bundled gems worked beautifully.
I had my issues with Bundler since trying out earliesh version of it last year, and jumping on the early Rails 3 betas during the January bugmash and intervening months. There were warts for sure, but the biggest one was the pace of development and the outdatedness of the blog posts.
By now, however, it feels like all this complaining has put Bundler through a kind of crucible that RubyGems never had. From my perspective Bundler 1.0.0 is a pure joy to work with. The problems it solves were very real, and from where I sit it solves them in a very thorough way. At the moment I'm contemplating how to bring my 50k line 3-year-old Rails app up to Rails 3, and converting to Bundler is the first step with immediate and significant benefits. I lump Bundler in with efforts like RVM and Rubinius, as ambitious projects that are pushing the state of Ruby forward and keeping it relevant in a world of Clojures, Scalas and node.jses
I find it interesting that most people who claim that RubyGems is flawed don't state what the flaws are, or only give vague explanations and assume everybody can read their minds.
About the only thing I can imagine is that the ability to simultaneously activate multiple gems should be a feature in RubyGems itself rather than an external feature.
Bundler papers over this by putting each app in its own enclosure such that it's at least possible to package a gem-using app atomically without requiring too much effort on the author's part.
First a few axioms we must agree on:
1. Libraries eventually have to break compatibility, otherwise progress is impossible.
2. There must be some way to deal with incompatibilities, either socially (release and compatibility guarantee policies) or technically (package managers).
Your point of view: social ways to deal with incompatibility are superior to to technical ways to deal with incompatibility because the latter encourages developers to be lazy about compatibility guarantees.
I have a few objections about that point of view.
1. It assumes a perfect world.
Although it doesn't happen often, an app can break due to reliance on library bugs or unformalized/undocumented behavior/internals. Sometimes a newer version of a library has a regression that isn't found until it's released.
Yes we all know things like these ideally shouldn't happen, but they do. Is it a good idea to ignore the reality that things like this happen from time to time?
According to the logic that supports your point of view, we should abolish protected memory as well. Protected memory allows programmers to write apps that don't bring down the entire OS when they crash... it makes programmers lazy!!
2. It assumes that DPKG/RPM don't make life hellish for sysadmin and end users.
For the most part, DPKG/RPM works fine. For Ruby stuff, not so much. Suppose there's an app called frobnicator, and it works on foobar 1.0.x but not on foobar 1.2.x. Thanks to the fact that DPKG/RPM only supports installation of single versions, your distro can only package foobar 1.2.x but not 1.0.x. Now the user cannot use frobnicator at all because you insist on DPKG/RPM. How exactly is this better for the end user?
3. It inherently excludes some valid compatibility cases, unless the developer is forced to hack around the package manager
Suppose 'foo' depends on 'bar'. bar consists of features A, B and C. Furthermore bar has made the guarantee that it will never break the compatibility of feature 'A', ever. It also has a compatibility policy that states it will increment its major version number every time it breaks B and C.
foo only uses feature A in bar, so we know that foo is compatible with every version bar that will ever be released.
foo also depends on 'libcat'. libcat depends on bar, but uses all its features. The current release of bar is 1.0.0, so we know libcat depends on 1.0.0 <= bar < 2.0.0.
Suppose that 'frobnicator' depends on bar. It uses feature B and it uses the interface provided by bar 2.x.
This seems like a completely reasonable case. Developers aren't being lazy about compatibility guarantees here. The only way to solve this problem is by versioning bar on the filesystem level. That is, while bar 1.x requires developer to type
require 'bar/feature_a'
require 'bar/feature_b'
require 'bar-2/feature_a'
require 'bar-2/feature_b'
> You have to rename the entire library directory. In my opinion this is an ugly hack around the fact that your package manager doesn't support multiple versions. I think this is solved much more elegantly if the package manager allows you to switch the version on-the-fly... in other words, what RubyGems does.
It may be an ugly hack, but it's fairly traditional at this point.
Only if you activate gems linearly. Bundler solves exactly this: it looks at all dependencies then figures out a combination that works. Bundler will only activate bar 1.x because it sees that foo is compatible with all versions but libcat is only compatible with 1.x.
> It may be an ugly hack, but it's fairly traditional at this point.
I argue against the notion that being traditional here equals being good. There's no good reason why the developer should do this manually instead of having some system do it for him.
RubyGems actually implicitly does this. By installing each gem to its own directory it separates the versions on the filesystem level.
You're absolutely right, I misread your example.
> I argue against the notion that being traditional here equals being good. There's no good reason why the developer should do this manually instead of having some system do it for him.
It's a trade-off. Either the developer has to do it manually (once, which will then work with every sane distro), or the sysadmin has extra work to do (many times, once for each language-specific package manager).
In an ideal world, the language communities would gang up on Debian and Red Hat and improve dpkg and rpm to make this issue go away. I don't see this happening.
Not quite. Social ways to deal with incompatibility are superior to the specific technical way that rubygems encourages.
> Although it doesn't happen often, an app can break due to reliance on library bugs or unformalized/undocumented behavior/internals. Sometimes a newer version of a library has a regression that isn't found until it's released.
> Yes we all know things like these ideally shouldn't happen, but they do. Is it a good idea to ignore the reality that things like this happen from time to time?
I'm not ignoring that reality at all. I'm specifically encouraging point releases with backported bug-fixes. These happen all the time in dpkg-land.
> According to the logic that supports your point of view, we should abolish protected memory as well.
Extrapolation to extremes will destroy any rational argument, so I'll ignore this.
> 2. It assumes that DPKG/RPM don't make life hellish for sysadmin and end users. > For the most part, DPKG/RPM works fine. For Ruby stuff, not so much.
That would be the point. I'd also point out that it's not just a question of dpkg vs rubygems here. From a sysadmin's point of view, it's dpkg vs (rubygems + CPAN + PEAR + cabal + virtualenv/pip + maven + ...).
> Suppose there's an app called frobnicator, and it works on foobar 1.0.x but not on foobar 1.2.x. Thanks to the fact that DPKG/RPM only supports installation of single versions, your distro can only package foobar 1.2.x but not 1.0.x. Now the user cannot use frobnicator at all because you insist on DPKG/RPM.
That specific case isn't a problem, because you'd just package foobar 1.0.x and ignore 1.2.x completely. The problem comes when something else comes along that depends on 1.2.x but won't work on 1.0.x at all. That would be a case of either foobar's author not following version numbering convention, or a bug which can/should be patched.
> How exactly is this better for the end user?
If it's not, foobar's author is responsible. However, for the distro to be able to turn around and say "We value stability over all else, and we cannot provide this combination of packages while supporting a stable system, so we're not going to" is actually fairly awesome.
Suppose we substitute "1.2.x" with "2.0.x". foobar now is following version numbering convention but the problem didn't change. Now what would you do?
> We value stability over all else, and we cannot provide this combination of packages while supporting a stable system, so we're not going to
This is clearly false. Package both versions and have apps load the version they're compatible with. Now the user can have all apps without compromising stability. RubyGems supports exactly this, DPKG/RPM doesn't support this without filesystem-level renames. Why do you think the latter is superior?