Bundler to be integrated into Rubygems | Hacker News Reader

Bundler to be integrated into Rubygems

github.com

113 pointsemyl24 comments

From the bottom of the comments on that issue:

sovereign> @drbrain It's a different approach, where a dependency resolver in runtime is no longer needed. I've been using that approach for the past six years, and during all that time I have been the happiest rubyist in town while my friends struggle with Bundler. The idea is to trade space for complexity. Once the gems are installed in isolation, there's no need for dependency resolution in runtime. </sovereign

Isn't the point of the Gemfile.lock that your dependencies are not resolved at runtime? If I've misunderstood I'd really appreciate an explanation of the difference compared to say NPM. I've also never seen any practical difference between NPM and Bundler and have often wondered why people like the NPM implementation so much.

With Bundler, you have a single copy of each (version of each) library installed, and Bundler decides at runtime which to make visible (which is trivial with Gemfile.lock, since all of the interesting logic is done when creating that). This decision is a global one, so if you depend on two libraries that themselves depend on libfoo, both of those libraries have to use the same libfoo, and there's no difference between having a direct dependency on libfoo and depending on a library that depends on libfoo.

With npm, each library has its own set of dependencies (and those libraries have their own sets of dependencies, and so on) that are entirely separate. Thus, if you depend on two libraries that use libfoo, those two libraries have their own copies of libfoo - and those two copies can be different versions, if needed (and if libfoo depends on libbar, you'll have two copies of that too). This makes the presence of dependencies much less leaky and eliminates most problems with incompatible gem versions at the cost of wasted disk space, (usually trivially) increased memory usage and some potential confusion. It also happens to make npm pretty mind-numbingly simple compared to bundler's dependency resolution logic.

Rubygems doesn't have the luxury of providing that kind of system, though, because of how 'require' works in Ruby. AMD/RequireJS allows requires to be scoped, while Ruby requires dump directly into the global namespace.

Right and the guy of the last comment of the issue is proposing also an solution for it:

https://github.com/soveran/cargo

No matter what it's too leaky an abstraction. Everything in the stdlib uses require. Every fundamental piece of ruby code not in the stdlib uses require. The process of getting from point A to point C involves crossing a point B so frustrating, limiting, and ugly no one is really willing to cross it.

Ruby's global constant namespace was probably a mistake in hindsight, but at this point it'd be a language fork to change it meaningfully.

I was wondering how you could achieve the same level of isolation in ruby, this looks like a very interesting project. Ruby projects move fast and even though they're pretty good about semantic versioning this isn't enough to save you when your Gemfile starts filling up. Cargo looks like a huge improvement over require.

It sounds like npm quite easily can become a dependency hell where you have no idea whatsoever what code is actually being run.

On one hand that's certainly true. On the other: Do you care how $LIB does some work for you? Does it matter to you whether $LIB uses $OTHERLIB in version 0.1 or 0.2? Does it matter to you whether $LIB bundles $OTHERLIB as a package or whether it just includes the source code of $OTHERLIB?

Yes. I see the security implications of running an outdated $OTHERLIB, but that's $LIB's responsibility, no matter whether they include a copy of $OTHERLIB as its own package in a subdirectory or whether they just copy $OTHERLIB into their own code base.

Once you bundle dependencies, you're responsible for security holes in them.

The other issue of course is RAM usage once you start running multiple identical copies of $OTHERLIB, but that's only if they use identical versions. If it's different versions, you will have to run multiple copies at once or you risk breaking your dependencies. Or you don't run at all (Bundler's "solution")

I'm not sure there's any universal answer to the questions you pose in your first paragraph. The problem is that sometimes you do care (sometimes a library is actually an interface into some kind of stateful machine) and sometimes you don't (when it's just objects that hold their own state). To some extent javascript gets a bit of a free ride on this question because most libraries are designed around there being only one global state machine (the event loop), but ruby doesn't impose an execution model on you.

Yeah, I think npm's model is the correct one for node, but it's not universally better than all others by any means. It's not possible at all in many languages, would result in absurd compilation times and disk space usage in compiled languages, and makes security audits much more difficult. It's certainly not very compatible with an environment where every dependency requires external approval.

> Do you care how $LIB does some work for you? Does it matter to you whether $LIB uses $OTHERLIB in version 0.1 or 0.2?

Only when debugging. But that's a pretty big 'only when'

Can't you just "vendor everything" and now each project will install all of the gems locally to vendor and not be concerned about global choices?

http://ryan.mcgeary.org/2011/02/09/vendor-everything-still-a...

Sure, but if you use Git this can get pretty hard to deal with especially on large repos with lots of history. A `git clone` downloads an entire repo, and all of its history...so if you have a bunch of gems or executables in there that boosts the contents of your repo tremendously.

This is why there is `bundle install --deployment`, which "freezes" the Gemfile and ensures it has not changed from what has been committed to the repo. If it has, `bundle install` will not succeed.

I do like npm's simplicity. You used to have to be really careful with the versions, because if you didn't specify it would just download the latest version, and sometimes some of your dependencies would use the wrong versions of libraries and APIs would change and things go haywire. I think they've fixed a lot of that now, though...it was certainly easier a few months ago than it was a few years ago.

Despite being a quote from the maintainer, this is a needlessly link-baity title.

Perhaps a better one would be "Bundler to be integrated into Rubygems"

The title has been changed. What was it before?

"Bundler is scheduled to die in under two years time"

What was the logic behind rubygems adding support for Gemfiles? I can see why Gemfile.lock support is a good thing, but adding a new, incompatible runtime dependency resolution mechanism to that provided by Bundler seems like an odd choice.

Gemfiles _are_ bundler's dependency resolution mechanism; it's not a new incompatible mechanism added to that provided by bundler -- it's bundler's mechanism being merged into rubygems itself.

Which is great news, in my opinion. With the same team, release schedule, and set of tests, there will no longer be issues of which version of bundler works with which version of rubygems (which isn't an issue except when it has been, and then it sucks, especially for newbies). No extra thing to install (including in your provisioning/deployment scripts). It's just there, it just works. The number of seperate interacting parts you had to install to 'bootstrap' a proper ruby installation has been growing -- which really does add increased burden to newbies understanding what's going on -- nice to shrink it again.

> Gemfiles _are_ bundler's dependency resolution mechanism

No, Gemfiles are the dependency specification protocol. The resolution mechanism is the algorithm which turns that into a set of gems to install or, if you're feeling particularly crazy, require.

Right now I can have a Gemfile which bundler accepts, and rubygems barfs on. That's the incompatibility.

I see what you mean, yeah. If I understand it right, this is a temporary state of affairs, as the rubygems/bundler integration is only partway done.

The end goal is rubygems handling of Gemfiles and Gemfile.lock will become equivalent with bundlers, and then bundler will disappear.

I am curious why they decided to even release the partial implementation on the way. Either way, I'm grateful enough that a rubygems/bundler merge is happening, and we'll never need to deal with incompatibilities between certain versions of rubygems and bundler again once it does, that I'm willing to not be too picky about how they get there.

HOO RAY. Seriously. This is great news. Thanks drbrain!

I've probably been living under a rock, so Bundler dying in two years comes as a real surprise. Does anyone have a link or two with more info?

And here comes the bloat. Soon Rails will be replacing WEBrick in Ruby 3.0.

Reply on news.ycombinator.com