Hacker News Reader

The artifact approach is unambiguously better for any production deployment.

Which is precisely why many Linux distributions use them, and "app stores" were already a thing in Debian in the mid-90s. Every time I read about a programming language's "package manager" I can't help but feel that the creators of such are reinventing the wheel, poorly, and mostly to paper over issues in backwards OSes that don't have proper package management.

.. or they don't like the limitation of only having 1 version of any artifact installed at a time, artifacts being installed globally, etc. etc.

Of course they could target Nix instead, but how many people use Nix?

"Of course they could target Nix instead, but how many people use Nix?"

More than use the new system they developed instead, at the outset.

You make it sound like "papering over issues in backward OSes" is a bad thing. The fact is, those "backward OSes" have a HUGE user base, no matter how much you may judge them for being inferior. There is clearly demand for a tool that:

1. Allows developers to distribute their stuff to users on those OSes. 2. Requires the least amount of effort as possible on part of the developer. 3. Does not require the developer to custom-package their stuff multiple times, once for each OS.

Face it, developers are lazy (not meant in a derogatory way). Most solutions do not meet requirements 2 and 3, which is why developers will largely choose to avoid them until they have no choice.

And also, what matthewmacleod said in this thread is very true.

Hmm. This is an appropriate implementation for an operating system, but I don't think it's suitable for a programming language.

To take Ruby as an example - I have lots of different interpreter versions installed. 1.8.7, 1.9.3, 2.0, 2.1, jruby, rubinius. There's some scope for managing this in OS package managers, but it's already complex. Then for each of these environments, I want a different set of packages with different versions (some are compatible with older versions, and some aren't, for example). And finally, I want different versions and combinations of these packages for different apps that I might want to run - again, there's limited scope to implement this in an OS package manager.

You ideally want both. Source packages for development work, and artifacts for deployment. I've switched to .debs for deployment. Oh so simple.

Oh yeah, don't get me wrong - that's lovely. There's nothing more frustrating that trying to get a mildly complex Rails app deployed on a mutable server.

It's only a problem because the versioning on most ruby gems is broken, because Ruby package managers enable such breakage.

The big difference between dpkg and gem is that gem allows you to install more than one version of a gem at a time. With dpkg you overcome this by creating a whole new package when the major version changes. Thus I can install libcap and libcap2 simultaneously. Any breakages in minor version changes are considered significant problems.

This is the right thing to do in a production environment: minor & patch level increments are very likely bug fixes and security fixes, and should be applied to all packages that use that library.

But it's a PITA for development. And since package manager developers are mostly developers, they did what was right for them, and allowed multiple versions to be installed simultaneously.

Today, the right answer is to use something like Vagrant or Docker so that development environments are much closer to production environments.

There are two easy/sane ways to solve your problem.

1. RHEL / CentOS solve this problem with "software collections," aka "scl." Using scl gets you the advantages of ye olde retargetable RPMs but without all the hassle and suffering. Makes it relatively easy to scope your packages that way.

2. fpm doesn't do nearly as bad a job as it used to. Given a gem, it can correctly build it, assign its C dependencies, and assign a prefix so your gems for ruby-ee don't overlap with ruby19 or ruby18.

> creators of such are reinventing the wheel

It's kinda funny how they named the latest and greatest in the Python world because of this: http://wheel.readthedocs.org/en/latest/

I don't think the artifact approach is unambiguously better. Sure, "consistent deployments on two different Linux systems" is important but you can achieve that with rubygems along with the extra flexibility that distributing the packages as source provides.

Two approaches you can take:

1. bundle package will package up all of your installed gems, which you can then deploy with your project.

2. Your build server can install all of your gems, and then you can package up the resulting directory and deploy it - you could also build e.g. an rpm package.

"bundle package" still just bundles the source packages.

Installing all of the gems then packaging the resulting directory is an example of an artifact-based approach.

I prefer to do it gem-by-gem using 'fpm' in order to maintain better dependency data -- fpm will invoke rpmbuild with the right options to find shared objects and automatically attach correct dependency data for the ruby runtime, C libraries etc

Python has wheel, which is a binary package format that pip supports. It is something I've been looking into for deploying Python packages in production.

Edit: Link to PEP for Wheel: http://www.python.org/dev/peps/pep-0427/

I used to be in the Ruby / Rails / Bundler camp, and this was indeed an annoyance.

But with Node.js, the typical deployment scenario for us, and I believe others as well, is to simply tar up the entire app including dependencies.

I'm not sure if this is possible with the other scripting languages you mention.

I also don't believe this is as big an issue in development for Node.js, because most packages are pure-JS, and don't run any build steps whatsoever.

Finally, I think it's also very convenient to have my dependencies' sources available in development. I trace into them quite often.

Bundler offers similar functionality, but it seems like most people don't use it. http://bundler.io/v1.5/bundle_package.html

"bundle package" caches the source packages, the gems themselves. not the artifacts produced by installing the gems.

I think virtualization can take care of consistent deployments. And developing under a virtualized container ensures consistency from multiple platforms.

My point is: a source-based approach is not the tool to blame for consistency, it's used with lazy concern, and that's a mistake onto the crafter.

I work on two different platforms as one. Virtualization achieves that.

Yes, freezing particular OS configurations into golden images on a virt platform can make a source-based approach repeatable. Unfortunately, this just moves the problem: I now have to maintain the golden images.

Relying on virtualization for repeatable source-based deployment is like screwing in a lightbulb by rotating the entire building around the bulb.

The artifact approach is unambiguously better for any production deployment.

Which is only a problem if you're still doing application deployments directly to production machines, which is becoming less and less common. Tools like Docker, Packer and Aminator allow you to do the package install at build time.

If anyone else was wondering: http://en.wikipedia.org/wiki/Halting_problem

Specifically, much gem metadata is stored as a ruby script. To get some information about a gem, I have to execute the ruby code.

Python libraries that use setuptools are the same way.

What gems modify the Ruby code?

I assume he means: any gems that monkey-patch any of the built-in classes. I believe ActiveSupport modifies things like Fixnum (1.days.from_now, for example).

Yes. That's what I meant.

Ah. Ok. I was thinking something changing core files on disk.

I bet that's doable ...

Change string.c, recompile, if a gem can execute arbitray code.

Does it really happen at install time or (like in Python) at runtime everytime the package/module/gem is loaded/required?

ActiveSupport doesn't modify the base Ruby classes until it's been loaded. Merely installing gems will not impact anything except disk space.

puppet, facter, rake, mongrel use 'sed' to edit the code on the fly at install time, to set shebang lines etc

There are surely many others, these are just the ones I have at hand as examples.