Hacker News Reader

> packages [...] don't fully solve the dependancy issues (IE I might expect Ubuntu 14.04 to have a specific version of libc, but the user might have upgraded it, and I can't install my own incompatible version)

I feel that once a user circumvents the package system, it is unreasonable for that user to expect that the package system will continue to work correctly. In that case, the user has taken over some of the responsibilies of the package system, and software vendors should feel no obligation to ensure compatibility with such changes.

I don't disagree, and of course swapping out libc might be an extreme example, but my point still stands - ideally I shouldn't have to rely on any specific dependencies being in place to deploy my software.

Some amount of dependencies is a reasonable expectation; for example, requiring a certain minimum version of libc is entirely reasonable. Or, requiring a certain minimum version of the kernel.

However, if newer versions of libc or the kernel are not backwards-compatible, that's a failure in the system itself (not the packaging). (Linus, as an example, has a strict rule about not breaking userland applications due to kernel changes.)

It's impractical (and undesirable from a security and other standpoint) to statically link all dependencies into every binary simply to "simplify" packaging. Not only would that be a waste of storage space and memory, but it prevents fixes (security or otherwise) applied to the libraries used by the application from taking effect without patching everything that statically-linked it.

Even if you ignore the storage space issue with some amount of hand-waving, larger binaries means greater I/O and network bandwidth requirements, which means it takes longer to update systems or create images for deployment. That in turn leads to increased downtime or increased sustaining costs, which effectively leads to reduced availability of the system.

This can be mitigated somewhat by using difference-only-style update mechanisms, but it only partially mitigates the issues created by statically-linked binaries; it does not eliminate them.

Dynamically-linking dependencies also ensures that future performance and security updates don't require a rebuild of existing applications; you can simply update the dependency, and if done properly, every application linked to it will automatically receive the benefits.

Obviously, if a given dependency is only ever used by a single component, and the component is always rebuilt when the dependencies are, it doesn't matter if they're statically-linked. I'm referring only to the system dependencies common to many system components (such as libc).

In the end, a carefully chosen set of dependencies that provide robust backwards-compatibility provides the least downtime and the most compelling administrative experience.

I agree that functional package management is the way forward. Having a system that contains the precise details of how to go from source code to binary, in a completely automated fashion, for the entire set of packages all the way down to the bootstrap binaries is a critical foundation to build upon.

Seems to summarize down to a love/hate relationship with shared libraries. Most of the flaws are best explained as the devs, admins, users, security team, and distro maintainers sometimes having different positions on the spectrum of love/hate for shared libraries.

Also don't forget architecture... you want that binary in 32 bit i386, 64 bit amd64, or arm?

In a nutshell yes. The binary argument is true as well, however many developers don't want to make their sources avaialbe which means other than going source -> some obfuscated IR -> native, thre is no way to allow the end user to run your app on any arbitrary platform.

There is. It's called fat binaries. Ryan C. Gordon was working on an ELF extension for it, but ended up quitting due to getting shit on by kernel hackers and common users alike.

I feel your pain. I'm trying to figure out how to get some code with some unique dependencies shipping to a wide range of users (both in skills and platform), and I don't even know of a good (multiplatform) solution that doesn't look like "bake it all into a static zip and unzip into /opt" or "run a docker container".

I need to think about that a bit more instead of solving the actual product problem at hand. :-/

There are solutions to this problem (e. g. ZeroInstall [1]), but the distributions don't support them, since they want to keep you vendor locked-in. Guess why Ubuntu is developing Snappy and not using xdg-app [2]? Also purely functional package managers exist [3], distributions just ignore them. It's hard to use something not based on deb or rpm these days.

[1] http://0install.net [2] https://wiki.gnome.org/Projects/SandboxedApps [3] https://nixos.org/nix/

I find it hilarious how you make the implication that Snappy is some deliberate NIH of xdg-app, when xdg-app itself is GNOME/Red Hat's NIH of Nix.

If anything, we should be lauding Canonical for keeping to themselves in their corner instead of seeking greater validity across the ecosystem with half-baked ideas.

I think Docker is meant for dev ops, not for casual users.

It's supposed to be for devops, but in practice developers use docker as a way to solve dependency management on the server more so than to seperate instances of apps from each other. Especially with "microservices" it makes more sense to have docker available on the server and allow each team to upload a image with all the deps already baked in.

Yup. The tradeoff is that in operations we then need to work around all of the inherit disadvantages of running docker containers (network slowness, disk slowdown, logging, etc) to keep their systems performant.

You will be fine if the user installs a newer libc. Binaries and .debs linked against libc from 10 years ago work and install fine.

You mean sort of like npm?

npm is an example of a bad solution to dep management. So I want you to run my 6kb node.js script you need to:

1) Build obtain node for your platform (and now you've hit the exact problem I'm talking about) 2) install npm (which I can't really define as a dep, you're just expected to have it) 3) install all the deps I've defined and hope they all work on the version / configuration of node you have (execjs only works on node 0.10 but you have v4? Too bad! You didn't apply the increased memory patch before building node? Too bad!)

The ideal here to reiterate is given nothing but your installed OS, I should be able to give you a file that will go from nothing to a running app, and from the running app to a clean system if you decide to remove it, without you having to have any prerequisites (no node, no chef, no docker, etc)

I don't think you can really complain about having to have node.js installed in order to run a node.js script.

If you want self-contained binaries, learn Go. If you want lightweight scripts that run effortlessly on all unix systems, learn bash scripting. It sort of sounds to me like you're complaining about not being able to shoehorn your chosen language into all possible scenarios.

I think that's a very reasonable thing to complain about in the context of dependency management. The runtime (node.js) and it's compile time + run time options are actually a dependancy of your application.

As an end user, I shouldn't be expected to know how to configure x dependancy for my system to run y app and as a software developer I shouldn't have to build / package apps to match the client's system configuration.

The sticking point is (back to my point) that the vast majority of distributions don't give the app a way to manage the deps and get specific deps in required, the best you can do is list x as a dep in your package and hope that it won't break your app or the system in some way.

Agreed. Have two dependencies which have the same underlying dependency? Install and compile that dependency twice (with two different versions, by the way)[1]. Want to use grunt? Need to set that up as a dependency for your project, but only for development. A problem while installing? Have fun with the debug output detailing the fetching a building of every dependency, and every dependency's dependency, and every dependency's dependency's dependency... etc.

[1] I can't believe how many distinct copies and versions of the 'shell' module are downloaded and compiled when pulling in a seemingly simple set of modules.

I wouldn't use npm as an example of a stellar package manager.

Not using it as an example of a 'stellar' package manager. Just one where the dependencies can be defined with version. I was under the impression that apt-get will just use the most recent version.

apt-get/dpkg do have the ability to specify package versions, including specifying versions equal to, greater than, or less than. It also has the ability to specify build time and run time dependencies separately.

You can specify a version. The real trouble is that you generally cannot have multiple versions of the same package installed on the same system. Thus, if multiple packages specify different versions as dependencies, you're hooped.

So that is something npm does, AFAIK. Isn't it even possible to have multiple different versions of a sub package in the dependency tree of one project?

The downvotes are undeserved. The solution is npm on the OS level - you specify the version of the dependency you require, and each version is installed across the OS only once.