Hacker News Reader

Comment by inglor | Hacker News Reader

When you `npm install` a package by default when other users `npm install` it it will install the most recent patch version - even if it's different from the one you installed. So if you install dependencies through `npm install --save` which is the default and advertized way - you can get completely different code between production and staging.

As a library maintainer, patches breaking the library is something that happens (not often, but still) - testing can eliminate a lot but not all bugs.

View on HN

I'm not trying to dispute that a problem exists, only that semver is a red herring here. It seems like the problem that you describe doesn't have to do with semver, rather that it has to do with npm lacking something like lockfiles.

> npm lacking something like lockfiles

`npm shrinkwrap --dev` will give you a shrinkwrap file, with dev-deps also locked. Still, it took me 3 months of heavy Node usage before I found that out. Over in PHP land, while Composer has its own set of faults, it's at least generating a lockfile automatically.

It is semver compounded with the "^x.y.z" version requirements for dependencies that NPM uses as a default when a package author `npm install --save` something.

When someone else installs that package it will bump y or z if `x > 0`, and z if `x == 0 && y > 0` for all dependencies.

You can manually freeze deps to 'x.y.z'.

The main problem is the "^" default.

The main problem is the "^" default.

This is true.

A secondary problem is the culture of having so many dependencies, which has been much discussed this past week. Even if you lock everything to a static version for consistency, how many people really know which packages from which sources they are relying on to build their system today? Presumably you trust your direct dependencies, and they in turn trust theirs, and so on, but all it takes is one package five levels deep in the tree where the developer was in a rush and npm install'd something vulnerable or malicious to compromise the entire tree.

Even if you manually freeze deps to 'x.y.z', the deps of your deps might still be specified with '^'. Ideally you'd shrinkwrap, and commit node_modules to your repo. (In case packages get deleted)

What's the point of shrinkwrap if you're commiting node_modules? And also why not use a package manager like RPM, Debian, etc so you don't bloat your Git history? (Genuine questions, I know many advocate what you're saying).

Vendoring by storing node_modules works great. It's not necessary to shrinkwrap in that case -- all the same data is derived from the node_modules directory tree. For production, I often use a separate git repo for storing node_modules, referenced as a submodule of the code repo. This has two advantages:

* You retain history for your full dependency tree separately from your codebase.

* Every commit of your code repo exactly specifies the contents of its dependency tree, since the submodule references your deps repo by commit hash.

Re-re-posting a comment I've made in a few threads now, because I feel this needs to get more awareness:

I'm still learning and prototyping my first JS/NPM/React project, but https://github.com/JamieMason/shrinkpack seems to me like it solves _most_ of the issues involved here. It pulls down all the tarballs, and updates the npm-shrinkwrap.json to point to those instead. That way you check in a much smaller "node_shrinkwrap" folder of a few hundred tarballs and 15-20MB, rather than a node_modules folder of 30K files and 150MB and a bunch of platform-specific build outputs.

Still doesn't solve the issue of installing new/updated dependencies that might actually require pulling in something that vanished, but at least once you've done an install and run "shrinkpack", you've got everything you need to rebuild right there.

what is the recommended way to globally change that default?

    npm config set save-exact true

> you can get completely different code between production and staging.

That's not NPM's fault, its the fault of the way you deploy code. Even if you locked down a version, git is mutable so someone could change their code. That's why I "rsync" the code to production, so I know its the same as development.

It's also why one wag suggested using Twitter (which "doesn't have an edit button") as a repository:

* https://gist.github.com/rauchg/5b032c2c2166e4e36713

I mean, don't use `npm install --save` then. I'm not really sure why people started using it in the first place, it's such a lazy thing to do. Instead, add it to your package.json yourself with the exact specific version you want (none of the ^a.b.c funny business).

When I bootstrap code I npm install --save, I have 10 packages I usually need right off the bat and I don't want to start an investigation every time I do this.

or rather, npm install --save-exact :)

then upgrade with npm shrinkwrap

echo "save-prefix=''" > ~/.npmrc

Instead, add it to your package.json yourself with the exact specific version you want

Unfortunately, the same problem then arises for your dependencies. If any of them don't specify exact versions, you are still vulnerable to getting uncontrolled changes.

This is why things like npm shrinkwrap exist, but it's still crazy that NPM's default behaviour is the uncontrolled case.

Yes, libraries should specify exact versions as well, it's insane that they don't.

Reply on news.ycombinator.com