Comment by freeqaz | Hacker News Reader

Comment by freeqaz | Hacker News Reader

I've used NPM shrinkwrap before back in versions 1-3 of NPM. It's a little confusing that the author of this post calls it "new", so I investigated.

Since NPM version ~2 (current is 8), you're allowed to publish a npm-shrinkwrap.json file[0] in your package. That's in contrast to a package-lock.json, which may not be included in a published package.

Back when I used shrinkwrap, it was pre-lockfiles and it was trying to accomplish something similar to what lockfiles accomplish today. (Lockfiles were added in v5 [1])

I dug up this old StackOverflow post to verify that the behavior of shrinkwrap hasn't changed[2] since many years ago.

That doesn't leave me very hopeful. If package maintainers aren't doing this already, and they've been able to for 6+ years, then it's unlikely they will in the near future.

(I'm currently investigating how to deal with bad deps with some Open Source tooling I'm building. Feel free to ping me if you have any thoughts to share)

0: https://docs.npmjs.com/cli/v8/configuring-npm/npm-shrinkwrap...

1: https://nodejs.dev/learn/the-package-lock-json-file

2: https://stackoverflow.com/questions/44258235/what-is-the-dif...

It's because having an actual shrinkwrap file in a library introduces a huge number of conflicts. If library A has 20 transitive dependencies, and library B has the same 20 transitive dependencies, deduplication based on ranges can leave you with only 20 transitive dependencies. If both libraries have shrinkwrap files in them, you can end up with 40 transitive dependencies, even if those are nominally compatible with each other. When you consider that even with the current behavior projects end up with thousands of transitives, you're looking at adding literally thousands or 10s of thousands of duplicate dependencies to a project. That's not even considering that some duplicate dependencies will just break things if they show up multiple times in a project (older versions of react, for example).

Ah, that totally makes sense. I know that's what "peerDependencies" is used for by many projects. For example, `react-dom` has a peerDependency on `react`. This can either be pinned to an exact version (react-dom@1.1.1 must be used with react@1.1.1) or it can require a range (react-dom@1.1.1 may be used with react@^1.0.0 (any version 1.0.0 and up)).

That seems tricky with the shrinkwrap, as you describe above, because it might create additional versions everywhere instead of trying to collapse them. (If you have 3 versions of `react` in peerDependencies, but each package had a npm-shrinkwrap.json, then you'd end up with 3 different versions of `react` installed.)

Would the burden then fall on your package manager to resolve this, ie npm or yarn? If it sees 3 different shrinkwraps (1.1.1, 1.1.2, 1.1.3) but they all have compatible peerDependencies (>=1.1.1, >=1.1.2, >=1.1.3) it could then pick the version that is compatible with all of the peerDependencies but that also exists in a shrinkwrap (so 1.1.3).

That would protect you from this "colors" scenario of somebody publishing `1.1.4` with malicious changes because, unless somebody were to shrinkwrap it, then it wouldn't get picked up by default. (In contrast to the current semver, which likely would pick up 1.1.4 upon a fresh `npm install`).

The inverse side of this is highlighted by another comment on this post: That semver makes absorbing security updates an easier process in the event of a log4j-style vulnerability. There is always a tradeoff, for sure!

It is worth noting that this is not an impossible problem. I'm not a js expert, and it seems that js loves imports, but Nix solves exactly this problem on Linux. You have, for each end binary, a set of dependencies. If they are exactly the same, they can be shared among programs, otherwise you would have different versions of the dependency, all the way down, per program.

It can be bloaty, and you have to manually look at and test packages if you want to get everyone on the same set of dependencies, but you end up with reproducible environments.

This is probably the only sane way to proceed when our software has dependency chains more than a few levels deep. Establish mechanisms to try to prevent bloat, but otherwise make it possible upgrade independently and make it starkly apparent if the is duplication.

That is exactly how npm works. Except instead of one at least somewhat-aligned group of maintainers of a Linux distro trying to keep things under control, you have a much larger set of packages individually maintained (or abandoned) by totally independent people.

So practically nothing will ever exactly match, which leaves you right back at many slightly-different copies of hundreds or thousands of dependencies.

Reply on news.ycombinator.com