Hacker News Reader

Comment by montroser | Hacker News Reader

Everyone crying about this on the Internet would do better to just take it as an easy lesson: pin your dependency versions for projects running in production.

This was an honest oversight, and even somewhat inevitable with so many expected supported ways to import/export between cjs mjs amd umd etc. It will happen again.

And when it happens the next time, if it ruins your life again, take issue with yourself for not pinning your dependency versions, rather that package maintainers trying to make it all happen.

View on HN

And everyone who depends on projects that pin their dependency versions gets to be victims of security exploits long after they are fixed.

Dependency management is not as simple as you seem to think.

I’m working on a thing I’m calling DriftWatch that attempts to track, objectively, how far out of date you are on dependencies, which I call dependency drift. I’ve posted about it here before [1]. I’m using it in my consulting practice to show clients the importance of keeping up to date and it’s working well.

I agree with the parent that it’s important to lock to avoid surprises (in Ruby, we commit the Gemfile.lock for this reason), but it’s equally as important to stay up to date.

1. https://nimbleindustries.io/2020/01/31/dependency-drift-a-me...

There are commercial tools like blackduck, sonartype/nexus which are used to sczn dependancies of not1 just node code, and highlite ourof date packages, known vulnerabilities, and license problems.

Well, I guess you can choose whichever poison you like.

Pinning isn't meant to be a forever type of commitment. You're just saying, "all works as expected with this particular permutation of library code underneath." And the moment your dependencies release their hot-new you can retest and repin. Otherwise you're flying blind and this type of issue will arise without fail.

The "magical security updates" theory has never worked. Breaking insufficiently-pinned dependencies are vastly more common than unnoticed fixes on patch releases. On balance, semver has been good for javascript, but to the extent it contributed to the popularization of this dumb theory it has been bad. Production apps (and by a transitive relation, one supposes, library modules) should be zealously pinned to the fewest possible dependencies, and those dependencies should be regularly monitored for updates. When those updates occur, tests can be run before updating the pins.

Right, making significant changes to the entry points of a library should be marked as a breaking change and bump the major version.

Yes -- pinning dependency versions does not have to be at odds with security.

In fact, how secure is it, really, to keep dependencies unpinned and welcome literally /any/ random upstream code into your project, unchecked? This is yet more irresponsible than letting dependencies age.

But even then, it's not as if you have to choose -- you can pin, then vet upstream updates when they come, and pin again.

Tools like Safety can help in the python world, https://pypi.org/project/safety/, and cargo-audit https://github.com/rustsec/cargo-audit in the rust world. Stick them in your build chain and get alerted to dependencies with known exploits, so you can revisit and bump your dependency versions, or decide that that project is not worth using if they can't be bothered to consider security to be as important a feature as it is.

The unspoken assumption is that you don't just pin and move on with your life. You take as much ownership over your package.json as you do with your own code, and know that you must actively review and upgrade as necessary (as opposed to just running "npm install" and trusting in the wisdom of the cloud)

There are tools out there (like npm audit) that can alert you to known vulnerabilities.

And everyone who just upgrades whenever possible gets to be victims of security exploits too.

Users of Debian Stable missed Heartbleed entirely. It simply never impacted them.

What you do is you pin dependencies, then automate regular dependency upgrade PRs. If your test suite and your CI/CD pipeline is reliable, this should be an easy addition.

We run Dependabot in our CI pipeline to flag security upgrades, and then action them. I'd much rather have that manual intervention than non-deterministic builds.

Exactly: pin dependencies to avoid surprises, and use a CI to test compatibility of new versions, so you can deploy security updates on your own schedule, best of both worlds.

Github even bought Dependabot last year, so it's now free.

> pin your dependency versions for projects running in production

Works for existing apps, but people using create-react-app and angular CLI can't even start a new project.

I don't know much about those projects, but why did this break them? Are they not pinning versions?

Because they are starting a new project from scratch and would have nothing to pin their dependencies against?

Maybe I'm misunderstanding how those projects work. From what I recall, they generate a project, including the package.json. So I'm not sure why they couldn't just generate the package.json with pinned versions?

I don't write much JS, and have only used create-react-app just a few times, so feel free to explain why this isn't possible.

package.json only lists top-level dependencies. package-lock.json tracks all dependencies, and dependencies of dependencies. is-promise is one of those dependencies of a dependency, which you don't have much control over.

How could a dependency-of-dependency change version if one of the direct dependencies doesn't change version? I guess, if the direct dependency isn't pinning that version? Another case of, everyone should be pinning dependencies.

Exactly, node's conventions are to allow a range of versions (semver compatible). True, if all dependencies were pinned, this wouldn't come up as often.

That also means that there would be a lot more updating when security issues are found.

How would a top level dependency change versions if it bumped a transitive dependency? Is that a thing in js-land?

Nah, create-react-app and others could easily pin dependencies of libraries they install in your new project to known-good versions.

Without doing that bit of diligence, this type of issue should be 100% expected.

By doing that they would avoid this issue, for sure. They would also introduce security issues by using old versions.

And this would do nothing for the fact that `npm install eslint && ./node_modules/.bin/eslint` was also failing.

It's not like pinning means you can /never/ update. You just get to do it on your own schedule.

You can even automate updating to some degree -- running your tests against the latest everything and then locking in to those versions of all goes well.

Again, this only works for project skeletons, and not for any other package that happened to have a transient dependency on `is-promise` (which is a lot more than project skeletons).

Pinning dependencies might introduce security issues.

Not pinning dependencies is a security issue.

Then you can’t upgrade anything unless create-react-app releases a new version (or you eject), which, in addition to the obvious release cadence problem, might introduce other compatibility problems.

I'm a novice in this area but if your project relies on a bunch of external node packages why wouldn't you download them all and host them locally or add them to version control?

You'd use a proxy, yes.

Adding them to your own version control is a nightmare: Your own work will drown in all the changes in your dependencies. The repository will quickly grow to gigabytes, and any operation that would usually take seconds will take minutes.

It's also just not needed. Simply specifying an exact version ("=2.5.2") will avoid this problem. The code for a version specified in this manner does not change.

Yes, putting your dependencies in version control alongside your project is no fun. Commit history is muddied, but also if your production boxes are running on a different platform or architecture than where you and your team develop, that can make a big mess too.

That said, with a big enough team and risk-averse organisation, it can be a brilliant idea to put your dependencies in /separate/ version control and have your build process interact that way.

In that scenario, even if your dependencies vanish from the Internet (as happened with left-pad), you are still sitting pretty. You can also see exactly what changed when, in hunting for causes of regressions etc.

To me an even bigger nightmare is your entire project or product depending on some external resource you don't control.

Checking them into your repo is called “vendoring” and it’s one way of solving the problem, yes. Personally, it’s my favorite approach. But it does have some challenges, as other commenters point out.

> pin your dependency versions

And then to see "npm detected 97393 problems" or whatever the message exactly is.

That’s good: it’s easy to update and it means you do it in a controlled manner rather than the next time something deploys.

You don't need to pin them forevermore -- just when you don't want everything to break unexpectedly :).

When you want to upgrade your dependencies, then go ahead and do that, on your own schedule, with time and space to fix whatever issues come up, update your tests, QA, etc.

Reply on news.ycombinator.com