Show HN: Tips to stay safe from NPM supply chain attacks | Hacker News Reader

Show HN: Tips to stay safe from NPM supply chain attacks

github.com

96 pointsbodash55 comments

Hi everyone, given the recent increase of attacks on the NPM supply chain, I've put together a list of tips and tricks to help developers stay secure on this specific topic: https://github.com/bodadotsh/npm-security-best-practices

I'd love for you to check it out, and contribute your own insights and best practices to make this a comprehensive resource for the community.

Cheers!

Open articleView on HN

For most projects, overriding every single transitive dependencies to be pinned is impractical.

Instead, for those using npm, I'd highly suggest using `npm ci` both locally and of course on CI/CD. This will ensure the (transitive) dependencies pinned in the lockfile are used.

TIL on the `npm install --before="$(date -v -1d)"` trick; thanks for that! Using that to update (transitive) dependencies should be really helpful.

For those using GitHub Actions, I'd also recommend taking advantage of the new dependabot cooldown feature to reduce the likelihood of an incident. Also make sure to pin all GitHub Action dependencies to a sha and enforce that at the GitHub repo/account level.

For GitHub Actions, i found http://safedep.io/ to be helpful, not only it guard against known attacks, but also it has its own malware detection engine.

Also, don't use npx.

With the colors incident back in 2022, random stuff started to break not when people updated their dependencies, but immediately, because npx would resolve dependencies when the command is executed.

This means it's not really possible to reason about what code is going to execute, and forensics is going to have a really hard time figuring out what a computer has executed.

If your software uses npx in any capacity, you've auto-failed the SBOM compliance checkbox.

> npx would resolve dependencies when the command is executed

I hate that this is becoming a thing. I was pretty miffed some time back when I realized go build just went ahead and installed a whole new version of golang on my machine. These are devtools ffs, why so much mollycoddling! And what happened to half a century of good conventions where the default is always to prompt?

And what happened to half a century of good conventions where the default is always to prompt?

The good old "be careful what you wish for" :)

What makes node supply chain attacks so dangerous is the CI/CD pattern whereby all dependencies are downloaded from the internet every time a build is created. NPM attacks move fast.

I previously worked in an environment where our ci servers weren't internet-connected. One of the things we did get get node builds to work was we had 'node_modules' for our projects in a separate repository that got joined with our source code in CI to complete a build. When a developer added a dependency, they had to update this repo from their local version. It was annoying to have to synchronize two repositories, but this ended up being a forcing function for the development team to adopt several of the suggestions listed here. When you see a PR with a massive diff for a small dependency change, eyebrows raise and the team starts conversations about how to improve things.

Lockfiles are a more standard and probably better way to do this. (People do need to pay more attention to lockfile diffs.)

And if you want to get the same "we serve the code directly" benefit as well, you can set up an npm proxy and require its use. That way you're getting a very specific version, and downloading that version from a location you control.

(And then for the ultimate level of "slow your project to a crawl, but hey at least it's really secure", you can only allow versions that pass an internal security review to be added to the proxy and disable automatic fetching of un-cached versions. Ain't no sneaky code getting in unawares there!)

A two week delay on including new versions would probably work more or less as well with a bunch less effort, but a local proxy looks like it’s going to be a lot more common very soon I’m guessing.

These tips are great, but they don’t address some of the core ways that these supply chain attacks may happen: global modules and npm modules installed with editor extensions.

So `yarn global add nx` will still install the latest version by default, unless you specifically have a `~/.yarnrc` disallowing lifecycle scripts they will still be executed. Using a package manager that doesn’t allow lifecycle scripts by default is the solution here I guess.

I don’t know what the solution is for stuff like [this](https://github.com/nrwl/nx-console/blob/d2fa56509679fc942bbc...) where the editor plugin automatically uses the latest version, or where in general you have little control over what version is used. Any eslint, typescript, nx, prettier, etc plugin will presumably depend on their corresponding package from npm, and if any of those gets compromised then just installing an editor extension could be enough to get you in trouble.

Somehow it is missing the "read the code of your dependency" step … :)

Even occasionally having a glance (e.g. when reading the docs) might be super helpful in discovering strange things going on.

Or maybe just read the commits between now and a reasonable date far enough in the past so that if there is some hostile code injected before that point in time, then at least you will share the walk of shame with a lot of people and you can play the sound of "who could have guessed?"

There's no point in reading the code in the Git repository or its commit history because that's not the code that you're actually executing. You have to read what's in your node_modules, everything else is irrelevant.

This is often overlooked, to the point I created a website focusing on "the code we actually put into our computers":

https://whatsrc.org/

It doesn't index all of npm, only if the package was reference by a Linux distribution somehow (e.g. package-lock.json in a tar file used in an Arch Linux PKGBUILD).

Plug: I've been building a tool to detect software supply-chain cyberattacks: https://github.com/ossillate-inc/packj

Packj uses static+dynamic code/behavioral analysis to scan for indicators of compromise (e.g., spawning of shell, use of SSH keys, network communication, use of decode+eval, etc). It also checks for several metadata attributes to detect impersonating packages (typo squatting).

Every decent malware author just adds tools like these to their test suites, and only release new malware that evades all detection.

That game of cat and mouse never ends.

The only solution is just actually reviewing the code we ship to our customers. Yes, even the code we copied off the internet with a magic "npm install" command.

I appreciate all this guidance. I hope Node developers read it all and consider adopting most/all of it.

Node and the NPM ecosystem has been so productive for me and package.json scripts got me more into shell scripting than I ever thought I would.

All that said, there are some major insecurity deal breakers that frighten me when using Node in public-facing services.

Sneaking in compiled native binary blobs as part of NPM install, transitive dependencies with unpinned versions, the vast wasteland of unmaintained packages in NPM ... Node just needs to be superceded, I feel like.

It would be really great for a newer tech company with deep pockets like Tesla to pull a Sun Microsystems and release a new secure-by-design OS and language stack - maybe in support of a modernized hardware platform offering.

My preference would be for a deeper standard library like the jdk. I would like some sort of digital provenance that runs from the dev environment, through the os and package manager, through to all device types all the way through to the one and only global app store (or an enterprise-hosted proxy.) The whole kit and kaboodle signed and delivered at all levels.

I would like more energy efficient network hosting and service delivery patterns codified.

I would like public developer guilds with certs not as a prereq for employment, but rather to encourage developers to have something to show for their training other than a nebulous college diploma. Senior guildsmen can present their work products for review as an ongoing proof of their craftsmanship.

Huge props for point 13 which is basically the crux of the matter, and often overlooked.

Personally I would like to see more awareness around the dangers of blindly trusting compiled javascript (and non-human-readable code generally).

Someone recommended this to me on another thread and tried it yesterday and it seems very good:

https://github.com/safedep/vet

Isn't this just checking packages against known cves, which wouldn't help for undiscovered or unannounced vulnerabilities. Let me know if I've misunderstood, I'm basing off the documentation site.

Also I find the irony goes hard in their recommendation of installing another attack surface (brew) on Linux and missing the point.

I think, they have an malware detection engine of their own, so not only they help protect from known vulnerabilityes / malwares but also have thier own database

their blog: https://safedep.io/dynamic-analysis-oss-package-at-scale/

That first recommendation of pinning exact versions of each and every dependency is borderline insane. That's exactly what lockfiles are for. Which are used by default.

The lockfile is updated _after_ any new malicious version is downloaded and installed. If we pinned the exact version, `npm install` will _not_ download and execute any new published versions.

That's why we use `npm ci` or `--frozen-lockfile` to install the exactly versions as lockfiles. But, by default, the `^` operator and just `install` command will check registry for any new releases and download them.

The primary arguments against pinning versions are missing security updates and increased maintenance overhead. But given the patterns we've seen, the attackers really _hope_ we automatically install new releases

npm install does install the exact versions from the lockfile. Even though this misconception gets repeated in every single thread about npm here on hn. npm install will not randomly update your direct dependencies, let alone transitive dependencies.

To be honest, NPM is a complete shitshow when it comes to this, and I wish each and every single person who had a hand in developing it have their keyboards taken away, and never be allowed to touch any developer tooling ever again.

See this Stackoverflow thread:

https://stackoverflow.com/questions/45022048/why-does-npm-in...

The top answer has 3 updates to it, and links to 2 github issues, with conflicting information.

One says:

>If you run npm i against that package.json and package-lock.json, the latter will never be updated, even if the package.json would be happy with newer versions.

The other says:

>The module tree described by the package lock is reproduced. This means reproducing the structure described in the file, using the specific files referenced in "resolved" if available, falling back to normal package resolution using "version" if one isn't.

>This holds no longer true since npm 5.1.0, because now the generated module tree is a combined result of both package.json and package-lock.json. (Example: package.json specifies some package with version ^1.1.0; package-lock.json had locked it with version 1.1.4; but actually, the package is already available with version 1.1.9. In this case npm i resolves the package to 1.1.9 and overwrites the lockfile accordingly, hence ignoring the information in the lock file.)

So good luck figuring out what is true, but it seems to also depend on your version of NPM. Also, don't get me started on the presence of an entirely separate command "npm ci", which is supposed to be the one that is reproducible.

Absolute clowns.

Figuring out what is true for npm v5 is quite the waste of time, given that we are currently at v11. And that's what this ancient stackoverflow thread is about. npm certainly has a troubled past, otherwise we wouldn't have yarn and pnpm and whatnot. But _today_, npm install works very reasonably with lockfiles.

You are right, but starting with an insane default (there is a lock file, but installing overrides and updates it), then introducing another command that does the expected behavior (npm ci), and then finally making that the default is at least confusing.

One thing I haven't seen talked about at all is the local development setup. I was thinking of putting node/js projects fully into docker containers (and mounting the project directory as a volume for hot reloading). While this doesn't fix the CI attack vector, it should mitigate risk for personal/work machines.

I'd be interested in hearing the setup other people have for their dev envs, also are you using separate browsers for Dev/Internet?

I actually posted about my own Docker setup this morning: https://ryansouthgate.com/secure-node-in-docker/

I use this on all my front end projects and it protects my "host" machine from malicious packages, it's not a silver bullet though; other practices, e.g. good secret management, will help harden your dev environment from these attacks

This needs to be solved at the language level by defining whitelist-like contexts. Some sort of

  with MyContext(allow_sys_cmds=false, network=false, read_disk=false):
     ...

I've seen at least a paper trying to bolt-on such a feature on golang but it's way too convoluted. Of course this doesn't solve everything, especially for languages with magic/collateral effects like running logic on module import.

For context, it's called "an effect system". IIRC, these were developed in the 80s-90s, but as far as I know, OCaml is the only industrial language that offers any kind of support for effects.

Didn't know about the terminology, thanks!

> I've seen at least a paper trying to bolt-on such a feature on golang but it's way too convoluted.

A friend of mine has been working on [1] which is a less computer-sciency solution to the problem for Go modules that doesn't require adding effects to the type system (to be a bit snarky -- good luck getting more type changes into Go, it might take another 3 decades).

[1]: https://github.com/AkihiroSuda/gomodjail

I created https://github.com/JamieMason/shrinkpack in 2015 to try and help with this problem, by creating an offline mirror of decompressed tarballs from the npm registry that you'd check into your repository.

I thought it might have some merit at the time, but it never really took off as an idea.

> Show HN: Tips to stay safe from NPM supply chain attacks

1. Don't use the damn thing.

2. If it needs an internet connection to compile, uninstall it.

For reducing external dependencies, it would be nice to somehow know every call made to a package, generating the call tree to replace. That becomes the API of the internal, replacement package.

Not sure that's possible with JS.

You could theoretically overwrite the public functions of a module, inject some logging code, then execute the originally intended function when it's called with ".apply()" and passing the original arguments in.

That might get you part of the way there.

I'm thinking eval's and prototype overloading would make this a permanent cat and mouse game.

Frankly, I don't see how the node ecosystem can ever be secure. It's random-libraries all the way down. With half of them being useless frippery.

Imho, WASM compiled from languages that have strong standard libraries are the way forward for security. And, maybe node will get fixed in 5-10 years.

A good example of a language going "this shouldn't be a library."- Net used to only have a third party json parser (newtonsoft) but microsoft created a first party json parser and deprecated newtonsoft from their examples and templates

Yes, some automated instrumentation equivalent of `debugger' at the start of the module's first such function, and stepping into each function down to some depth, then starting over with the next line of the module's implementation.

A partial or semi-automated, tool-assisted approach might be possible, if there were a way to restart Node.js between decisions. Each decision point writes an entry "event" that records the filename, function, caller, line number, etc. Then `debugger' is inserted before the next line, and so on.

You mean this: https://npmgraph.js.org

Thank-you, that looks pretty useful. Prior attempts involved `git blame' in projects' package.json per package. For example, to find when a project adopted React x+1 in which version, to determine the next minimum upgrade (to React ^x.y).

Consuming a package also means following its development alongside one's own. Multiplied by each dependency, it's a commitment to a particular constellation of dependencies. Migration guides and codemods help, but ultimately maintenance requires active participation.

That takes time--often more than what's available.

Anyone know how to set this in the repo itself with a yarn config file so it applies to the whole team?

Edit: too long to paste here, but if you ask ChatGPT it will show you how using a yarnrc file.

Here is the entire guide you need to protect yourself from supply chain attacks as a software engineer.

Pick whichever of these will consume the fewest resources over time:

1. review an existing library and all dependencies, and all security updates to them forever (or ensure someone capable does or did)

2. implement the minimal functions you require on top of the language standard library yourself

Yes, this is serious advice, and I have followed it while shipping web applications to millions of people at multiple companies, as a consultant for many more companies, and as a founder and security engineer.

Advice that you know, or should know, won't and indeed can't be followed isn't serious advice, it's just posturing. "Security engineer" or not, if you stood up in that kick-off meeting and with a straight face proposed that the team spend the first 3 months reviewing React before starting work - you're out, and rightly so.

Security and convenience are always in tension, but there is usually a productive, "sweet spot" middle ground. Your "solution" is way off to one side of that sweet spot. The status quo is probably a little too far off in the other direction. But a happy medium can be found where most teams are fine, most of the time, while retaining the ability to take advantage from the open source ecosystem.

Whilst I basically agree with the general point you are making, I wonder if the time has in fact arrived when a greenfield project should choose Web Components over React?

Your suggestion is that I should reimplement React from scratch to avoid supply chain attacks. Like an American fab should extract ore locally to prevent shortages.

If you ignore the other half of the suggestion, yeah. Designating trusted reviewers to audit dependencies like React would be downright cheap at scale. The issue is just setting up and popularizing the systems to achieve this. It’s a little harder than it should be because lots of companies don’t take software security seriously enough.

(And hey, if anyone is looking for people to do this kind of work for their node_modules at very low cost, I’m available right now! `unfrosted_handsaw${107 * 2}@simplelogin.com`)

Good resource

Like the projects but not a Show HN, see the guidelines.

it isn't any of these:

Off topic: blog posts, sign-up pages, newsletters, lists, and other reading material. Those can't be tried out, so can't be Show HNs.

it's not reading material.

it's a set of instructions that you can actually try. so i think it fits.

All tutorials are set of instructions. But tutorial blog posts are not Show HNs. If this was a script, or a tool I could run - yes.

i disagree. i suppose it is borderline, but to me a tutorial is something i try out. it's not reading material.

I can't lie to you about your chances, but... you have my sympathies.

For what it's worth, I prompted GPT-5 Pro to produce a npm supply chain best practices guide, to see what it comes up with. I've not read it yet.

https://gist.github.com/pschleger/c1c36fbde003bea5eee7ce4291...

And a prompt to review a site I built for GitHub Pages, which I'll try this week.

https://gist.github.com/pschleger/8d5fcea6b96d8504ac58bb2f8d...

Reply on news.ycombinator.com