Npm Audit: broken by design?
overreacted.io
overreacted.io
So many people are reporting these in tons of different projects: https://github.com/search?q=regex+denial+of+service&type=iss...
Anyhow it is just annoying and they broke NPM Audit based on these reports.
It is good to fix all possible bugs, but many of these are not anywhere close to the level of bad that the reports are making them to be.
But maybe this is needed to just get rid of these issues in genera? So a wave of regex vulnerability reports and then we build this type of checking into prettier or similar and we do not have these in the future?
EDIT: It appears there as a project that found 100s of CVE reported Regex vulnerabilities in npm projects -- this is maybe one of the sources of mass reports. See the bottom of this resume: https://yetingli.github.io
Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast...
Edit: LWN mention https://lwn.net/Articles/801157/
I'd like to ask what, other than money directly motivates people? Is it prestige? A line on their resume? A requirement for a bootcamp class? In addition, we should re-evaluate the difficulty of submitting a CVE. Is it too easy? The story about a mass of "hey your regex parser could choke on this weird expression[1]" reports suggest that perhaps so. What can we do to make it so that CVEs and equivalents are truly meaningful? Also, just the fact that CVE reports are given a great deal of respect could be the problem, although at this point that seems to be self-correcting.
[1] Some classes of regex parsers are known to be vulnerable by nature, those that do backtracking for example, because their worst-case runtime grows exponentially and can run in unbounded time. This has been known since at least 2009. There are other implementations with better worst-case runtimes, but worse performance in typical cases. The fact that it's trivially easy to look at a regex parser to see if it does backtracking and construct an "evil" expression that breaks it means it's trivially easy to file a DOS report against any such parser.
Although I still have a deep admiration for the Klapötke "Energetic Materials" group at Munich Uni.
Even OpenBSD had to change their tagline to "Only two remote holes in the default install, in a heck of a long time!" (from "Five years without a remote hole in the default install!") Still a pretty impressive track record.
Those "bugs" can be features though - or the work involved to fix the bug meant that high-impact feature work - or other bugfixes, had to be postponed or even cancelled.
Our SaaS frequently gets security "researchers" (read: people running online scanners) submitting emails through our contact-form informing us about click-jacking attacks on our login-page - the problem for us is that we have a lot of second-party and third-party integrations on unbounded origins that offer access to our application, and by extension our login-screen through an <iframe> on their own origin, which is sometimes even an on-prem LAN web-server accessed through embedded devices where we can't use popups to do it properly - let alone switch to a more robust OIDC system - so there is no easy solution that makes the "I ran a tool, gimme $100" people go-away without causing a much bigger problem to now exist.
I use the "mark as spam" button :)
Maybe consider setting up a free-tier HackerOne bounty program? I think they triage to some degree on your behalf.
Spender has a much more nuanced, informed view. I think it covers the issues of the CVE process well, but doesn't make the same mistakes that Greg does.
https://www.grsecurity.com/reports_of_cves_death_greatly_exa...
But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a "critical" bug in Express, potentially allowing my server to be compromised.
I'm not really sure what the solution is here; NPM's just a package manager and doesn't know how you're using a given package. A simple heuristic distinguishing development dependencies and runtime dependencies in NPM Audit might be a start, but that doesn't help with situations like create-react-app's react-scripts where everything, runtime or dev dependency, is a transitive dependency of one package declared as a runtime dependency.
But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?
That's "just" a browser freeze for end users, but still a potential DOS vulnerability if it's in the application's critical path.
For example, a RegEx DDoS vulnerability in Express would show up as high severity, while the same would not show in the bundler you use, or any package that your bundler has in its dependency tree.
A “Critical” bug in a dev context should mean something very different from a “Critical” bug in a prod context. A “Critical” devDependency bug should be either a direct threat to the developer’s context, either by infecting the dev machine or by injecting a supply-chain problem, worming it’s way into downstream contexts.
npm audit is just not granular OR careful enough to address these issues appropriately.
Accepting regexes from user input is a really insidious class of bug that can go undetected for years. I've seen real outages caused by it, so it's absolutely worth doing something proactive about.
They seem pretty adamant on filing CVEs despite what the owner says (It's normally fine but these DoS vulns require very large input to be handed into the function by untrusted sources, which given how these libraries work isn't going to be very common).
Now, I have people yelling at me about dependent packages not being updated because they don't understand version ranges, or because some audit states they are high vulns, or whatever.
Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say.
Because this isn't true. Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. These Regex issues should be fixed, libraries should update to safe versions, things should advance and any incentive we have we should use to make this happen.
I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally, and knowing under which circumstances package lock files were implemented.
They are broken.
> These Regex issues should be fixed
They do, pretty much immediately after they're reported.
> libraries should update to safe versions
I check all the version ranges of dependent libraries when I push a patch with vuln fixes. They get pulled just fine without needing to update every single package. This is what version ranges are for.
> things should advance
Yes but this is nebulous and vague and aside from the point.
> and any incentive we have we should use to make this happen.
I don't see where the disagreement is. This is exactly what happens all the time, nothing is the problem here. I don't get your point.
---
Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time) since they promised it for the v5 release. They were on a time crunch because they were trying to keep with Node.js's next major release timeline, which operates independently of npm's (at least, that's how it was conveyed to me).
So this change got pushed out, had an absolute mountain of bugs that took ages to fix (e.g. at one point adding a new dependency would delete your entire node_modules folder), and promised added security when in reality they do nothing of the sort.
Instead, they cause subtle caching-related bugs, they add an artifact to source control (which is always code smell in my book), crap up diffs/PRs, cause headaches across platforms, and do very little to help... anything, really.
They're super, super broken by design. Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
It was a shit show. They add absolutely nothing to the industry.
Unless we're not talking about the same reproducibility property. Also I don't know what "hermetic" means in this context but I doubt it's anything that npm solves correctly.
Not re-building doesn't make your build reproducible. It just means that you're... not building. If I save the result of a single iteration of an RNG, I can't claim that the RNG always produces the same result because I saved the result somewhere...
The input is irrelevant. I think you should have a look at what reproducible builds really are before evangelizing them.
A reproducible build means anybody on any machine can build the same thing someone else has on theirs. That’s it.
No. A reproducible build is a guarantee that two builders of the same codebase, or the same codebase built multiple times, will result in a bit-for-bit identical of all other builds of the same codebase, every time, guaranteed.
*There are no Node.js-related build systems in mainstream use I am aware of that have any such guarantees. No, docker does not make any such guarantees. No, just because you pinned dependencies does not make that guarantee. No, just because you archived the codebase and vendored your dependencies does not make that guarantee.*
Please educate yourself before dying on a hill for a topic you're misrepresenting entirely.
That's what I said :)
> Please educate yourself before dying on a hill for a topic you're misrepresenting entirely.
I'll say the same. I've only been doing this for near 30 years ;)
But here you go, here's one example:
1) copy source to destination directory 2) run private npm 3) use private npm repo 4) freeze private npm repo 5) use npm install like normal
here's another:
1) check all node_modules directories into version control 2) ensure no native packages are used 3) copy entire directory structure to destination dir
You are clearly inexperienced, or very focused on node.js only.
So when you say "npm staff at the time", do you mean at the time of node 0.10?
> and knowing under which circumstances package lock files were implemented.
> Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time)
The amusing thing about your comment here is the parts which are accidentally correct.
`package-lock.json` files use the same file format as `npm-shrinkwrap.json` files. Always have, although of course the format of this file has changed significantly over the years, most dramatically with npm 7.
The "design" of the shrinkwrap/package-lock file was done rather quickly, since it was initially just a JSON dump of (most of) the data structure that npm was already using for dependency tree building. However, as far as I know, the days were the standard length of 24 hours, so while that may be "short", certainly shorter than I'd often prefer, they were (as far as I know) no shorter than any other days.
This was indeed shipped without any review by even a single "npm employee", which should not surprising, as "npm" was not at that time a legal entity capable of hiring employees. The initial work was done by Dave Pacheco, and reviewed by npm's author (at that time its sole committer and entire development staff), both of whom were Joyent employees at the time.
The use of a shrinkwrap as a non-published normal-use way to snapshot the tree at build time and produce reproducible builds across machines and time was not implemented by default until npm v5, but there wasn't really much to rush, on that particular feature. You could argue that npm 5 itself was rushed, and that's probably a fair claim, since there was some urgency to ship it along with node version 8, so as not to wait a year or more to go out with node v10.
> So this change got pushed out, had an absolute mountain of bugs that took ages to fix...
Idk, I think calling it a "mountain" is relative, actually ;)
> They're super, super broken by design.
I know you're using this phrase "broken by design" in the same sense as the author of the OP means it, but... has language just changed on me here, and I didn't notice?
As I've always heard the term used, something is "broken by design" when the actual intent is for a system to fail in some way, to achieve some goal. For example, a legislative or administrative process that is intentionally slow-moving and unable to accomplish its goals in a reasonable time frame, with the hope that this leaves room for independent innovation. Or a product that requires some minor upgrade or repair to continue working, so that the seller can keep tabs on their customers more easily. That kind of thing.
I think what you mean is not that it's "broken by design", but rather it's "a broken design". Unless this is like "begging the question", and I should just accept that I'm gradually coming to speak a language of the past, while the future moves on. It's certainly not intended to cause problems, as far as I'm aware.
If you really do mean "broken by design" (in the sense of a tail light that goes out after 50k miles so that you will visit the dealership and they can sell you more stuff), I'm super curious what you think npm gets out of it.
> Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
As of npm v7, there's no longer any practical reason why it can only audit the lockfile, rather than the actual tree on disk. Just haven't gotten around to implementing that functionality. If you want it changed, I suggest posting an issue https://github.com/npm/cli/issues. There's some question as to whether to prioritize the virtual tree or the actual tree, since prioritizing the actual tree would be a breaking change, but no reason why it can't fall back to that if there's no lockfile present.
But even approaching build reproducibility is impossible without lockfiles. If a new version of a transitive dependency is published between my install and yours, we'll get different package trees. If we both install from the same lockfile, we'll get the same package tree. (Not necessarily the same bytes on disk, since install scripts can change things, but at least we'll fetch the same bytes, or the build will fail.)
> So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
You do not recall correctly, sorry. (Or maybe you correctly recall an incorrect explanation?) The answer is reproducible builds. Using a lockfile does reduce network utilization in builds, but not very significantly.
> Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
I hope my tone is civil and playful enough in this message to not consider my response "yelling".
Might be nice to be able to tag libraries that should be ignored in audits. Perhaps there is such a function, not really a NPM expert. But if your projects has too many of these "high severity" problems, you probably stop doing them.
Still, I think the availability of such audits from the package manager is quite neat. As an embedded dev I think these are certainly luxury problems.
If people are competing for CVEs, then why not work out a way to better differentiate them them through scoring and make this visible. The goal would be for attention to shift to the scoring instead of only a CVE count. Offer both views of the world, so tools could still fall back on the problematic listings they get today.
Apply machine learning to classify CVEs based on the reputation of the reporter, blast radius, or other criteria. Use that to drive community review and scoring.
I would not see this a panacea because it brings a lot of challenges (a la StackOverflow), but it would be much better than what we have today.
I suspect it's an impossible challenge, but I only dip into this domain casually so maybe someone has better ideas.
I don't think there is a perfect way to do this, and I don't think there is an absolute standard that can be applied. It will be unfair to some people, but the system should have options for resolution when there are egregious mistakes. I'm not making the claim either, that the views of the data you are interested in are the ones I might be interested in. A good system would provide some different levels which itself is an incentive towards better research that would break through.
I think regexes are often used as a quick and dirty solution to problems, which should be solved differently. But once the regex "works" and is in place, others begin to rely on that output. Over time cruft begins to accumulate and the regex is forgotten or at least never replaced with anything more appropriate.
Good news (well, probably). JavaScript (ECMAScript 2018+) now supports named matching groups.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
Surprise: The most common parser combinator libraries do backtracking. That's exactly the problem. Any solution as widely used (if not overused) as regular expressions ends up exposing a number of dark corners where the design isn't as clean and tight as you would want it. There are lots of better ways, but most of them are specialized and are totally unsuited for significant areas where people need something.
That said: yes I've used LR(1) parsing (not LALR) using a library that uses parser combinators with a good interface, and it's more powerful than regex and worth it for the right usecase.
I'm not a react developer, I was experimenting with it for a new project. I finished the tic-tac-toe tutorial, then tried to throw bootstrap on top to build from there. It told me there was 97 vulnerabilities (85 moderate, 12 high)...
I just deleted the directory and went back to vanilla JS. This is a fun side project, I don't need that.
My tweet about it: https://twitter.com/preinheimer/status/1402785757962592256
However a beginner that doesn't know what impact they have of course is scared if when installing a library it tells you that there are all that vulnerabilities.
After all, npm can execute any script with the users permissions on install…except often (compared to bash) it’s less easily inspected due to the common use of nested dependencies!
I, too, would delete my node_modules, and if I even wanted to move forward at that point, would probably waste at least half a day looking up the Critical vulns and discovering that they are probably not at all critical in my particular scenario. Like not at all for the 99.99% use case.
After experiencing something like that, it’s just like the article says. “The boy who called wolf.” Really terrible use of the labels “Critical” and “High”. The labels are fine, but the way they are applied is just stupid.
I don’t see any recommendation in the nodejs or npm docs for any other approach.
It may be commonsense and obvious to you, but I would be really surprised if commonsense and common practice overlap significantly in scenarios like this for all but the most security conscious.
Next up, npm link: broken by design
Synopsis of the chapter: A command with broken behavior that has been reported since as early as 2015, but that "got lost" every time the winds changed and the project decided to change where to manage bugs. What will happen in the latest attempt from an affected user? Tune in and be ready for an exciting ride!
https://github.com/npm/cli/issues/2372
Spoiler: bugs are not sentient beings that solve themselves just by closing the issue (or the whole issue tracker, for that matter).
EDIT to clarify: Sorry for the snarkyness. I just find it funny in a sarcastic way that up until I reported the issue in 2020, the issue had been reported repeatedly but "lost" in the way because the project closed or ignored the issue every time it changed issue tracker. Which happened twice since 2017! so go figure the amount of reports that had gone to waste. On the flip side, this time they haven't changed platforms (yet), although the issue has been closed prematurely anyway.
I would like to wait to see if the rearchitecture for npm 7 actually allows them to test for regressions more productively, but at this point I don't know if I have the stamina to wait for my company to migrate to node 16.
Someone offers me a job doing Elixir or non-webapp stuff and I'm out. Probably permanently.
And when you inlined a version that later really does have a vulnerability, it is not easily flagged or fixed by your consumers.
The tension between "upgrades (especially of indirect dependencies) might break" and "upgrades (especially of indirect dependencies) might be necessary to fix bugs or patch security vulnerabilities" is real. There is no magic bullet to get around it. There are practices to try to balance it -- which generally involve ecosystem-wide commitment to backwards compatibility, reflected in semantic versioning (and minimizing major releases).
That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem. It makes the dependency tree even more insane, and it also accomodates a false belief that it's fine to lock to very specific versions of dependencies -- because it won't prevent other dependencies from co-existing in tree with other conflicting requirements after all. Which then accomodates a false belief that dependency maintainers don't need to worry too much about backwards compatibility, after all consumers can just lock to specific working versions... and now we wind up with insane dependency trees which depend on vulnerable versions in ways that require a whole bunch of releases to resolve.
I think a big part of it is that due to much stronger pressure on bundle size than most other environments, each library tends to be small, so there have to be more to carry the same amount of functionality.
Duplicates are certainly a contributing factor as well, and small bundles compound with allowed-duplication to further increase the tree size. I think that small package size also probably makes it harder to require a single version for each dep, since there are going to be more edges in the graph and therefore more relations for library maintainers to keep track of (including what would in other languages be intra-package requirements), so you're more likely to get version incompatibilities.
I would add that I think the devDependencies solution is underrated. Not using it is a bad practice. npm has a nice feature, `npm prune --production` which will remove all the dev dependencies for you resulting in a clean build of the program. You can easily have things set up so that none of the development dependencies that have all these audit issues are ever present on your production machines if you do things right.
Furthermore, if you have a project with end users, devDependencies allows you to make clear in your issue template that audit issues that don't show up in a production build are very likely to be false positives and will probably be closed without comment. If you aren't properly isolating your dependencies, you can't take advantage of this.
At the end of the day, as you say, the issue is largely with Node projects having too many dependencies and a large number of relatively minor issues that affect very specific use cases get reported. That's a hard ecosystem problem to solve, not necessarily something that indicates an inherent problem with npm audit; but if someone isn't using devDependencies, that alone could constitute an enormous improvement in their workflow.
However it's incompatible with optionalPackages, so I'm carrying around some tertiary dependencies that are a small but noticeable fraction of the entire archive size.
Aren't npm dependency trees so large because JavaScript doesn't have much of a standard library? And also, similarly, the community is so large and has been moving so fast that even de facto standards have difficulty forming and surviving at a large scale across the community.
Add to that everything else—the fast pace of changes, javascript "culture", the weak standard library, the tendency to patch in what ought to either be basic language features or else avoided in favor of more-vanilla idioms, often in competing and incompatible ways—and all that is how you end up with 20 slightly-different copies of the same damn library in your dependency tree, and then 20 other copies of another library that does the same thing.
447,211 weekly downloads for what can be done in vanilla JS with foo % 2 === 1;
If you're writing straightforward application code where you already know you have a valid number, then this package probably isn't for you. You can just do num % 2 === 1.
function is_number(val) { return typeof val === "number"; }
With a function that easily written, no one should have any excuse to depend on a third-party dependency for it.
Here's `is-number` (https://github.com/jonschlinkert/is-number/blob/master/index...):
module.exports = function(num) {
if (typeof num === 'number') {
return num - num === 0;
}
if (typeof num === 'string' && num.trim() !== '') {
return Number.isFinite ? Number.isFinite(+num) : isFinite(+num);
}
return false;
};
He abandoned the projects, but:
* https://github.com/jonschlinkert/is-odd
* https://github.com/jonschlinkert/is-even
Of course that doesn't stop them from getting ~500k downloads/week.
For my hundreds of repos (Java, Scala, JS, Typescript, Python...), Snyk flags 99% of the CVEs for the JS repos. Shocking how I've only seen a few dozen or so Java based CVEs flagged over the last few years.
Perhaps it's because my NPM based repos have ~10K more dependencies? That and the Java stdlib handling most needs w/ the vanilla lang.
Some of us are considerably salty about it. Especially the design-by-PR aspects of the whole thing that have resulted in confusing gyrations from one version to another.
https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabili...
We disable [1] audit entirely because it's not a good default behavior within a monorepo. It spams the hundreds of developers with the list of "vulnerabilities" on every install, but only a few folks should really be upgrading packages.
We then run audit in non-blocking CI and track the total number of issues and mostly focus on critical ones.
For vulnerabilities that we determined weren't an issue ever (vuln in frontend framework we didn't use), or weren't high priority enough to P0 through, we needed some way to ignore either permanently or temporarily specific vulnerabilities.
Given the enormous dependency sets eg react create, you'd think the tools would be better at managing them.
I can't believe no one here has mentioned the fantastic tool "better-npm-audit" which can be included as an npm dependency[0] and lets you add specific vulnerabilities to an ignore list.
The ignore list is actually a JSON config file stored alongside package.json in the repo, so only one developer ever needs to see the npm audit warning and can mute it for everyone else (after getting their PR approved).
Even better, the config file lets you specify an expiry date for each entry in the ignore list, and provide a note, such as a link to the upstream issue being worked on, so that you can periodically be reminded to go back and check if a new version is available which can give more confidence that your code really isn't affected.
I think that developers might have to be instructed to use the "--no-audit" option to "npm install" if they don't want to see the (false positive) warnings that the default behaviour produces, and that's a bad habit to learn if not all projects they work on are using "better-npm-audit". I don't know if there is a way to make that option the default on a per-project basis.
edit: we're going to move to this. My implementation is a is a file with cve to ignore | reason | jira ticket but I think we can slam all that in there. And our expiry is manually checked monthly, so this is a big improvement.
From my experience most audit flags happen because a new vulnerability is discovered, which means stopping a deploy doesn't actually do anything helpful.
You can read a bit about the approach’s benefits and drawbacks from someone at Google here: https://medium.com/@Jakeherringbone/you-too-can-love-the-mon...
Once you have the tooling to do this you can use the monorepo to store all kinds of interesting things, like built artifacts or the contents of the CDN, since it's basically just a giant hierarchical KV store with a global version.
https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
Or, in other words, it's not that insanely complicated, because they have tools that make it look a lot like multiple repositories. And the large companies using multiple repositories have tools that make them look just like those monoreps. And HN has all those interesting threads full of people saying why one is better than the other.
I never entirely understood what it was in subversion's internals that prevented it from being used as a DCVS. We could figure out version numbers with branches and multiple repos. If not then, certainly now.
There's a space between subversion and Git that could be occupied with something that sheds the worst behaviors of each and makes something better.
https://research.google/pubs/pub45424/
That said (1) “you are not Google” as the saying goes (2) Google built their VCS from scratch in house for scale.
However it’s not tops hard to do this with just git and some rules/patterns for organization.
It's more complicated than that. Initially, there was Perforce. Then, Google grew client-side tooling on top of that ("g4", rather than "p4"). Then, someone grew client-side Git on top of that (for reasons, good or bad). And at some point, the Perforce backend simply stopped scaling, and a new backend was written (named "piper", but not the "piper" most non-Googlers think of). And once that was in place, assorted more "make it scale" tooling could be built on top of that (and, at that point, I stopped using the client-side Git, because CitC was simply That Good).
So, "from scratch" is a bit of a stretch.
We come from separate repos on past versions of the code and the monorepo setup has sped dev up dramatically, even for a team of 2. Having the docs next to our code and autogen some of it from the source code, all is the same repo is just one advantage.
It's a small project but you can view the setup here: https://github.com/lowdefy/lowdefy
While complicated, in my experience the tooling to get the same productivity from hundreds of separate repos is at least as complicated and generally discourages folks from contributing to codebases outside their own.
Our monorepo encourages a collective velocity culture where everyone is pushing every project forward. If you upgrade React or Node, fix a security issue, or implement an optimization, the hundreds of apps are all improved at the same time. It's harder for one team to "leap forward" in terms of tech stack quickly, but at the same time it's far less likely we end up with hundreds of outdated or abandoned codebases since everyone is collectively improving the repo together.
For example I've rarely seen a single engineer upgrade a thousand separate projects across a thousand git repos with complex nested dependencies very successfully, but that happens every day in our monorepo.
Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities.
If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense.
I do agree with the author's note about providing a better way to provide feedback on severity reviews.
npm audit is better than no npm audit...telling people it's broken by design is going to discourage them from using it completely. smh.
A big part of the problem is there is no reliably way to "just upgrade it" today in npm:
- `npm audit fix --force`, which is supposed to do that, is buggy and doesn't work
- There is no way to override a transitive dependency with npm (there is with Yarn though, so hopefully this feature will come to npm soon)
- Sometimes the fix in transitive dependency _also_ includes breaking changes (e.g. because it wasn't backported), and so updating it subtly breaks the logic
>Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea
I don't think databases can do it, but what I'd like to be able to do is to be able to provide advisory that the way _my package_ uses a concrete transitive dependency is not affected by that vulnerability. Because as the package owner I _do_ have that context. I understand there may be significant issues with this approach though!
But what if one of your contributors slips in a merge that uses the vulnerable code path of your dependency... Does this "not affected" marker still exist, and now you have vulnerable code? Does it disappear with each version?
What if someone maliciously adds a "not affected" marker? To a package they intend to exploit?
Edit: Again why the heck am I being downvoted?
If your threat model is a contributor submitting malicious code, your problem is not something npm audit will help with either way.
If a malicious actor is able to add the "not affected" marker, you have bigger problems.
The threat model you're talking about neither seems realistic, nor like something npm audit can help with. The attack vector of a contributor sneaking in malicious code is dealt with by only giving the commit bit to trusted people, and reviewing code yourself.
How is this not realistic when it has already been seen in the npm ecosystem multiple times. For an example of this in the wild see the event-stream (crypto-mining trojan) fiasco: https://github.com/dominictarr/event-stream/issues/116
Dependencies are a target for exploit, your package may be safe now and then become unsafe in the future, either intentionally or unintentionally.
I do not think this is a good idea as it allows consumer's of a library to utilize that dependency with untested, and unspecified transitive decencies What happens when a transitive dependency breaks the 1st level dependency? What a PITA that would be to try and fix.
The last Create React App I worked on (~3 months ago) had over 500 "vulnerabilities" reported by npm/yarn audit. Most of the reported vulnerabilities were obviously junk. As the author noted, there's no need to report vulnerabilities in the same dependency in every path through the dependency graph. The noise made it very difficult to sift through the output for anything useful. Even then, I have my doubts about how applicable the results are because with tree shaking of an SPA, it's quite possible the vulnerable part of a dependency is never even used.
Upgrading a dependency can go anywhere from trivial to absolute nightmare. Usually somewhere in the middle where it takes time and effort to do right. A typical JS app nowadays has hundreds if not thousands of dependencies. I'd love to see a world where "just upgrade" is reasonable advice, but we are not there.
At this point, what should you do now?
The author of dependency of dependency is probably not going to touch it because it is fixed in the new major version. The author of dependency is probably also not going to fix it instantly because it requires major rewrite.
Yes, you now don't get vuln. notifications for the original repo, which is an issue in itself. It would be nice to mark a CVE as mitigated in your package.json and to also mark a resolution to still pick up CVEs from the original package. E.g
{
"dependencies": {
badRepo: "1.0.0" // has a dependency that is vulnerable
},
"resolutions": {
"badRepo/**/dependency": "git://github.com/org/dependency" // fixed, but now won't report new cves from the original badRepo/**/dependency package. Would be good to specify that we still want reports for the original repo
That is how we handle such issues in our team, we also review any forks at the start of every sprint to see if it has been resolved and we can remove the fork dependency.
If the vuln is valid and exploitable in your system, what other choice do you have? It's the pitfalls on depending on 3rd party packages. If you are using an old major version that now has a vuln that is only fixed in a newer version, NPM doesn't make that situation worse.
Caveat: These are my 5 minutes thoughts, I could probably do a better, more thorough write up.
If I wanted shitty false alarms about bogus security issues, I'd get a PCI-DSS audit.
Imagine if you had 3 days to fix the regex DoS issue shown there, screw your release freeze and your current sprint plans, and you have the real working environment in some companies.
I've also heard reports of people trying to claim bug bounties for similar reports, or security vendors that run automated tools that detect for issues of similar (lack of) value.
CVEs try to supplement with flags for remotely exploitable, etc. but it still intentionally leaves a lot of space for interpretation, which is necessary for any normal enterprise.
The problem comes in when analysts (or their managers) interpret inflexibly, without appropriate technical context, or without understanding business impact and tradeoffs.
If you look at the workflow, it is hard to close the loop from engineering or IT back to security. We need a set of controls for secops departments' output relevance and departmental interoperation.
One if them is scanning all your dependencies, and its so frustrating. Because that team obviously has no idea what any of the dependencies do or how they’re being used. All they see is a red flag, and tell you to fix it. Good luck when they tell you this days before a release, and a week after the code is frozen. They’ll just block your release without a second thought.
Funnily, in our last release, some of our NPM packages were flagged as a risk, obviously without explanation. The thing was, these packages where dependencies of another package. Obviously we can’t go around updating open source code, just because the security team in our company told us.
This isn't obvious to me. Most open source projects accept contribution from others.
Of course they do, and I'm more than happy to help with open source projects. My point was that, we can't do it, just because a security review at my company says so. It's not just as simple as updating the version of the affected package, there's also testing involved, potentially fixing issues due to using a later version. This would almost be a full-time job.
What if it requires major upgrades of their framework or toolchain they don't want someone doing drive by?
What if they require a CLA that your legal team won't let you sign?
So the article boils down to “a bunch of these vulnerabilities aren’t applicable to my app which is built using a specific NPM package”.
Congratulations. Welcome to the world of practical information security.
As a security engineer, we’re lucky if your favorite package manager even associates vulnerability information with your packages. Never mind that you’re pulling in code at build time from who-only-knows-where that almost certainly wasn’t security reviewed. But that’s for another post.
Now you have a package manager that is kind enough to tell you that there might be a vulnerability, and you’re upset because NPM did not have specific logic to understand the mechanics of one of the packages it manages? And the upshot is that you have to apply judgment and attention to each notification? Is that a tear in my eye- no, wait, it’s just an eyelash.
How many packages are there? I’m sure the NPM guys have nothing better to do than to build context awareness for every package in their repository.
In all seriousness, I would love to see context awareness in vulnerability reporting. But expecting a package manager to understand that because of your specific choice of framework, that the DoS could only be conducted by an admin of your app, seems unreasonable to me.
People throw around “false positive” as a catch-all for “I don’t care about this”. But there are a number of distinct reasons one might not care:
- the scanner is wrong (e.g. there’s a code bug in the scanner like detecting “printed” instead of “sprintf”.
- the output is wrong because the vulnerability isn’t a vulnerability anytime, anywhere under any circumstances
- the scanner is correct, but environment or mitigation’s mean it doesn’t apply to me or the severity is wrong in my environment (this is the case here)
- the scanner is correct, but is giving me output I don’t care about (eg I want to filter for only high/critical but I can’t)
- there is so much output that I can’t pay attention to all of it; it’s so overwhelming that I can’t stand to look at it
Many security products have problems with output that is too verbose. This seems like a trivial problem to work around here; after you’ve triaged that a particular vulnerability doesn’t apply to a particular project, then filter it out with grep -v (our put a bunch of such lines in a bash script and always pipe npm audit output to the script.
Also, I sympathize with concerns that the vulnerability reporter perhaps scored the vulnerability too high. But there’s no perfect solution for that, and I’d rather be aware of a vuln and choose to ignore it, than not be aware at all.
I'm not even sure if I'd classify it as a security bug at this point, I'd just classify these examples as configuration options to be aware of. Document and ignore.
With the explosion in Javascript packages to implement trivial behaviours, the NPM dependency hell and all the other cruft that "modern" frontend development requires, there are more important security issues to monitor.
Many real issues exist, but their classification is ridiculously flawed. Every security researcher tries to mark their vulnerability up to be the next Eternal Blue of Javascript development, but these vulnerabilities rarely matter.
I think a much bigger problem from a security standpoint is not necessarily the quality of these bug reports, but the sheer number of dependencies even a basic React project has these days. Supply chain attacks are real and the javascript world can do with some dependency purging. Any of the 200 single-line Javascript libraries can be compromised at any point and infect developers all over the world the next time they update their dependencies.
The way people "just" seem to add new dependencies to projects terrifies me. left-pad hit the frontend world and changed nothing. twilio-npm infected developer machines, and create-react-app imports over 1500 libraries as if it's the most normal thing in the world. It's absolute madness.
I have no idea how to fix all of this. It's only a matter of time before someone replicates the research people have done in uploading packages with typos in the name (which have reached into big, famous companies like Apple and Facebook) and start doing some serious damage. Maybe it's already happening: a package having been taken over for scraps, the owners lying in wait while everyone downloads and updates their React/Svelte/etc. packages, ready with their cryptolockers to strike frontend devs and build servers the world over with the push of a button. I wouldn't be surprised, not in the slightest.
Ideally you'd want to show only relevant alerts, but... how? You'd need to know which kind of errors are relevant for a particular project, but that'd require solving the halting problem. This is made much worse by that it's JS.
Some libraries have an enormous complexity and attack surface. Take a database interface -- there probably is a vulnerability in some obscure corner the typical person may not even know exists.
I think though at the very least some improvement could be made by better priorization and categorization. DoS by exploiting a regex parser isn't that big of a deal if your project is just getting started, but an exploit allowing arbitrary code execution would still be.
In Dan's twitter thread, he calls this out as a viable solution.
It seems like a lot of this has been designed for Node (backend) development, whilst ignoring the fact that NPM is probably used more heavily for front-end development at this point.
In highly regulated industries, shipping code flagged as having a vuln without a manual approval could be a liability.
This wrapper around npm takes an allowlist argument, and our procedure is for an engineer to review the failing build, determine if the vulnerability (ugh, usually regex ddos or prototype pollution) is present in code that runs only at build time with trusted inputs, only on the client which is by definition untrusted, or in our webserver which takes in untrusted input.
As long as it's either of the first two, we document it in a commit and comment and redeploy. It's annoying, but it's far better than npm audit forcing a fix.
Let me plug this as it contains a lot of references https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Meanwhile I'll try to get someone from IBM involved in the OpenJSF collab space
A vulnerability is a vulnerability, whether it applies to your context or not. A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?
- If your dependencies have security vulnerabilities, apply the updates.
- If you cannot update because there's no fix available, let your org or you assess the risk and go from there.
- If you cannot update because it breaks your app, {find a replacement, fix it yourself, let your org or you assess the risk and from from there}.
A sensible org has a process that freezes releases until known security issues are fixed. Freezes can also be opposed by devs and are evaluated on a case-by-case basis (because sometimes they are not relevant to the product, or someone steps up to take the blame for incidents and the org agrees).
We might not like it because it disturbs the "flow", but it's just part of the engineering process. More to the point though, why not take this opportunity to teach newcomers how to code properly, pick well-engineered and -written programs, and handle this vulnerability management process altogether? In any case, I hope newcoming-dev is not going to push to prod anytime soon. ;)
edit: formatting
I think a better metaphor might be, the button on your key fob that opens the trunk doesn't work, so you have to open the trunk using a physical lever. Every time you start the car, a loud warning siren sounds, and a red message appears on the dashboard to tell you that there is a "high impact" problem with your car, and you need to take it to be serviced. If you were merely the owner of the car, and other people also had to drive it, you might understandably be the target of several complaints about this "high impact" problem.
Ever seen apps which are basically impossible to patch because devs have ignored patching for so long that it’s basically impossible to version bump things sanely? I have.
For projects you own you'd have to flag each dependency path though, because for example, one dependency may not have the input for the regex exposed to the end user, while another dependency could.
Maintainers of libraries should also flag the security issues, and an issue with these two statuses on them wouldn't be raised by default. Options should be available to list them though for auditing.
For more security critical teams/projects, a per project setting to alert about any issues the maintainers have flagged irrelevant or mitigated and you'd have to accept them before it it would stop alerting about them.
Is yarn the better option? What is our path forward?
Java has rich standard library. Some would say - too rich. But anyway - very few people would need customized collections, standard library covers all the needs. And where it does not cover all the needs, there are few commonly accepted additions like apache-commons or google guava. So that one solves `leftpad`-like issues.
Another difference is that Java is old. Most needs were covered by many libraries and few libraries survived which are good enough. It's again some commonly accepted wisdom, so you don't really need to search for many options. You have one good enough option that you'll use and move on.
So it's not about maven/gradle vs npm, it's more about ecosystems. I don't think that porting maven to JS world would change anything.
It simply don’t happen in Python, Go or Rust (or even Java) because the languages comes with a rich standard library. Javascript comes with just the basics, everything else is a dependency. It’s not uncommon for people to audit their dependencies in Python or Go, but you pull in maybe 10 or 20. A basic Javascript app easily pull in 100 times that, how are you suppose to deal with that?
This has not been my experience at all. Most Rust developers unfortunately seem quite happy to adopt modern programming bad practices. (There are many exceptions, of course.)
Example: I went with the first Rust program I could find installed on my computer, tealdeer. It's a dead simple program: run `tldr progname` and it will print a handful of examples of how to run `progname` in your terminal. Run `tldr --update` and it will download the latest version of the database containing these examples from a web server.
To build this extremely basic CLI program, I need ONE HUNDRED AND NINETEEN distinct crates.
[1]: https://pnpm.io/
There are still problems:
- The decisions file gets unweildy, mainly because every time it fixes something it writes to the file. You probably only care about ignores. It's also append only, though you could manually clear it down sometimes.
- It always defaults to fixing at the deepest level, which is.. not ideal for NPM. On my machine (a not very old Macbook Pro) NPM simply can't update a dependency 20 layers deep in the tree, ie `npm update nested-dependency-from-hell --depth 20` will eventually time out and won't fix anything. So you have to manually crawl up tree yourself and find the thing that can be updated - or just ignore it until the thing right at the top of the tree gets updated.
I'm not surprised to see Dan posting this though. I agree with everything he said, so I don't mean this as an attack, but a lot of the time the thing at the top of the tree we're waiting for an update on is create-react-app. It must be incredibly annoying how many Github issues get opened on that repo every time there's a new NPM advisory on some 20-dependencies-deep parser it uses for something or other.
I do like the suggested fix that a maintainer can use their knowledge of the specific usage to say the vulnerability doesn't apply. Often in these threads there's a perfectly good explanation of why it isn't a real issue, and then people come back with "Okay but can you please update it anyway because I'm forced to audit and my security team/CI are yelling at me".
>> Unfortunately, there are hundreds.
This is primarily a result of the absurd number of dependencies NPM encourages (requires?) people to use. The duplicates are also there in part because of the large number of dependencies and should not be shown more than once by the tool.
Stop building projects with an absurdly large dependency tree, this is just one problem that results from it.
The answer here is probably some kind of static analysis to know which packages end up shipping in the actual bundle to users. I think Dan referenced some work in that regard.
So even more tools?
Even a simple app with only React as a production requirement will have dozens of issues a month.
There are some packages that don’t have as many dependencies such as Typescript or Prettier, but that’s not enough, since the most popular bundlers have hundreds of of dependencies.
No matter how careful you are, you get flooded by security issues.
Javascript seems to prefer millions of tiny dependencies of thousands of larger libraries, which is a choice that can be defended. The difference is not necessarily one in lines of code or binary size, but one in amount of vendors trusted. Many libraries also handle trivial code that (in my opinion) should be part of the programming language or basic developer knowledge already. The is-something packages that fill Javascript dependency trees to the brim can only be considered as failings of the language in my opinion.
As a developer, I trust parties like webpack, gulp, and Facebook, but I haven't heard about jonschlinkert (nothing against him, just a random name I picked) and I don't know who maintains is-number, is-path-cw, is-path-in-cwd, is-path-inside or path-is-inside and how reliable they are. All of these dependencies seem like excellent methods in a library, but they all could've been part of a single dependency no more than 60 lines of code in length. Many NPM packages feel less like libraries and more like automated StackOverflow answers. Adding a vendor to your supply chain for just 40 lines of open source code is just inefficient; why risk trusting yet another vendor to not inject malware in the future like this?
The Java world has some popular names like Apache, Google and Jetbrains that maintain large libraries so it's easy to build a chain of trust. Rust is moving the Javascript way, with hundreds of megabytes of dependencies from thousands of individual repositories, but at least most of its packages add something nontrivial.
C++ doesn't have a package manager, at least not in the same way other languages do. C++ libraries usually come from very specific toolkits or single sources (like Linux package managers). There's tons of packages for C++ development, but all of them are kept up to date by a single organisation on my machine.
If I put a database with default credentials on the internet, there is both a vulnerability and it is exploitable. Bad. If I run a database with default credentials on my dev machine, it is vulnerable, but not exploitable. Perfectly fine.
For real security work you also need to think about impact. Hacker dropping production database = we all lose our jobs. Co-worker connecting to my computer and dropping database as a joke = no real harm done.
So three things to think about: - Vulnerability - Exploitability - Impact
What I really don't like about npm audit is how it presents itself as "security tool" and how vulnerabilities are presented. "6 critical, 10 high vulnerabilities" with a red color screams "fix me now!!!". This is not fair to users because npm has no idea of either the exploitability or the impact of the vulnerability.
Why present users with a prompt "please fix me now!!" and not even mention that exploitability and impact need to be measured first? Seems like they forgot that prompt...
This is like getting mad at the guidebook for showing which plants are weeds when your neighbors complain that your unmaintained garden is full of weeds.
If Dan says "npm audit is a stain on the entire npm ecosystem", maybe it's safe to say that Create React App is a stain on the entire react ecosystem. The best time to maintain it was every single month of its existence because that's how you maintain software.
Facebook has abandoned Create React App. Dan stated that he intentionally does not maintain the project. Rather than complain about npm audit, they should give Create React App over to the community who actually use it instead of keeping it shambling along as a zombie with their name on it.
And if Facebook doesn't want to give it up, the best thing we can do as a community is to move on to any of the other great tools available that are actually maintained.
Wouldn't it be more like getting mad at a guidebook which falsely claims that every plant in your garden is a weed, forcing you to second-guess all of its assertions and therefore wasting a lot of your time?
I've never ever got `audit fix` or `audit fix --force` to solve any of the mentioned vulnerabilities. Ever. I even relied on downloading every dependency one by one to find that there where other offending packages. I just gave up.
It's really useless and deceptive.
That doesn't seem to be the meaning of "broken by design".
There's a push to address the npm audit situation. It's an initiative under the OpenJS Foundation. I kinda started the whole conversation by implementing a tool that makes it acceptable instead of ditching npm audit.
It's called npm-audit-resolver and I've written about it here https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Also check out the collab space and the tool itself https://github.com/openjs-foundation/pkg-vuln-collab-space https://www.npmjs.com/package/npm-audit-resolver
The [1] yarn package manager is much nicer to work with in many more ways.
What are alternatives? A way to ignore or mark a dependency as safe? Could this be abused if an author can just mark a dependency as safe?
Or perhaps, actually analyze syntax with a tool like ESLint (parse -> AST -> validate) to check that dangerous parts of libraries are not in use? This solution comes with it's own complications. Who is authoring these validations?
Perhaps there are other strategies I'm not aware of.
Also going through an audit result in a CLI isn't really the best experience. I wish I could just click a link and open up the report in a browser to drill down into issues.
Don’t like it? Try using more maintainable dependency trees.
In essence, if you are scanning an environment that is already compromised, `npm audit` results can't be relied upon if you are running it in the same environment. It should be self-evident but I'm sure plenty of people use the tool this way.
For a proper assessment, such differences need to be encoded in the security advisory, and the audit tool needs to analyze if the code is called at run time or build time, and then act accordingly.
Because I've felt that way for years.
Well yes, correct, well done. By this metric, every security tool ever written is probably pointless.
I mean, this is why people love language with deep, solid standard libraries. You don't have a situation where a problem in a sub-sub-sub-sub-dependency provokes five different groups of people to all issue an update, one after another. You just upgrade your underlying installation to the latest patch version and continue.
Language ecosystems where a few lines of code constitutes a library fundamentally result in you being dependent on a huge number of outside people to cooperate on updates. That's what's broken. Not a tool which tells you that you have out-of-date libraries and, by the by, hooks into CVE databases.
OP should stop and consider whether it might be beneficial to inline some of those dependencies before dismissing it out-of-hand. If you never use the dependency in such a way as to present a real security risk... and you don't need feature updates from upstream, i.e. the software is fine as-is when you first incorporated the dependency... then why wouldn't you inline the dependency?
If anything, inlining the dependency will allow static code analyzers to point out all the parts of the dependency which you're not using (i.e. dead code) and eliminate it all. That way, even if the dependency were to be discovered to have a security fault, if the faulty code was in a section that you eliminated as dead code... then you don't have a security problem in the first place!
Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.
Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.
Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....
There is no way to "mark them as such". That's half of the issue. The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. They just ran `npm install`, and npm adopted default behavior of showing these warnings. For a lot of people this is their first programming environment.
>you can't automate this process, so the only alternative is to ignore all vulnerabilities.
I don't think the answer is necessarily automation. But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
Or at least I'd like npm to offer a reliable way to update packages deep in the tree and override the resolved versions. Currently, there isn't such a way.
This attitude makes me kind of uncomfortable. Like, I have taught software development to a decent number of folks, but I've always done so in a relatively isolated environment. If one is buying into web programming, I have a hard time feeling like it matters that it's their first programming environment--it is a hostile place (the web) and some understanding of that hostility is pretty high on the list, I think, of Things To Get Used To. There's definitely a tension there with "don't overwhelm a novice", but I don't necessarily think optimizing for the novice case is wise, especially when we want those novices to have their heads on a swivel, too.
> But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
I definitely agree with this, though, and this is a good way to help make something like `npm audit` more intelligible and useful.
I think that, even today, this is a bold claim to make. Do you have something to substantiate it?
I think the point though is that security warnings need to be actionable and high-signal. Experienced folks are absolutely tuning out the security warnings on npm install, because 95% of the warnings are like the examples in the post -- I know they don't affect me/my use case and there's nothing I can do about them anyway. The effect is only compounded for novices who run "npx create-react-app hello-world" and immediately see something incomprehensible about a vulnerability in react-scripts > webpack > watchpack > watchpack-chokidar2 > chokidar > glob-parent. It either discourages them from programming entirely or it teaches them to ignore security warnings.
I don't disagree with your overall point -- e.g. we should absolutely teach novices "here's what XSS is and how to avoid it" early and often. But if a dependency manager is going to surface a vulnerability alert every time I install dependencies, the alerts should be 1) high severity (to the point where I should actually stop using the package if I am unable to patch/upgrade) or 2) at least immediately actionable. The current npm audit implementation does the opposite -- 95% of the alerts are totally irrelevant to my actual security posture, and the suggested command to upgrade a vulnerable dependency is unreliable and can actually downgrade to an older, even-less-secure version (!).
Can we please tell beginners not to start programming with node.js?
Teaching beginners to start with “go-to” technologies became industry standard already as it helps corporations to become more and more monopolist and dictate new industry standards.
Or do commenters here actually believe that npm audit should treat a DoS of a development machine as a non-vulnerability?
(Please tell me it’s the former)
I believe that npm audit should treat a DoS of a development machine by a trusted developer as a non-vulnerability. "Code I (or a fellow committer) wrote uses a lot of CPU" isn't a vulnerability. If I care to prevent this, I should run said code within a cgroup with limited resources, not panic about theoretical expense in one part of the codebase while necessarily allowing arbitrary execution elsewhere. "npm audit" is crying wolf, just as the author said.
I like the proposal [1] near the end: "If I own an npm package I need to be able to tag a certain transitive vulnerability category as not affecting my usage of that transitive package." This is particularly important for npm given things like create-react-app but would also be a good idea for "cargo audit" and such.
[1] https://twitter.com/dan_abramov/status/1412380714012594178
Or even, since it's in fact the language itself that sets the tone for the entire ecosystem: javascript: broken by design.
But, one of the goals in software engineering right now is reproducible builds. This means building from source. And of course we'll want to automate that. We've already made inroads with CI
So, correct me if I'm wrong, these are still vulnerabilities.
Tragedy of the commons stuff.
This article might be honest. But I hope in the future we don't need devil's advocate arguments.
All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it.
I think it's totally worthwhile to figure out which tools rely on insecure dependencies.
Also looks like you can specify npm to ignore dev dependencies:
> Any packages in the tree that do not have a version field in their package.json file will be ignored. If any --omit options are specified (either via the --omit config, or one of the shorthands such as --production, --only=dev, and so on), then packages will be omitted from the submitted payload as appropriate.
Quite the opposite! Quoting the article:
As any security professional will tell you, development dependencies actually are an attack vector, and perhaps one of the most dangerous ones because it’s so hard to detect and the code runs with high trust assumptions. This is why the situation is so bad in particular: any real issue gets buried below dozens of non-issues that npm audit is training people and maintainers to ignore. It’s only a matter of time until this happens.
My point is that in the sea of non-issues, real issues are easy to miss and ignore.
>If you don't find this useful that is fine, don't use it.
You can't "not use it" because it's literally the default behavior built into `npm install` now. Of course there are ways to opt out, but this doesn't alleviate the confusion.
It seems like a simple algorithm that works pretty well. Perhaps ignoring certain dependencies makes sense, via an ignore list.
I just find the title "NPM is broken by design" to be a little hyperbolic, when it seems like the complaint is that it's tedious removing all the low-quality dependencies from your project. node security/npm-audit has at least increased the conversation around security for many around the npm ecosystem, where there wasn't much-if-any discussion prior. I think they deserve credit for this.
EDIT: I'm not sure why I'm being downvoted.
So what? Your job (if this was your customer) would be to figure out how to make them happier. Maybe you are getting down-voted because (while you're correct on some fine points), you're broadly dismissing the concerns of the article. Case in point: those "low-quality dependencies" aren't something you can easily switch out for quality parts... they're deep dependencies of many of npm's flagship tools and frameworks.
If we are in a situation where swapping out dependencies becomes so difficult that we just throw our hands up, is that really an issue with tooling?
EDIT:
BTW I'm not opposed to a PR that would allow for a .gitignore style list to ignore warning on specified deps. That could be useful. The issue I have would be respecting other peoples "auditignore" list or whatever... Because just because someone like @danabramov thinks including some package is not a threat, I may or may not agree with him.
A DoS on your build machine and dev machine can be indeed be critical issues. Imagine this scenario:
Your source code is somehow compromised and attackers slip in rogue code to your production site. It siphons off passwords or other PII. The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive, so you can’t just spin a fixed build and re-deploy. You’ll sync your main branch to figure out what is going on, perhaps ready to make a build from your dev machine, but running yarn build hangs. It might take you 1 minute to solve or 5 hours - hard to guess. But every minute you’re delayed is another minute the attacker is siphoning off your production data.
npm audit isn’t perfect, but I don’t agree with the author that devDependencies can’t have critical vulnerabilities. Build machines and dev machines are critical infrastructure. Recall the method of attack of SolarWinds [1].
Related: we all trust that the “many eyes” of open source contributors will keep our dependencies relatively clean, but this function is not infinite. There is some threshold of lines of code and rate of change that will outstrip the community’s natural ability to find and fix problems. I wish the npm community was more sensitive to the risks that are inherent in current practices. Efforts to limit dependencies and perhaps somehow tag which versions have completed a security audit (and by whom) would be great to see.
[1] https://krebsonsecurity.com/2021/01/solarwinds-what-hit-us-c...
Really at this point it's too late to do anything else, instead of trying to dos your dev machine he can instead do simpler things like delete your ssh key from the machine. But let's play along:
> The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive
There is nothing any attacker can do with the static files on the server that will trigger and RegEx DoS in your local development. Aside from the fact that you wouldn't download whatever is on the server back to your machine, even if you did it would never trigger such a DoS since (in the examples in the link) these are modules related to running a dev version of a frontend project based on the raw source files.
Your scenario is only true when an attacker pwned both your production server and your laptop. A regex DoS is really the last thing you worry about at that stage.
> There is nothing any attacker can do with the static files on the server that will trigger a RegEx DoS...
IIUC, an attacker could change my package.json to include inputs to browserlist that trigger a RegEx DoS. To do that, the attacker only needs to make a fraudulent commit. Given how easy most teams make it to commit code, this isn’t too high of a bar.
We've seen packages published through compromised dev keys or the maintainer granted a new bad actor rights to publish. Here we could envision a situation where someone reports a legitimate, but low impact, security issue and also publishes a bogus package that everyone is now upgrading to. On the whole, these messages aren't encouraging people to upgrade to a new, verified release (i.e., a fixed version). They're encouraging people to upgrade to the latest release. Even if you suspend the bad actor thought experiment, the latest release of a package in all likelihood hasn't been audited beyond ensuring the previously reported security issue has been addressed. Upgrading indiscriminately is a risky activity as well. I don't think we should be encouraging people to do that (in any language ecosystem).
It's not lack of imagination of attack vectors at play here. Treating extremely low risk factors as if they're high priority reduces trust in the system as a whole. Getting developers to care more about security is a laudable effort, but I think the `npm audit` approach (as reported in the post) is going to encourage bad practices.
Are we shooting the messenger (npm audit) here? Seems like the problem is lack of trust and lack of information about security of dependencies. Npm audit is just pointing that out. The size of the problem makes it very uncomfortable.
What could have been a useful tool for CI is not going to be enabled because who wants their build failing for a week while trying to orchestrate a mass dependency upgrade? I just took a look a Create React App (CRA)-based I have and "yarn audit" indicates 2,288 packages were scanned with 337 vulnerabilities (that's pretty good, I've seen it list over 500 vulnerabilities in the past). This is an app using CRA, with Relay as a code generating GraphQL client, and Antd as a component framework. There's also supporting tooling such as TypeScript, ESLint, and Prettier. Beyond that, very few dependencies are being used. Almost none of them are actually shipping in the product, whether that be due to them being build-related tools or because tree-shaking just strips most of the library out. It's a SPA hosted in CloudFront, so even DoS attacks have a very different profile than products where the server is rendering views. In this particular case, there's no shared user-generated content and XSS mitigations are in place, so the best a customer could do is DoS themselves.
I try to be very diligent about auditing dependencies before upgrading, but I can't reasonably extend that to transitive dependencies. 2,288 packages is an absurd number and this app isn't even doing all that much. I believe it's quite likely that it'd be impossible to get the npm/yarn audit report down to 0, given how quickly the JS ecosystem moves -- upgrade one dependency, deal with whatever API changes are needed, and now another has a vulnerability of some sort listed. It's certainly a burden on a small development team.
Just to be clear, I'm not advocating sticking your head in the sand and pretend everything's fine. I'm saying when you have vulnerabilities listed as high severity and you've verified they're not problematic in a project, then you've addressed the issue. But, now you have a tool that you need to satisfy and if you don't then you're going to miss the next security issue because CI shut off the "npm audit" check.
2. If you are doing a full build and fail because of the regex DOS, then that build would also contain the attacker injected siphoning code, which would make your entire exercise futile in the first place
3. Not obviously messing with the network or crashing build machines would be a better way of siphoning data for longer.
4. This is very contrived.
In the moment, your first thought is that there is some type of quick fix that will restore functionality (if your site is down) or evict the intruder if something funny is detected. As a sibling commenter said, most teams would try to deploy a previous known-good build asset.
But I stand by my point that a DoS of a development system can indeed be critical! I’m surprised to find that I appear to be in the minority here...
That seems to be the root of your confusion - it's not that development system DoS is not bad, it's that if you are at the point it's possible your security is already broken in much bigger way. It's like complaining that running "rm -rf /" under root would wipe all your files and that's a DoS - without taking into account that if somebody could run commands under root on your system it's not your system anymore. It's not that wiping all the files isn't bad - it's that the reason for why the situation is bad is much earlier that that.