NPM's release cooldown is security theater | Hacker News Reader

NPM's release cooldown is security theater

blog.outv.im

44 pointsoutloudvi75 comments

> If everyone sets a cooldown, there will be no early adopter. Everyone is waiting for everyone else to be the canary, and the canary does not exist.

Except there are researchers chomping at the bit to download every new release of packages in search of vulnerabilities they can exchange for fiat, reputational credit, or both.

100% agree. I hated this post because it's all based on this unfounded assumption as an axiom, which it presents using zero evidence.

Different groups have different risks tolerances, and it's absurd to think that nobody is going to pull releases before the default cool down period.

> it's absurd to think that nobody is going to pull releases before the default cool down period.

Why is that absurd? It seems like a more likely outcome than just assuming there are enough "security researchers" out there doing this for every package?! IMO both sides here seem to have no evidence that the other is wrong.

Because security companies/researches are doing just that. The vast majority of recent supply chain attacks have only lasted minutes to hours. A large portion of them were credential losses or tag takeovers that aren't hard to detect, but by the time you do, your CI/CD has already downloaded and ran the attack 200 times.

Actually, you make an excellent point. Account takeovers have been responsible for a number of recent high profile attacks. Even if there are 0 security researchers reviewing new releases, just having time for the real owner to announce that a release was fraudulent before it's downloaded by half the Internet is a huge boon.

This entire article seems to say that if it doesn't prevent 100% of all possible attacks that it's "security theater". That's nonsense. It's just one more tool in the tool belt, and a highly useful one at that.

> The vast majority of recent supply chain attacks have only lasted minutes to hours

More claims without evidence! And how are you even going to defend that claim when it’s absolutely possible that lots of attacks are not even detected?!

Because I can always bump up the cooldown period. I'm not naive in thinking that I never have to worry about anything with a cooldown period, but having rolling releases is inherently what cooldowns give you. Companies have been doing rolling releases on their own products for years because it's a lot easier to handle when something falls over for a small percentage of users that when in nukes everyone all at once. That same concept still applies here.

It's worded a bit strongly, but it encourages critical thinking about downloading dependencies, how vetting should work, and where the human factor comes in. I think the top comment was right that some security researchers will audit packages no matter what, but it's good to not take that for granted.

I do appreciate these security companies a lot (for example, Snyk), but I feel it hard to believe this is sustainable. Especially in the current world where LLM is devaluing security researchers' work. If they cannot get enough fiat or credit, this will eventually turn to some incident like OpenSSL heartbleed or so.

It's bold to call something that has prevented real-world attacks "security theater", with the justification that one day the world might change enough that it's less likely to prevent future attacks. Was the printing press worthless, just because we have laser printers nowadays?

Anyhow, If you believe LLMs will kill security research, then surely you must also believe a cooldown of 1 week would be enough time for your favorite LLM to analyze a new package version?

I think it's great if people actually use LLM for the analysis. I did mention it in the solution part in the post:

> Run LLM-assisted audit on vendored code.

What do you call the time period between "new package is released" and "automated security scanners have analyzed the package"? That sure sounds like a release cooldown to me.

I applaud you if you do setup automated security scanners, without counting on external security groups or individuals (that doesn't have a security contract with your company).

This post is based on an assumption from what I see (I would be very happy if it's wrong) that most companies do not event bother to do these scans. They are merely waiting for the free kindness.

> LLM is devaluing security researchers' work

I'm confused by this take. In general, LLMs don't replace the expertise required to use an LLM.

Knowing the right input to give the LLM is not enough, nor does it ever end. The LLMs themselves need to be continuously updated and/or specialized for use case, and the "right input" is a moving target.

Keeping both ends of this aligned is the exact same expertise we already had before LLMs. These implementation details would only impact the credibility of those researchers who don't adapt to using this new tool. Who would stumble over something as trivial as this? LLMs aren't even the most important tool in the toolbox anyway.

I agree that LLM does not (and believe it never) fully replace researchers, but it produces artifacts (e.g. writeups, PoCs) at a cheaper price.

That makes me think LLM impacts human researchers' rewards, but now I realized the result might actually go in the opposite direction according to Jevons paradox. People still need experienced and professional human security researchers after all.

Artifacts have always been delegated away by the expert when possible because expertise is more about "planning" and less about "doing".

I'm glad you brought up Jevons Paradox, because it's for this reason I believe it's an extremely poor description of what's happening with AI.

The broader topic people should be discussing is not Jevons Paradox, but marginal utility.

More artifacts have diminishing utility.

https://en.wikipedia.org/wiki/Marginal_utility

LLM is only devaluing security work by competition. As in, it's making dependency scanning so easy and cheap that researchers don't have much left to do on that front. The end result is very easy and cheap dependency scanning available to everyone. It makes the next heartbleed less likely, not more (though actually the class of bugs it protects against is closer to log4shell).

I would honestly hope that the LLM usage would help with that very thing. If AI systems automatically perform security audits on brand-new NPM releases, doesn't that accomplish a similar purpose?

That's better than nothing, but if you're expecting people to be looking at something before you use it, you should be waiting for a thumbs up from them, not waiting some arbitrary timespan for the absence of a thumbs down.

You should probably also be paying them directly.

I don’t think it makes a ton of sense for individual OSS maintainers to pay companies for security. The point of cooldowns is that they’re free and they trend towards security at no logistical cost to maintainers.

(I agree that companies should pay for security products, of course. The bigger problem there is that so many security products are terrible, and supply chain products appear to be no different as a class.)

It's a game of probabilities. Waiting longer means a higher probability that someone found something before you upgrade.

Most of the attacks we have seen lately are from legitimate projects which got compromised. The maintainers of those projects are generally looking out for that.

*champing at the bit

So I went down the rabbit hole of the meaning of "champ", thinking it might mean something esoteric in the context of "champing at the bit" like "speaking the first line of the skit". But no, "champ" as a verb is just an alternative version of "chomp", or at least a word that differs very subtly from "chomp".

https://www.merriam-webster.com/dictionary/champing%20at%20t...

https://www.merriam-webster.com/dictionary/champ

The migration of "champing at the bit" to "chomping at the bit" is inevitable.

There don't seem to be many idioms like this, where the newer version is both semantically intuitive/meaningful and accepted by authorities. The closest is probably "hone in on", but the verb form of "home" as in "homing device" is still current, unlike the verb form of "champ". Another interesting one is the migration from "just deserts", using an archaic meaning of "desert" as in something deserved, to the modern form "just desserts". However there's much greater distance between "desserts" and the archaic meaning of "deserts" than between "chomp" and "champ".

The word champ comes from the world of horses actually

https://www.npr.org/sections/memmos/2016/06/09/605796769/che...

This is ... not really true outside of some very big/popular libraries. Often it takes one person to be bit before you can find the snake's nest.

The proliferation of these posts with such obviously terrible argumentation almost has me believing that they're a psy-op from some malicious actor (such as a US agency) which doesn't want people to improve their security. I mean why would you post drivel like this if you don't benefit from the status quo.

The most likely explanation is obviously that it's just a bunch of attention seeking amateur. But damn.

I don't agree with this take, there are a bunch of security companies that are screening new versions of packages looking for vulnerabilities, so waiting a bit before upgrading to the lastest package is miles better than immediately upgrading.

A decent amount of recent issues follow the pattern where someone's credentials are compromised, those compromised credentials are used to release a malicious package, and then the original maintainer is the one to notice it but it takes them a little while to get the bad package pulled. A cooldown pretty much completely solves that problem.

Beyond that, a lot of these cooldown criticisms assume everyone will use the same length, which isn't how it has to work. There are people who run Arch because they want the latest version of everything, and there are people who run Debian because they never want a new version to break their system. Some people camp out for the newest iPhone on day 1, and some people wait to see reviews and find out if the antenna is sensitive to how the phone is held. Adjustable cooldowns let everyone choose their own balance between getting the latest features and getting something that is tested.

> If everyone sets a cooldown, there will be no early adopter.

This ignores all the security companies & researchers screening packages.

Also ignores the common vector of the maintainer getting compromised and their credentials immediately being used to push malware.

Most maintainer are going to notice that themselves.

Exactly, and the vast majority of the attack success is in a very short period of time.

It will make credential attacks much harder to pull off successfully and the returns from using them will drop pretty dramatically.

Isn't this what alpha/beta/RC releases are for, ensuring that there are no issues before a general release? I just don't get ecosystem has decided to use client-side configurations for this rather than just marking the releases in the ways that already exist.

When the attacker gets your npm/gh credentials they just publish a new 'full' release making your idea worthless.

If it helps, think of cooldown periods as a way to do staged rollouts.

The archaic model where every tool gets full access to your home directory was never designed for the scenario where a single npm install will install and execute 100s of packages written by various authors.

One should sandbox all npm command executions.

https://github.com/ashishb/amazing-sandbox

I also believe sandboxing will get more and more important. There might be some trade-off on user experience or convenience, but given the security enhancement and (LLM agent's) freedom I think it will be well worth it.

Yes I highly suggest working in a container or setting up selinux. Selinux is especially powerful for this kind of thing but also requires a lot of advanced knowledge and skills to setup and maintain sadly.

Switch to another user, idealy called 'npm'.

        su -l npm

Then, just in case, run it under bubblewrap so it just can use $HOME and nothing more.

This is what I'm transitioning to. All third-party code runs in isolated containers: node/npm, python/pip.

How do posts like this make it to the top of HN? Nearly all the comments here are saying the same thing I thought:

1. Security researchers and companies are incentivized to install packages early.

2. Even without security patches, people and teams can still set the cooldown period that is appropriate to their risk tolerance. Even if someone in the wild does "get bit", it's better that a smaller number of people get bit than everyone getting bit at the same time. That's not "security theater", that's limiting the blast radius.

I'd also add that adding in cooldown periods lowers the incentives for bad guys to try to compromise a package in the first place because there is a much greater chance all their work will be for naught.

So I started reading this post with interest, thinking it would have some reason I hadn't thought of, but it just boils down to the faulty premise that "Godot is not coming". Bullshit, at the very least I now have agency over where I am in the download timeline. I just don't understand how such bad posts built on the flimsiest of assumptions make it to the top of HN.

It has a pessimistic title and just like redditors most HN voters don't read the articles they vote on.

I have no idea either. Every argument I hear about why wait times won't work are just factually wrong and using terrible assumptions they have zero insight into.

And, the complainers are free to set the wait timer to 0 and go back to the way it was. I'll be glad to let them be the first penguin to jump in the water.

I don't disagree with this take. But I'm not going to risk installing a package uploaded 1 hour ago considering all of the compromises this year. I think a cooldown period is prudent to let the automated security scanners to do their thing.

It also includes time to allow discovering of compromised accounts. If the owner of a popular package is hacked, this gives the owner a week to raise alarms about this.

Even setting aside that there are, in fact, many human and robot eyes staring hard at the firehose of published packages on popular registries, the whole argument feels like a false dichotomy.

If you're going to "DYOR" or whatever, having your automated dependency management on a cooldown timer seems like a snazzy way to smoothly provide a window for all your fancy dependency validation to happen.

Depending on those vaunted “security researchers” to take point and find all vulnerabilities in all packages is also fairly naive. That process costs money and produces a fairly valuable result - why would they give it away for free? So this will inevitably evolve into targeting only the high-value most popular packages and the exploiters will retreat to the long tail of less-popular ones. Lower payoff? Sure, but beats zero payoff.

Finding vulns in popular OSS and disclosing is probably good for your reputation as a security researcher, even if it's not immediately profitable.

How is that sustainable?

Are security researchers going to be doing free labor for you indefinitely for exposure?

They sell their products using the credentials they gained.

I’d never heard of socket until they found and reported shai hulud hiding in pytorch lightning. It pays off.

What happens after a couple of players have obtained marketshare and the market has consolidated?

I think there are diminishing returns to be had.

This is essentially Snyk's business model. Sure, they do upsells, but it seems fully sustainable to me.

The conventional meaning of “security theater” requires performance instead of effective action. But the author doesn’t reach that meaning: they claim that cooldowns don’t universalize, which then gets misconstrued into “security theater.”

I don’t agree with either claim: I don’t think they’re security theater, and I do think they universalize, contingent on there being security parties that are incentivized to scan packages.

The latter is a big assumption, but I think it’s a win for us either way: if supply chain companies are actually capable of defending us, then we win. But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.

Thanks for your explanation on the definition of "security theater"!

> But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.

If they cannot serve as the cooldown vanguard, we are indeed going to realize they are not trustable, but by the time the damage has already been done. Therefore, if companies want to prevent the damage as much as possible, I believe they should do the scan by themselves.

I don’t agree about the “damage being done”: there’s no evidence that a cooldown is itself harmful. The worst thing it can do to you is deliver some updates to you slower-than-instantly.

I agree cooldown is not harmful in general. What I intended to say is that people (mostly) only know a security company is not trustable because they once failed to detect a malware package, and the reason people know a package is malware is that it has caused damaged to at least someone.

I am so uncultured that I was wondering why on earth we were waiting for Godot... But it's of couse because it's THAT Godot and not the game engine.

What Godot is that? I'm only passingly familiar with the game engine.

The one who will come tomorrow. https://en.wikipedia.org/wiki/Waiting_for_Godot

Sorry! That's 100% on me failing to make the analogy understood.

Shall have thought about that... but I'm playing too many Unity games recently.

It's not your fault that I've become unclutured. If my wife had read it she would've never thought twice about which Godot it was. Kind of fun to think about how things change though. I don't think my children will read the play in school like I did, so perhaps the game engine being named after it (I assume it is) will mean they learn about it that way?

I wonder if the author has ever worked in IT. The idea of not moving quickly to new software versions has been around for decades.

There are always early adopters and researchers looking at new, publicly available software. Critical industries have long waited for software releases to be stable and "recommended" by the vendor before upgrading, letting those early adopters who can afford some availability risk take time with it.

The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.

I have seen this occur in some fashion - my company's product has a software version that's been out for almost six months and we still tell customers in critical industries to stay on the older version out of an abundance of caution and with the understanding of what upgrades require for them. Stability > features.

> The idea of not moving quickly to new software versions has been around for decades.

There are COBOL users and CentOS 6 users. They aren't affected by this issue.

They might need to face another set of issues like vulnerability backporting and so on. People not worrying about cooldowns might still need to patch their log4j libraries, but that's another thing. This post is not for them.

> There are always early adopters and researchers looking at new, publicly available software.

I agree with that. These hacks are very unlikely to go unnoticed, especially for popular packages (although they do happen; `chalk` has 3M downloads per week when it was hacked).

> The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.

Yes. However, I think this is eventually becoming some type of pointless arms race, as cooldown does not act actively towards reducing ecosystem attacks.

To add a somehow random example, pnpm's default cooldown time is 1 day. Yarn's 7 days. I'm not sure whether one day pnpm would also change it to 7 days because of people asking.

>think this is eventually becoming some type of pointless arms race, as cooldown does not act actively towards reducing ecosystem attacks.

The nature of some of these attacks, it does. If your developers machine with publishing rights gets rooted a cool down makes credential gathering attacks via script in npm so very much harder it's not even funny.

In some of the recent supply chain attacks the entire fiasco was only a few hours long, but in that time tens of hundreds of thousands of credentials may have been stolen.

One day timers have the weakness of weekends and holidays reducing the number of eyes/systems seeing the issue.

After reading the comments I now agree a short-length cooldown (maybe 1 or 3 days) is beneficial, given the following assumptions:

- Independent security companies are scanning the packages (be careful if the project is depending on some no-so-popular packages)

- Maintainers react promptly (it would be nonsense if people need to cooldown for 21 days because people may have 21-day vacations)

We're so close to a full circle moment. Next up is "npm stable", a curated set of packages (a "distribution", say) known to work well that are supported for a couple of years with security patches.

Atleast now they have a wide variety of Pixar movies to choose characters from

A curated list of stable packages won't accomplish anything. A compromised package can just as easily be published as a security update.

Unlike most commenters here, I actually agree with most of this criticism. But there is one simple thing the delay gets you, which is a window for the dev to go on Twitter and announce that they were hacked.

I'm pretty sure it isn't.

I actually proposed a cooldown for apt-get in my PhD thesis in 2013 and calculated its impact. Because the meta-data for the Ubuntu apt repository can be completely downloaded you can simulate the impact of a cooldown on how many less packages would be installed and it isn't zero.

There are always people in a community running beta version, or up to date versions. There will be less with a cool down, but then depending on how critical your system is you could adjust cooldown length

I do not use NPM. Works like a charm. Your mileage may vary

What a poor take. Cooldowns certainly don't solve everything but 'out of the box' even with no vetters they'll help catch unauthorised releases on active projects because it's more time to notice.

Account takeovers, deployment exploits whatever the root cause, given a few days it's incredibly likely maintainers will notice they've released an unexpected update and fix/sound an alarm. A lot of existing cases like this people realise pretty fast, but the packages might still be live for a few hours as NPM catches up. Cooldown entirely removes that 'updated at the wrong time' issue, an no-one even needs to actually look at the code at all.

For inactive or intentionally malicious maintainers, yes it relies on vetting. But it's hardly a mythical 'canary' - there are already a lot of companies scanning every popular npm package, auditing new ones, or just generally analysing threats. I can think of at least 4 such companies without even trying, I'm sure there's loads more.

And they'll continue because it's not warm hearted so much as promoting/testing their security scanning services and tools so they can get the kudos from being the first to spot X etc. Which often does hit the front of HN or reddit, but they don't need the massive cyber loudspeaker to be effective. Flagging the issue to maintainers, reporting to NPM security - with a cooldown in place the package can be voided before anyone really gets the chance to install it. This is literally how a lot of things are spotted already, just with that 'updated at the wrong time' issue catching unlucky people out.

Does it solve everything? No. Will it reduce the amount of problems that make it to end users? Obviously yes.

And for all the author's 'no one will do this', 'waiting for Godot' cries - what do they recommend? Running your own audits, using LLMs, static analysis and testing it yourself. Basically being the veter they claim no-one will be! Unless they don't intend to sound the alarm themselves, they are literally the canary they're looking for.

Cooldowns aren't magic, but they're not security theater either.

Went through the comment page and found this comment that explains well on most of the popular opinions. Thank you, hennell!

>If everyone

In no circumstances will everyone. This is just a bad take based on a silly premise.

I think the author is just totally ignorant, there's a whole community of security companies and researchers who are scanning every single package as it's released and have been doing so for some time now. Some of the details like how long the cooldown should be for are certainly up in the air, but the numbers have reasoning behind them.

I think this is such an nefariously unnecessary negative argument.

Most, if not all, of the shai-hulud attacks that hit npm and other ecosystems were preventable with cooldowns. And these were not detected because regular users reported the worms, but because security researchers did. I don’t think I’ve ever seen an attack that was discovered because a user reported it.

Reply on news.ycombinator.com