Hacker News Reader

Comment by paulryanrogers | Hacker News Reader

> If everyone sets a cooldown, there will be no early adopter. Everyone is waiting for everyone else to be the canary, and the canary does not exist.

Except there are researchers chomping at the bit to download every new release of packages in search of vulnerabilities they can exchange for fiat, reputational credit, or both.

View on HN

100% agree. I hated this post because it's all based on this unfounded assumption as an axiom, which it presents using zero evidence.

Different groups have different risks tolerances, and it's absurd to think that nobody is going to pull releases before the default cool down period.

> it's absurd to think that nobody is going to pull releases before the default cool down period.

Why is that absurd? It seems like a more likely outcome than just assuming there are enough "security researchers" out there doing this for every package?! IMO both sides here seem to have no evidence that the other is wrong.

Because I can always bump up the cooldown period. I'm not naive in thinking that I never have to worry about anything with a cooldown period, but having rolling releases is inherently what cooldowns give you. Companies have been doing rolling releases on their own products for years because it's a lot easier to handle when something falls over for a small percentage of users that when in nukes everyone all at once. That same concept still applies here.

Because security companies/researches are doing just that. The vast majority of recent supply chain attacks have only lasted minutes to hours. A large portion of them were credential losses or tag takeovers that aren't hard to detect, but by the time you do, your CI/CD has already downloaded and ran the attack 200 times.

Actually, you make an excellent point. Account takeovers have been responsible for a number of recent high profile attacks. Even if there are 0 security researchers reviewing new releases, just having time for the real owner to announce that a release was fraudulent before it's downloaded by half the Internet is a huge boon.

This entire article seems to say that if it doesn't prevent 100% of all possible attacks that it's "security theater". That's nonsense. It's just one more tool in the tool belt, and a highly useful one at that.

> The vast majority of recent supply chain attacks have only lasted minutes to hours

More claims without evidence! And how are you even going to defend that claim when it’s absolutely possible that lots of attacks are not even detected?!

It's worded a bit strongly, but it encourages critical thinking about downloading dependencies, how vetting should work, and where the human factor comes in. I think the top comment was right that some security researchers will audit packages no matter what, but it's good to not take that for granted.

I do appreciate these security companies a lot (for example, Snyk), but I feel it hard to believe this is sustainable. Especially in the current world where LLM is devaluing security researchers' work. If they cannot get enough fiat or credit, this will eventually turn to some incident like OpenSSL heartbleed or so.

It's bold to call something that has prevented real-world attacks "security theater", with the justification that one day the world might change enough that it's less likely to prevent future attacks. Was the printing press worthless, just because we have laser printers nowadays?

Anyhow, If you believe LLMs will kill security research, then surely you must also believe a cooldown of 1 week would be enough time for your favorite LLM to analyze a new package version?

I think it's great if people actually use LLM for the analysis. I did mention it in the solution part in the post:

> Run LLM-assisted audit on vendored code.

What do you call the time period between "new package is released" and "automated security scanners have analyzed the package"? That sure sounds like a release cooldown to me.

I applaud you if you do setup automated security scanners, without counting on external security groups or individuals (that doesn't have a security contract with your company).

This post is based on an assumption from what I see (I would be very happy if it's wrong) that most companies do not event bother to do these scans. They are merely waiting for the free kindness.

I would honestly hope that the LLM usage would help with that very thing. If AI systems automatically perform security audits on brand-new NPM releases, doesn't that accomplish a similar purpose?

LLM is only devaluing security work by competition. As in, it's making dependency scanning so easy and cheap that researchers don't have much left to do on that front. The end result is very easy and cheap dependency scanning available to everyone. It makes the next heartbleed less likely, not more (though actually the class of bugs it protects against is closer to log4shell).

> LLM is devaluing security researchers' work

I'm confused by this take. In general, LLMs don't replace the expertise required to use an LLM.

Knowing the right input to give the LLM is not enough, nor does it ever end. The LLMs themselves need to be continuously updated and/or specialized for use case, and the "right input" is a moving target.

Keeping both ends of this aligned is the exact same expertise we already had before LLMs. These implementation details would only impact the credibility of those researchers who don't adapt to using this new tool. Who would stumble over something as trivial as this? LLMs aren't even the most important tool in the toolbox anyway.

I agree that LLM does not (and believe it never) fully replace researchers, but it produces artifacts (e.g. writeups, PoCs) at a cheaper price.

That makes me think LLM impacts human researchers' rewards, but now I realized the result might actually go in the opposite direction according to Jevons paradox. People still need experienced and professional human security researchers after all.

Artifacts have always been delegated away by the expert when possible because expertise is more about "planning" and less about "doing".

I'm glad you brought up Jevons Paradox, because it's for this reason I believe it's an extremely poor description of what's happening with AI.

The broader topic people should be discussing is not Jevons Paradox, but marginal utility.

More artifacts have diminishing utility.

https://en.wikipedia.org/wiki/Marginal_utility

That's better than nothing, but if you're expecting people to be looking at something before you use it, you should be waiting for a thumbs up from them, not waiting some arbitrary timespan for the absence of a thumbs down.

You should probably also be paying them directly.

I don’t think it makes a ton of sense for individual OSS maintainers to pay companies for security. The point of cooldowns is that they’re free and they trend towards security at no logistical cost to maintainers.

(I agree that companies should pay for security products, of course. The bigger problem there is that so many security products are terrible, and supply chain products appear to be no different as a class.)

It's a game of probabilities. Waiting longer means a higher probability that someone found something before you upgrade.

Most of the attacks we have seen lately are from legitimate projects which got compromised. The maintainers of those projects are generally looking out for that.

*champing at the bit

So I went down the rabbit hole of the meaning of "champ", thinking it might mean something esoteric in the context of "champing at the bit" like "speaking the first line of the skit". But no, "champ" as a verb is just an alternative version of "chomp", or at least a word that differs very subtly from "chomp".

https://www.merriam-webster.com/dictionary/champing%20at%20t...

https://www.merriam-webster.com/dictionary/champ

The migration of "champing at the bit" to "chomping at the bit" is inevitable.

There don't seem to be many idioms like this, where the newer version is both semantically intuitive/meaningful and accepted by authorities. The closest is probably "hone in on", but the verb form of "home" as in "homing device" is still current, unlike the verb form of "champ". Another interesting one is the migration from "just deserts", using an archaic meaning of "desert" as in something deserved, to the modern form "just desserts". However there's much greater distance between "desserts" and the archaic meaning of "deserts" than between "chomp" and "champ".

The word champ comes from the world of horses actually

https://www.npr.org/sections/memmos/2016/06/09/605796769/che...

This is ... not really true outside of some very big/popular libraries. Often it takes one person to be bit before you can find the snake's nest.

The proliferation of these posts with such obviously terrible argumentation almost has me believing that they're a psy-op from some malicious actor (such as a US agency) which doesn't want people to improve their security. I mean why would you post drivel like this if you don't benefit from the status quo.

The most likely explanation is obviously that it's just a bunch of attention seeking amateur. But damn.

Reply on news.ycombinator.com