Comment by ajross | Hacker News Reader

Comment by ajross | Hacker News Reader

Parent Full thread

IMHO those are both lipstick on a pig solutions. Ultimately all this stuff is just a variation of "make releases harder to publish", which isn't going to do anything but train people to evade them. Notably, neither would have prevented the xz-utils backdoor from reaching package distribution, which remains the gold standard for sophisticated upstream compromise.

The bug here isn't that we need to better authenticate already-trusted upstreams for packages, it's that the upstreams cannot be trusted as the sole source for security at all. Upstreams are a bunch of hackers[1] who aren't really interested in, nor will ever be good at, solid release engineering practices.

But some people are! The solution in the Linux world (and the one that saved us from xz-utils) is that there is a second level of human beings responsible for reviewing, auditing, packaging, and customizing those hacker-generated upstreams for the benefit of their users. These people have different eyes, different consumer requirements and different quality metrics. And they catch bugs and malfesance that the upstreams aren't prepared to do.

NPM (and cargo/PyPI et. al.) continues to think it can short circuit this requirement for human labor. It can't.

[1] In NPM's particular ecosystem, a bunch of web jockeys used to extremely fast release processes, loose compatibility requirements, and extreme reliance on reuse. This really explains why we see this with node packages more than Python or Rust: older and more conservative programmers just don't have as many rakes to step on.

> The solution in the Linux world ... is that there is a second level of human beings...

AKA "unpaid labor". I don't think that's a good solution, either. Certainly it's only by pure luck that no malefactors have infiltrated the ad hoc, anonymous social proof communities that Linux depends on, and I don't think other systems should emulate it.

The real solution (for Linux too) is a paid package curation service. Or really, a small handful of them competing on price, speed, reliability.

There is a version of that. It is called RedHat Enterprise Linux. : )

> Certainly it's only by pure luck that no malefactors have infiltrated the [pinko commie Linux hippy commune]

Yeah... no. Sorry, that's a wild misunderstanding of the economics of the Linux ecosystem, modern libertarian thought and the employment status of people with write access to the packaging layers.

> ... a second level of human beings responsible for reviewing, auditing, packaging, and customizing those hacker-generated upstreams for the benefit of their users.

> The real solution (for Linux too) is a paid package curation service. Or really, a small handful of them competing on price, speed, reliability.

That was also what I was thinking aloud a moment ago. And there would be a business opportunity, too. Perhaps not like RHEL et al. full-blown stuff per se, but say smaller scale guarantees with different pricing; web, AI, scientific computing, and whatnot. At the pace things are progressing, I'd guess you might even get desktop etc. users on board (for nominal pricing).

> Notably, neither would have prevented the xz-utils backdoor from reaching package distribution, which remains the gold standard for sophisticated upstream compromise.

Mandating that the final binary is compiled without having any access to any test file though would have prevented the xz-utils backdoor as it was conceived though.

A proper packaging setup would first verify that all the tests are passing and happen in an isolated environment. And that isolated environment either returns which tests failed or gives the green light.

When the greenlight is given (that all tests are passing), another environment should first delete all files related to tests and then build (in a bit-for-bit reproducible way btw and we're basically here already so that's good) the final binary / package.

If you prepare your final package in an environment that has access to test files, there are simply way too many ways obfuscated binary data can be hidden in test cases / test files.

I'm not saying the NSA (sorry, Jia Tan) wouldn't have tried something else but I think we should really move to build/packaging that discards non essential data/files before compiling.

P.S: note that as a side-effect of reproducible builds... If we have reproducible builds and if we add, later on, a builder/packager that discards tests files and ends up with a final package that's not bit-for-bit identical to the package created while having access to the test files during the build, we've just detected a backdoor hidden inside test files (like the XZ utils one). As a really mindboggling food for thoughts: if we were to recompile all the Debian binary packages that are already reproducible today (95% of them), but while discarding all tests files before the build, we may catch other backdoors.

> Mandating that the final binary is compiled without having any access to any test file

It would, but I'm not seeing that as a suggestion? That was a very clever side channel for hiding the build-time payload. It wasn't remotely the "root cause" of the exploit, which was that a malicious actor got write access to the release process of trusted software. I mean, if you can do that, you can surely find other clever ways to hide your junk.

To wit: you're not wrong, you're just stuck on minutiae. By all means make the case, but at best you're proposing a small constant factor optimization.

But there is a second level of people reviewing packages on npm. They're the ones that report issues like the github issue this HN thread is linked to, and they very frequently get malicious npm packages taken down within a day of publishing. The big issue is just that not everyone is using a cooldown to avoid packages less than a day old and so people who install new packages at unlucky times don't get the benefit of that layer of review.

I don't understand why you're confident that those Github issues won't just end up coming later if literally everyone adds this cooldown

The security companies looking for and reporting the issues aren't going to use the cooldown too.

They make their money from getting paid by other companies though, don't they? It's hard for me not to imagine that companies in general will see "testing stuff a day later" as a way to cut costs or not paying out as much for bounties because of claims that no one was actually affected yet

> they very frequently get malicious npm packages taken down within a day of publishing

If I'm reading the secondarily-linked blog post correctly, this was live for 12 days before discovery.

No, the reference to May 19 in the article is about a previous supply chain attack against AntV (https://www.stepsecurity.io/blog/shai-hulud-here-we-go-again...). I think there may be some copy-paste mistakes where they reused part of that previous article and didn't contextualize it correctly.

The npm package `@redhat-cloud-services/chrome` version 2.3.1, which was part of this current supply chain attack, was published on June 1. The malicious package version is no longer listed on npmjs.com's web UI since it was taken down, but the publish date of 2.3.1 can still be seen in https://registry.npmjs.org/@redhat-cloud-services/chrome by searching for the version number there, and the publish date was 2026-06-01T10:54:42.121Z.

I find the article extremely annoying for not having a clear timeline of when these malicious package versions were available.

Reply on news.ycombinator.com