What’s in a package | Hacker News Reader

What’s in a package

hpc.guix.info

106 pointsReventlov13 comments

This is a great article, that touches on issues like reproducible builds and software supply chain security. It also highlights the problem that some software can lack what we might call "reproducible source":

> In their ESEC/FSE 2021 paper on LastPyMile, Duc-Ly Vu et al. empirically show that “the last mile from source to package” on PyPI is indeed the weakest link in the software supply chain, and that actual differences between packaged source code and upstream source code are observed in the wild.

There was a clever attempt to solve this problem in the JavaScript/npm ecosystem a few years ago, but sadly it never became officially supported:

https://hackernoon.com/what-if-we-could-verify-npm-packages-...

> actual differences between packaged source code and upstream source code are observed in the wild.

Patches?

Patches that are applied to the code that is uploaded to a package repository, but which are not committed to the source repository.

Fascinating write-up!

Reproducible builds are highly important indeed, though from security perspective we shouldn't assume we're secure because we can reproduce the build.

As Ken Thompson shows us in Reflections on Trusting Trust: backdoors can live in binary form only and the source code may not be telling the full story.

What we need to trust are the entities writing and distributing the packages, of everything. Sadly today there is no way to answer: what is the set of entities I'm trusting by using this package.

You can trust community as the whole, because uses it own product. Community will protect itself, so you, as regular user, can benefit from that for free. No community - no protection, because the owner of a package is not a user of the product.

A person that had at some point control of a widely distributed toolchain binary (say a compiler, linker, or even a build tool) can trick an entire comnunity.

You're absolutely right! You can trust that a community, given sufficient time, will act to protect itself in the long-term and thus individual users. This just might not always be the same as every user being maximally safe at every point in time.

Case in point that gets at both: malicious python and npm packages stealing credentials. They were caught and handled, but not before hitting some people.

In Guix, every package is derived from a tiny set of "bootstrap binaries", that will soon be reduced to a ~512 byte machine code seed:

https://guix.gnu.org/blog/2020/guix-further-reduces-bootstra...

That means the famous "trusting trust" attack mostly does not apply. There are a few pre-compiled binaries lurking in Guix's dependency graphs, such as GHC, but they will be properly bootstrapped as soon as someone figures out how, see <https://www.bootstrappable.org/projects.html>.

Also important to note that no one is uploading packages to Guix. The CI system builds everything automatically, and you can opt out of "binary substitutes", compile everything locally, and still end up with (mostly) identical binaries.

Wow that Stage0 project is amazing, didn't realize anyone went that far, thanks for the reference!

A more direct link for other interested readers: https://bootstrapping.miraheze.org/wiki/Stage0

Another huge plus for source based package managers not elaborated in this article is how easy transformations are [1], so you can rebuild your software with different compilers, dependency versions, source branches, etc. from often a single command.

[1] https://guix.gnu.org/manual/en/html_node/Package-Transformat...

I didn't know what was guix so I clicked on the "About" page (https://hpc.guix.info/about/).

> Guix-HPC is an effort to optimize [GNU Guix] for reproducible scientific workflows in high-performance computing (HPC).

Ok this is like GNU Guix, but for HPC systems (which are helpfully defined). But what is "GNU Guix", though? Ok I will click the link (https://guix.gnu.org/) :

> *Liberating*. Guix is an advanced distribution of the [GNU operating system] developed by the GNU Project—which respects the freedom of computer users.

I happen to know what GNUOS is, otherwise I would have to click on yet another link to find out what it is (the name at least suggests it's an Operative System).

Guix needs extra help on the initial copy page though. Is "more advanced" than GNUOS... does that mean that GNUOS isn't "advanced" (How so)? It also says that Guix "respects the freedom of computer users". That seems to imply that GNUOS does not. Is that the case?

TL;DR: The "about" page in guix-hpc and guix should start with "Guix/Guix-hpc is an operative system." Then they can talk about how they're based on this and that. And Guix's first page copy text needs to be completely rewritten.

Guix isn't exactly an operating system though, it's a package manager, like apt on Debian.

I think the reason the Guix page calls it an "operating system" is for the same reason Stallman gets annoyed when people say they're using a "Linux operating system" instead of a "GNU/Linux operating system", since Linux is just one component, and not the actual "OS".

Not sure why this has to be so confusing. IMO, the homepage should just say it's a package manager, and note that they also offer a Linux distro based on it.

Guix isn't exactly an operating system though, it's a package manager, like apt on Debian.

The terminology went through some iterations, but according to the manual, Guix can nowadays refer to either: It's a "package management tool for and distribution of the GNU system" (emphasis mine). The distibution can be used standalone (in which case it's called Guix System), or as a package manager on top of a host system.

Reply on news.ycombinator.com