Independently verifying Go's reproducible builds | Hacker News Reader

Independently verifying Go's reproducible builds

agwa.name

137 pointsspeckx16 comments

I started compiling the Go compiler myself over well over ten years ago when you had to compile it yourself to enable cross-compiling. That has not been the case in almost as long.

I have not stopped. I really should stop. At this point it's just kind of fun, but I have an unbroken chain of self-compiled Go compilers going back to the days when Go was written in C and not Go.

I am frankly really curious if my Go binary lives up to the reproducible build, or if some sort of Reflections on Trusting Trust type flaw worked itself in 10 years ago.

It’s great that these reproducible builds are possible and this is an incredibly thorough and careful validation. Thanks!

This is important work, and I thank you for it. These public transparency logs are important for keeping honest people honest, but also for keeping dishonest people out - If someone does manage to backdoor Google's build process, this is how they'll know.

Why is this important work to you? Reproducible builds to me is a complete waste of engineering resources and times that could be used elsewhere. All of this work goes towards protecting against theoretical attacks rather than practical ones that are actually happening in the wild.

Supply chain attacks are not theoretical! Just take a look at npm, docker and other repo lands.

Those attacks were not prevented by reproducible builds. Those supply chain attacks are the kind of things resources should be put into preventing.

They were completely preventable by independent verification. Just that without reproducible build you can't independently verify anything.

Maybe some of them were preventable, but if it was in place attackers would easily adapt to fool the automated systems and we would be back at status quo.

>without reproducible build you can't independently verify anything.

This is myth propagated by reproducible builds people. Byte for byte similarity is not required to detect a Trojan was injected into one.

You are right, I should not have said "you can't independently verify anything", but then you generally need to know what you are looking for.

Distributing software is a lot harder than just building it (with the caveat that people don't want to install build dependencies). So we rely on centralized distribution (and build). Because of this we have to assume trust of that entire chain.

When builds are reproducible they are independently verifiable which means you only have to trust the code and not the entire distribution chain (build systems, storage, etc).

Of course if no one bothers to verify then it doesn't matter. This is sort of how xz happened, no one verified that the release tarballs were what they were purported to be.

I know what reproducible builds are, but they do not solve practical problems. That are actively happening.

>This is sort of how xz happened

Reproducible builds wouldn't have caught this. You would reproduce the malicous library the same since the vulnerability is in the input.

Right, my point was that nobody bothered to check the source tarballs which should be completely reproducible already,

Wasn't the vulnerability triggered by a malicious script that was added silently to the tarball? Reproducible builds would have shown that the tarball is not the exact output of the build. Even though the malicious payload was already in the code, the trigger was not and was hidden

>Reproducible builds would have shown that the tarball is not the exact output of the build

That is not what reproducible builds do. Reproducible builds shows that the compiled binary comes from the inputs. You have to use the same inputs as the distro else it will most likely not match. The vulnerability is part of the input which means that anyone else reproducing the build would have a byte exact copy of the vulnerable library and no discrepancy would be found. Reproducible builds would monitor for when the builds don't match.

In this scenario you could compare release tarbells against the git repository, but that has nothing to do with reproducible builds.

If you do reproducible builds for only the binary of the program and not what's around it I don't know if it makes any sense. Related software like the installation script should be checked too against the source. Otherwise that would be like signing the binary but not the whole package.

In case of XZ, the source code was modified, in the install script and not in the binary itself. Checking against a reproducible tarball would have shown the package is not identical, as the trigger was put manually by the maintainer and not checked in the repo. If you had a "byte exact copy" of the repository, it would show immediately it's not the same used to build the package.

Otherwise, reproducible builds are useless if you only check for the binary and not the whole generated package, as XZ has shown, because the malicious code could be somewhere else than the binary.

Nix packages seem to be geared toward reproducible builds of the whole package and not just the binary. So it seems possible to do.

Repo of sourcespotter: https://github.com/SSLMate/sourcespotter

Reply on news.ycombinator.com