Hacker News Reader

Comment by zamadatix | Hacker News Reader

Unless you are going to be the equivalent of a full time maintainer doing code review for every piece of software you use you need to trust other software maintainers reproducible builds or not. Considering this is Linux and not even Linus can deeply review every change in just the kernel anymore that philosophy can't apply to meaningfully large software like Nixos.

View on HN

That's too black-and-white. Being able to reproduce stuff makes some kind of attacks entirely uninteresting because malicious changes can be traced back. Which is what many types of attackers do not want. Debian, or the Linux kernel, for example, are not fool-proof, but both are in practice quite safe to work with.

Who are you going to trace it back to if not the maintainer anyways? If the delivery method then why is the delivery of the source from the maintainer inherently any safer?

No, it is not always the maintainer. Imagine you download a binary software package via HTTPS. In theory, the integrity of the download is protected by the server certificate. However, it is possible that certificates get hacked, get stolen, or that nation states force CAs to give out back doors. In that case, your download could have been changed on the fly with arbitrary alterations. Reproducible builds make it possible to detect such changes.

Same as when you download the source instead of the binary and see it reproducibly builds the backdoored binary. And at this point we're back to "Build from source. This will always be a deeply niche thing to do. It's slow, inconvenient, and inaccessible except to nerds." anyways.

It's not that reproducible builds provide 0 value it's that they don't truly solve the trust problem as initially stated. They also have non-security value to boot which is often understated compared to the security value IMO.

I guess reproducible builds solve some of the problems in the same way TLS/SSL solves some of the problems.

Most of the world is happy enough with the soft guarantee of: “This is _probably_ your bank’s real website. Unless a nation state is misusing their control over state owned certificate authorities, or GlobalSign or LetsEncrypt or whoever has been p0wned.”

Expecting binary black and white solutions to trust problems isn’t all that useful, in my opinion. Often providing 50% more “value” in trust compared to the status quo is extremely valuable in the bigger picture.

Reproducible builds solve many security problems for sure but but the problems it solves in no way help you if the maintainer is not alturistic or bad at security as originally stated. It helps tell you if the maintainers toolchain wasn't compromised and it does it AFTER the payload is delivered and you built your own payload not made by the maintainer anyways. It doesn't even tell you the transport/hosting wasn't compromised unless you can somehow get a copy of the source used to compile not made by the maintainer directly as the transport/hosting for the source they maintain could be as well.

Solving that singular attack vector in the delivery chain does nothing for solving the need to trust the altruism and self interest of maintainers. A good thing™? Absolutely, along with the other non security benefits, but has nothing to do with needing to trust maintainers or be in the niche that reviews source code when automatic updates come along as originally sold.

There are other solutions to the problem of trusting maintainers; namely incremental distributed code review. The Rust folks are working on that:

https://github.com/crev-dev/

> but the problems it solves in no way help you if the maintainer is not alturistic or bad at security as originally stated.

That same edgewise applies to your bank too. Pinned TLS certs or pre shared keys might help against "BadGuys(tm)", but you're still screwed if your bank decides to keep your money. (s/bank/online crypto wallet/ for real world examples there...)

The question isn't whether they're perfect, nor is it whether they prevent anything. But it does help a person who suspects something is up rule certain things in and out, which increases the chances that the weak link can be found and eliminated.

If you have a fair suspicion that something is up and you discover that when you compile reproduceable-package you get a different output than when you download a prebuilt reproduceable-package, you've now got something to work with.

Your observation that they don't truly solve the trust problem is true. But it's somehow not relevant. It is better to be better off.

> and inaccessible except to nerds.

So was most every part of computer hardware and software initially - this is just another milestone in that journey.

Reproducible builds still help a lot with security. For example, they let you shift build latency around.

Eg suppose you have a software package X, available both as a binary and in source.

With reproducible builds, you can start distributing the binary to your fleet of computers, while at the same time you are kicking off the build process yourself.

If the result of your own build is the same as the binary you got, you can give the command to start using it. (Otherwise, you quarantine the downloaded binary, and ring some alarm bells.)

Similarly, you can re-build some random sample of packages locally, just to double-check, and report.

If most debian users were to do something like that, any tempering with the debian repositories would be quickly detected.

(Having a few malicious users wouldn't hurt this strategy much, they can only insert noise in the system, but not give you false answers that you trust.)

Even if the original attack happened upstream, if the upstreamed piece of software was pinned via git, then it'd be trivial to bisect the upstream project to find the culprit.

This is great if you are looking at attributing blame. Not so great if you are trying to prevent all the worlds computers getting owned....

I'd imagine that if I were looking at causing world wide chaos, I'd love nothing better than getting into the tool chain in a way that I could later on utilise on a wide spread basis.

At that point I would have achieved my aims and if that means I've burnt a few people along the way, so be it, I'm a bad guy, the damage has been done, the objective met.

You can't solve this problem without having a full history of code to inspect (unless you are decompiling), reproducibility is the first step and bootstrapability is the second step. Then we refine the toolchains and review processes to ensure high impact code is properly scrutinized.

What we can't do is throw our hands up and say anyone who compromises the toolchain deep enough is just allowed to win. It will happen at some point if we don't put the right barriers in place.

It's the first step of a long journey, but it is a step we should be taking.

https://github.com/fosslinux/live-bootstrap is another approach, bootstrapping from a tiny binary seed that you could hand-assemble and type in as hex. But it doesn't address the dependency on the underling OS being trustworthy.

There is stage0 by Jeremiah Orians that is designed to be able to bootstrap on hardware that can be built from transistors. Currently it mostly runs in a small VM process that is somewhat harder to subvert.

Reproducibility is what allows you to rely on other maintainers' reviews. Without reproducibility, you can't be certain that what you're running has been audited at all.

It's true that no single person can audit their entire dependency tree. But many eyes make all bugs shallow.

No. I can review 0.1% of the code and verify that it compiles correctly and then let another 999 people review their own portion. It only takes one person to find a bit of malicious code, we don’t all need to review every single line.

That only works if you coordinate. With even more people, you can pick randomly and be relatively sure you've read it all, but I posit that 1) you don't pick randomly, you pick a part that is accessible or interesting to you (and therefore probably others) and 2) reading code locally is not sufficient to find bugs or backdoors in the whole.

The crev folks are working on a co-ordination system for incremental distributed code review:

https://github.com/crev-dev/

I actually wonder if it’s possible to write code at such a macro level as to obfuscate, say, a keylogger in a huge codebase such that reviewing just a single module/unit would not reveal that something bad is going on.

Depends on how complicated the project itself is. A simple structure with the bare minimum of side-effects (think, functional programming) would make this effort harder.

For something like C, all bets are off: http://www.underhanded-c.org/ or https://en.wikipedia.org/wiki/Underhanded_C_Contest

Crev is a great idea, unfortunately it is only really available for Rust right now.

I noticed there is a git-crev project, might that be useful for other languages? Also there is pip-crev for Python.

> It only takes one person to find a bit of malicious code, we don’t all need to review every single line.

This is just objectively wrong. I have worked on projects at FAANG where entire teams did not spot critical security issues during review.

You are very unlikely to spot an issue with just one pair of eyes. You need many if you want any hope of catching bugdoors.

You are misunderstanding what I am saying. I am saying that it only takes one person who finds a vulnerability to disclose it, to a first approximation. Realistically it’s probably closer to 2-3 since the first might be working for the NSA, the CCP, etc. I am making no arguments about what amount of effort it takes to find a vulnerability, just talking about how not every single user of a piece of code needs to verify it.

Reply on news.ycombinator.com