Interesting use case for IPFS, which has often felt like an (admittedly cool) solution in search of a problem.
Can someone enlighten me as to real-world examples where actually reproducible builds are critical?
Can someone enlighten me as to real-world examples where actually reproducible builds are critical?
One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT of compute power to do that.
With reproducible, content-addressed builds we can do things like early cutoff optimisations; making these changes less painful. SImple example; if somebody just changed a source code comment in glibc; then we get the same build artifact, and can skip building 60.000 packages.
If you do this at the source code level instead of the package level (like Bazel) then somebody can change how glibc does domain resolution; but packages that don't depend on that don't need to be recompiled either.
The example was given for linking but this can be done for many steps of the build process.
This is wonderful because it's totally sound. All efficiency methods (basically, hiding things, as the other comment says) is left to the build plan itself.
Switching to Nix for batch jobs is switching from DOS to an OS with actual process isolation. A complete paradigm shift.
Of course, since builds are reproducible you can catch this if you build it yourself and compare the actual content-hashes.
The caching model is about when the drv changes ((drv, output) is the cache key, basically), the trust model is one one can download something to avoiding build to fill in the value for the new cache key.
Note also that with the new floating content-addressed derivations, the trust stuff is much easier to think about.
Most people do not compile their own software, they used signed binary distributions being given to them. If you have reproducible builds, third parties can more easily verify independently that the code being distributed is the same as the code published in the open source repositories.
This doesn't just keep dev teams honest, it also provides defenses against situations like those where a hacker gains control of the website being used for distribution. I believe reproducible builds have helped to catch this real life issue at least once in the cryptocurrency community (with Monero).
To me, that's enough to justify their existence.
If you push something to CI that needs some new dependencies, the CI machine will build them and when you want to get a local dev environment or deploy, the binaries will be fetched from the CI machine. We can also send the binaries between machines to save compile time. This is possible because we have confidence that it doesn't matter which machine built them because the environment is so strictly controlled.
Reproducible builds (as in bit-for-bit) means I can trust a (secure) checksum of a binary I downloaded somewhere, let's say from IPFS. This is defense-in-depth, adding another layer of security to prevent sabotage.
Reproducibility means the whole system can reliably be built from source and I don't need to manually make sure the system has the correct constellation of implicit dependencies and their version numbers. This saves time and money in deployment and will pay off in a very big way 10 years down the line when that code is still in use somewhere.
NixOS doesn't solve reproducible builds as-in "bit-for-bit identical binary distributed files", but it does solve "same input" -> "functionally same system". This is the core idea of NixOS and what makes the package manager and the spawned ecosystem (like Guix) quite neat. I also think this isn't any more controversial then say having your infrastructure reproducible with terraform and ansible/salt. It gives you machines as cattle, and not pets, with the added flexibility that gives you. NixOS is just a complete package where in other systems there are several components accomplishing the goal.
As to why reproducible builds is important is to ensure a strong connection between the upstream source code and the distributed binary build. This gives you confidence that everything going into the build has been declared and that the binary can be reproduced bit-for-bit identical if you so wish.
This can be both important in a supply chain process. I recently saw tailscale provided pre-compiled binaries without signatures. What does those binaries contain? They are not signed, so the releases could very well be unauthenticated and someone could have compromised the server distributing the binaries. So I tried reproducing the tailscale binaries without much success as the build process is proprietary.
https://github.com/tailscale/tailscale/issues/779
So what does the tailscale binaries contain? I can't get a bit-for-bit identical binary without putting into a great deal of effort so for all intents and purposes they can very well be proprietary.
Now I don't think the tailscale people are malicious, nor that the binaries have been replaced. But it's a real-world example I recently went through.
When you have binary reproducible builds such a thing would immediately be obvious and hell would break loose if they attempted it.
See these:
And as an outsider I'm unable to read the RFC and understand what is going on and what tangible issues it solves for Nix.
Until that ideal is reached, you could call "leaving it to the plan writer" a cop-out, but what else is there to do? It's the stop-gap.
The benefit of the new system is that all store paths can be content-addressed. That means the inputs of all build stems can be content-addressed, whereas before they might also be "input-addressed" paths whose contents are also a matter of trust. This means one takes "small step" rather than "big step" trust steps, and also that upstream non-determinism won't "pollute" downstream builds.