I mean if 2 copies of a piece of software were compiled from the same source, what stops them from being identical each and every time?
I know there are so many moving parts, but I still can't understand how discrepancies can manifest themselves.
https://reproducible-builds.org/docs/
The main overall issue is that developers don't test to ensure they reproduce. Once it's part of the release tests it tends to stay reproducible.
It would be great if 100% of builds were reproducible, but I don't believe developers shouldn't be testing for reproducibility unless it's a defined goal.
As generalized reproducible build tooling (guix, nix, etc.) becomes more mainstream, I imagine we'll see more reproducible builds as adoption grows and reproducibility is no longer something developers have to "check for", but simply rely upon from their tooling.
We have the tooling, but it still takes a bit of effort from the developer's side to integrate those into their CI pipeline.
Eventually we will get to a place where this will be the default. It will be integrated into day-to-day tooling like `cargo release`, `npm publish`, ...
How do you get similar behaviour while having a reproducible build?
Can you, for example, have the final binary contain a reproducible part, and another section of the elf file for deliberately non-reproducible info?
Having information about which git has was used, as well as the time it was published, is part of the source distribution so an output can contain references to these inputs and still be deterministic w.r.t. those inputs.
If you REALLY want to know when/who built something, you could add in an auxiliary source file which contains that information, which is required to build. Which is essentially what compilers which leverage current time do anyway, it's just implicit.
The actual build date doesn't matter if the software is reproducible - but its a proxy for 'how out of date is this software'.
Conceivably there could be a standard for a sidecar file to specify how something was built (e.g. nixpkgs commit hash, or all of the parameters that went into the build). Or content address the inputs, i.e. invent Nix again.
So we could solve this problem by having everyone standardize on using Nix.
Regardless, whether or not you do that is a discussion of distribution format, not binary reproducibility. Your distribution can contain as much (or as little) additional material as you like along with your release binaries.
But including timestamp of build would necessitate “spoofing” the timestamp by the reproducer to be the same as the original.
Of course there would have been better designs for this toy example, but binary reproducibility is/was usually not of the highest priority historically in most compiler infrastructures, and in some cases it might be a relatively big performance regression to fix, or simply just a too big refactor.
Well no: that's really the thing reproducible packages are showing: there's only one correct binary.
And it's the one that's 100% reproducible.
I'd even say that that's the whole point: there's only one correct binary.
I'll die on the hill that if different binaries are "all correct", then none are: for me they're all useless if they're not reproducible.
And it looks like people working on entire .iso being fully bit-for-bit reproducible are willing to die on that hill too.
A binary consisting of foo.o and bar.o is correct whether foo.o was linked before bar.o or vice versa, provided that both foo.o and bar.o were compiled correctly.
This is used in theorem provers, for example, so that you don't have to verify proofs of theorems over and over again (which can be very slow).
Instead, you verify them once, save the state of the heap to disk (as a binary ELF, for instance) and then you can run the binary to continue exactly where you left off (i.e. with all the interesting theorems already in memory, in a proved state).
This is what the HOL4 theorem prover's main `hol` script does, i.e. it runs HOL4 by loading such a memory state from disk, with the core theories and theorems already loaded.
Presumably, to make this reproducible you'd need to make sure that all the memory objects are saved to disk in a deterministic order somehow (e.g. not in memory address order, as it can change from run to run, especially when using multiple threads).
Edit: Presumably you'd also need to make sure that you persist the heap when all threads are idle and in a known state (e.g. with all timers stopped), to avoid random stack states and extraneous temporary allocations from being persisted, which would also affect the resulting binary.
https://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20...
The Global Value Numbering pass in LLVM was iterating over `DenseMap<BasicBlock*, ...>`, so the iteration order was dependent on the value of BasicBlock pointers. This could lead to the same source files and compiler producing different binaries.
These comparisons don't have to go the same way for everything to be correct.
Imagine the program uses the current date or time as a value. When compiled at different moments, the bits change.
Same applies to anything where the build environment or timing influences the output binary
I’ve successfully built tools to compare Java JARs that required getting around two of those and other test tools that required the third. I’m sure there are more.