I've always agreed in principal, but fell short in practice until I started using Nix[1]. Nix makes it significantly easier to pull off by verifying the hashes of all inputs, eliminating non-determinism in builds (timestamps, usernames, etc), eliminating dependencies on OS packages/files/etc, and allowing you to pin to a specific snapshot of all packages. Pardon the evangelism, I'm just a happy user.