Debian have done a huge amount of work in this area - https://wiki.debian.org/ReproducibleBuilds
Their documentation has some really good pointers at common issues and how to solve them: https://reproducible-builds.org/docs/
And the diffoscope tool is very useful for debugging.
[0] I believe other projects had similar things going on but the Debian one became the main to serve as an umbrella.