You can build the source locally, then compare the MD5 hash value of your build to (1) the hash value they post publicly for their build and (2) the actual hash value of their build once you download it.
Assuming all three match, you know that the binary matches the source.
Someone who is more technically inclined can probably go into more detail on this.