Every time I see the phrase "open source", I mentally add "but not open binary"; how do I know that the service that is run uses binaries that are compiled from that open source alone? with trusted libraries? After all, anyone can employ a site-specific patch in their build process that adds additional "features" to their normally open source project.

Is there a reasonable way to have trusted individuals intermittently audit the service?