I use Gentoo which uses portage the package manager and the way portage works is it pulls source then compiles. Source is rarely checked by everyone. Small packages exist as well. Many Linux distro simply barrow binaries from "trusted" sources. The entire eco system is really a deck of cards.