It's probably a testament to the scope of the build + test that makes it four hours. If it were faster, would the scope tend to increase in order to hit more tiers/more tests?
I'd rather my commit sit in a test queue for several hours than push and cross my fingers like LLVM does it.