A quick remark: general purpose compilers (including all the common ones) aren't really built for "extreme" performance. They always pick big tradeoffs and don't try everything they could.
In other words, it is not "hard" to build compilers with better performance. What is hard is getting performance without build times exploding.