There are numerous barriers to optimization that mostly fall into the categories of a) too many operations break continuity of typing and data flow, and b) many small, independent compilation units.
For example: Pointers, casts, unions, casting pointers to different types, and doing all of this across procedure call boundaries all challenge/break data flow analysis. You can pass a pointer to a long into a procedure, and the compiler has no idea what other compilation unit you are linking to satisfy the external call. You could well be casting that pointer-to-long to something totally crazy. Or not even looking at it.
I was associated with a compiler group experimenting whole-program optimization -- doing optimization after a linking phase. This has huge benefits because now you can chase dataflow across procedure calls, which enables more aggressive procedure inlining, which enables constant folding across procedure calls on a per-call-site basis. You might also enable loop-jamming across procedure calls, etc. C really is an unholy mess as far as trying to implement modern optimizations. The compiler simply can't view enough of the program at once in order to propagate analysis sufficiently.