Wait, are we really that close to the maximum of what a compiler can optimize that we're getting barely 1% performance improvements per year with new versions?
However, for sure, if compiler optimizations disappeared, HW would pick up the slack in a few years.
Current compilers could do a lot better with vectorization, but it will often be limited by the data structure layout.
There are some transformations that compilers are really bad at. Rearranging data structures, switching out algorithms for equivalent ones with better big-O complexity, generating & using lookup tables, bit-packing things, using caches, hash tables and bloom filters for time/memory trade offs, etc.
The spec doesn't prevent such optimizations, but current compilers aren't smart enough to find them.
This would just be an extension of that. If the code creates and uses a linked list, yet the list is 1M items long and being accessed entirely by index, branch to a different version of the code which uses an array, etc.