> Of course instruction parallelism should be in the hands of the compiler -- it has so much richer information from the source-code so it can do a better job.

This is not true. Yes, the compiler has a lot of useful information, but it does not have access to runtime information, which the CPU does.

A modern superscalar CPU with out-of-order execution can execute instructions speculatively hundreds of cycles before they are even used.

Instructions after conditional branches can be executed speculatively using branch prediction and loads can be executed before stores (even though they might access the same address!). If the processor later detects that the load was dependent on the store, it can "undo" that operation, without any of these effects ever being visible to the running program.