One thing I would like to see is speculative execution. I have an 8 core machine. When the JIT compiler runs there is no reason why it can't generate 7 alternatives to any particular code sequence, run them all in parallel and keep whichever was the fastest. Or run both sides of an if statement simultaneously.

This helps solve the problem that compiler writers often have multiple alternatives they can generate, but have to make hard decisions as to which to pick.

I'm not saying that implementing this will be easy, but then again all of the easy things have already been done.