Modern C compilers have profile guided optimization for that. You run your code, profile it, and the optimizer uses that data in the next pass. So this is something you can get without a JIT compiler.
Don't get me wrong, a lot of java bugs the hell out of me and I'm personally much more comfortable writing c++. I'm just pointing out that a code running on a virtual machine isn't some horrible back water.