I realized this myself when I wrote my first JIT compiler. The x86 code generated from bytecode was frightingingly naive. With basic code inlining and redundant instruction elimination my scripting language ran at 95% the speed of the equivalent c program with -O3 optimizations. And the assembly code generated by gcc was at least 3 times shorter.

Processors do a damn good job at optimizing code, and cache locality is more important than you'd think. The CPU blazes through redundant mov instructions like there's no tomorrow.

The good news is that building a fully functioning native code compiler is doable by a single person.