ARM designs have been optimized for low power. x86 designs have been optimized for high speed. It has little to do with the architecture and lots to do with the design.
Nobody has ever tried to design a sub 1 watt x86 design. Nobody has ever tried to design a 100 watt ARM.
Only very recently have we had anything that's close to comparable. Medfield has a similar power rating to high performance ARM designs, and similar performance.