I don't know why the argument that x86_64 has some affect on efficiency keeps coming up. All modern OoO chips microarch is decoupled from the instruction decoders. All the "strange" x86 instructions are effectively microcoded, which doesn't affect the execution performance of the instructions actually contained in high performance code. In many ways the ability to easily spill/operate against memory/stack provides instruction density advantages, and simplifies certain dependency calculations.
So, whether ARM64 has an power efficiency advantage isn't clear at all when one tries to actually compare them fairly. I frequently see people linearly scaling power/performance curves, or comparing cores that contain ECC, significantly more IO, etc against cores that don't and claiming that the core with 10x the IO bandwidth is somehow less efficient computationally because it consumes 5 watts more to power a PCIe bus.
So, lets get this out of the way, performance is not linear to power. Simply having a clock rate 50% faster has a large impact on power given the rough P=CV^2f equation because often the voltage goes up to support the higher frequencies. Worse designing for a target top end frequency of 4Ghz may entail extra pipeline stages/etc than one targeting 2Ghz in the same process. The result is significantly higher power draw at the same frequency due to the fact that higher frequencies are supported. Then there is leakage current/etc, which will be proportional to the number of transistors, so a design with 15MB of cache is going to waste more on leakage than one with 1MB. The extra 14MB of cache may only contribute another percent or two to the bottom line in many workloads, but frequently the difference between a processors at X performance and one at 1.5X is not due to a single factor but dozens of design tradeoffs that individually only net small percentage gains.
Bottom line, its better to compare implementations, and when ARM vs x86 chips are compared on similar grounds, they are a lot closer than you hear in glib remarks on forums like this. Intel's handicap in mobile (lack of native x86 android apps) and ARM's handicap in desktop/server all really come down to the software (and maybe in the case of some ARM products what one might consider alpha/immature products).