And what's often an even bigger deal is many of the low-to-mid end devices have very, very little cache (some have no L2 cache, even!).

I was curious about this claim so I looked some numbers up. The latest A11 chip has 8MiB of L2 cache. I could only find a small handful of xeon models with 6MiB of cache while almost all of them contain equal or greater amounts of L2 cache. The chip does not have L3 cache so there is nothing to compare here. The A10 has total package cache less than 8MiB (split into L2 and L3). So I'm not sure this is a good explanation of the performance or size difference.

From what I can tell, while cache is still a large portion of the area, GPU die space is quite large now. The CPU performance seems to be due to expanded execution ports leading to more dispatched instructions per cycle, which is generally a good idea as long as one can keep the execution ports fed with a good amount of speculation in other stages of the pipeline.

Not so humongous they can’t fit in one of the thinnest phones.

Pretty sure you meant ARM64 not x86 :)

But yeah, the SoC in the iPhone 5S was crazy fast at the time, plus they have other advantages like Secure Enclave, and a smartwatch with ACTUAL all day battery life. I owned a Sony phone with a Snapdragon 810, which was Qualcomm's first high-end 64-bit chip... and it overheated at the drop of a hat. The phone was more or less unusable in Australian summer. Everything I read at the time indicated that Apple really caught the rest of the industry off guard.

Even Wikipedia puts it pretty bluntly:

"The first 64-bit SoCs, the Snapdragon 808 and 810, were rushed to market using generic Cortex-A57 and Cortex-A53 cores and suffered from overheating problems and throttling, particularly the 810, which led Samsung to stop using Snapdragon for its Galaxy S6 flagship phone."