First, the whole phone needs to be well architected - the CPU, GPU, Flash, wireless and cell radios can each be bottlenecks for good performance.
As the screen size and refresh rate increase, the needs of the CPU/GPU and Flash will increase.
Mobile phones tend to be more interactive and thus have a larger negative experience with slow/jerky scrolling, slow response to taps including app launch, etc.
Microcontrollers tend to have options from slowing down the clock rate to going into a deep sleep mode when there is no scheduled work. There has been a lot of work to optimize the scheduling of network/driver/app tasks to minimize power draw. It is also fairly common to see a CPU have lower power cores, typically for the OS to leverage for hardware support and background tasks separate from what might be considered the 'main' CPUs for this reason.
Likewise, Apple has stopped shooting for a 50-100% increase in performance year over year, instead going for a mix of performance and power efficiency improvements. They have also started to do significantly more custom silicon design to support their hardware features, such as the Secure Enclave and neural engine additions used to support Face ID.
10 years is a long time though. Four years ago would be what, the iPhone 3GS? Apple went several years doubling the CPU, GPU, _and_ Flash performance and it was extremely noticeable.