I don't think what I'm saying is controversial in the mobile industry. For example, here's an old article that has a quote about the effects of Intel's Quick Start technology from back in the year 2000 that captures what I'm saying:
http://www.anandtech.com/show/2782/2
“"Intel has figured out that it is best to use full CPU power for a split second to finish a task and then put the CPU to idle as this conserves battery life the best. Although one may suspect that when running complex operations the CPU would not have time to go idle, this is not the case. To illustrate this point, Intel used an example of DVD playback. Very stressful on the system as a whole, Intel's quick start technology allows the CPU to "hurry up" and perform the DVD decoding operations and then go idle until the frame is displayed to screen and the next scene needs to be calculated. This saves battery life because, although the system may require 3 watts or so to "hurry up", the power consumption goes down near .25 watts when idle. By averaging these two numbers, one can quickly see how quick start can extend battery life."”
The biggest demon is by FAR the radios. The amount apps and the phone use that is mostly determined 1> by how much the user wants notifications (which are basically implemented by constant polling, even in background notifications), or 2> how network intensive the apps are that a person is using. Additionally, stuff like having bad connections to your cell provider make the radio receiver power have to be turned up (as well as broadcast, but the receiver is the worst part usually), and constantly using bluetooth will kill you as well. The bad thing about the radios, especially if you have stuff like lots of apps doing push notifications, as well as a fast polling interval for your mail, is that you're doing this even when the phone is in your pocket.
Second to the radios is technically another radio, but people only do 1 way comms with it and don't really think of it as a radio: THE GPS SYSTEM. Apple vastly improved this (it used to pretty much leave the system on, now it pulses it on and off unless your app requires the older mode for some reason, such as Navigon type turn by turn navigation). Again, this is a part of the phone that optimization did help with (as often time the GPS radio doesn't come on if the phone can figure out your location from other mechanisms), but still, you're generally speaking going to see most GPS heavy apps running while the user is using them. Sure driving faster will get the phone in their pocket faster, but I'm not sure an added hour of battery life is worth unsafe driving.
I'm not saying graphics and the like don't eat up huge chunks of power, I'm just saying that a vast majority of people eat most of their power on much more pedestrian things than Infinity Blade.