I agree wholeheartedly, the only downside I would see is that volume cannot be controlled through the iPhone app if you're controlling the volume through the TV. When I have a bluetooth speaker paired to apple TV, i'm able to control that volume using the rocker switches on my phone, it's the weirdest thing.
It feels like magic to have my TV turn on by a touch of my phone, and to also be able to control the TV when the kids think they are sneaky by not giving me back the remote.