That's an interesting thought. I think the missing piece is really the Apple TV playing games, and for whatever reason (maybe storage?), they just can't or don't want to make it happen.
Personally, I think the hold-up is that Apple is working on some new method of interacting with a computer from across the room. [1] I can see them demanding some better interface for general software, before they put an app store on the Apple TV. At which point they're free to only barely support gaming (via the gamepad APIs) as is their tradition.
[1] A remote control is a terrible way to interact with arbitrary software. It's not even that good for modern video services that do little more than adding queue-management, search and account information management.
Gamepads are effective, but they're very gaming-specific and Apple is only accidentally a force in gaming; it's not in their DNA to release hardware that requires a gamepad.
Maybe it's something like Kinect. Maybe it's Siri. I don't have any good guesses there. I hope to hell it isn't AirPlay.
I just think the evidence is pretty clear that Apple simply doesn't value gaming enough to do a gaming-first product. They'd demand it do something more and how you'd do something more is a problem large enough to explain the delays (particularly given Jobs' autobiography, wherein he claimed that he'd finally cracked TV).