I've always felt that no physical controller access was a limiting factor for Apple killing the console.

The next (and IMHO inevitable) step, would be to open up the Apple TV via an SDK.