Smartphones have already stolen away the casual gamer. The people playing call of duty and final fantasy are not the people who will be wowed by the apple TV.
If you look at sales data for the Wii, there is a ton of people who bought a Wii, which game with Wii sport and then bought very few games. (https://en.wikipedia.org/wiki/List_of_best-selling_Wii_video...)
This will simply solidify that casual market into iPhone and Android apps. Consoles have nothing to fear IMO.
The old PS2 GTA titles had to be remastered for the iPhone and run fine on an iPhone 4.
I for one am sick of slow level loading and endless patches; every time I play on a console I want to throw something through the screen. And I've yet to see a single compelling PS4/XB1 title come out (I have high hopes for No Man's Sky).
For triple-A studios, the current-gen console hardware will likely always be a draw, though I'm sure this will get rereleases of old stuff, and possibly a downgraded version of new games (I suspect it isn't _that_ far from a Wii U, performance-wise).