Game consoles don't ship with TV remotes, and the remotes that you can buy are overly complicated affairs.
The WAF of using a controller to control video playback is low. We have a Roku 3 and it's far simpler to use than our 360 or PS4. Plus it's never "off". Side benefit is that I can remote play the PS4 when someone is using the Roku ;-).
The Switch could plausibly fill this gap, but I guarantee that those little controller thingies are going to get lost.