Because I can't feel the buttons with my thumb and have to look down, unlock my phone, switch to the right app, and then tap. It's a horrible experience.
Your phone is fine until you want to pause the movie to answer the door, but it takes tens of seconds just to reconnect to the TV dongle.
Or when you want to pause said movie because you're getting a call, which prevents you from pausing the movie. Happens to me all the time while Chromecasting.
Yet another reason to avoid using phones for phone calls!