Physical buttons. With my dedicated media device I can control it with the buttons and leave it in my pocket. When I play music on my phone I have to take it out and look at it to make sure I tap the right part of the screen.

Also a lower replacement price. Someone might want to steal my expensive smart phone, or it might break and make me sad. If I had a $20 dedicated media player, then there's nothing to worry about.