Ditto. Also unlocking your phone. Rewinding/Fast-forwarding on your phone. It's all so slow. And for me at least, it feels like there's a 50/50 chance if I seek in a video it'll just break the stream, I'll have to close the Youtube app entirely, and try again.
On the other hand when you're interested in something specific. Car reviews for example. The TV-Queue is really sweet.
But I just cast from Youtube to my Sony Bravia or Roku's. I'm not sure if that's Google Cast or not. I just discovered it by accident last week.