My Youtube connection has been strange for some weeks now. I'm in Brazil accessing Internet from very close to a core network (Terremark NAP). I started to notice the buffering after watching/listening to instant.fm by feross.
It's strange: sometimes a video downloads at 38Mbps - very fast - and some other it keeps buffering resulting in less than 1Mbps. I can see it with NetMeter (I love to always have a graph of my connection speed).
I analysed it a litte bit, all I could see was packet loss and retransmissions. It might be realted to the IP where I'm getting the youtube video, but I still have to dig it more.