Comment by ChuckMcM | Hacker News Reader

Comment by ChuckMcM | Hacker News Reader

When I read these I'm amused that people are surprised by Google's network. Its actual capability is nominally secret (my employment agreement and exit agreement both mention restrictions on revealing non-public details of Google's infrastructure) Except in this case there many publically available clues which should give you an idea here.

Consider that Google is investing in undersea cables [1] which nominally provide multi-terabit speeds between continents. And Google is cheap, really cheap, in terms of what they will pay for infrastructure which is why they started making their own switches [2] because even commodity ethernet vendors weren't cheap enough (although these days of state funded cyber armies one also has to protect against foreign made equipment [4])

What is more the amount of data you can send over a fiber is getting faster all the time [3] so replacing the inter-datacenter lasers can quintuple bandwidth.

There have been some interesting 'escapes' of course, once when one of the data centers became unavailable and all the YouTube traffic started coming from another one very far away serving a very large local population, and latencies went up but throughput didn't change, I know a couple of network administrators did a double take on what they were seeing :-)

[1] http://www.google.com/intl/en/press/pressrel/20080225_newcab...

[2] http://gigaom.com/2007/11/18/google-making-its-own-10gig-swi...

[3] http://www.newscientist.com/article/mg21028095.500-ultrafast...

[4] http://www.popularmechanics.com/technology/gadgets/news/4253...

While on the topic of YouTube... why does YouTube constantly buffer these days?

Your ISP? Youtube takes 10-20% of typical internet traffic (more than bittorrent) and competes with telecom TV offerings.

My ISP doesn't throttle, which is entirely why I picked them.

I have access to a connection that downloads 20 megabytes per second from S3, and YouTube still buffers on that connection.

Note that Vimeo doesn't buffer, and Vimeo has higher video quality usually. Also Netflix doesn't buffer.

YouTube has been getting worse with time rather than better. Tons of ads and slow video.

The issue is definitely on the YouTube end, and I'm not the first person to notice it.

Good question. I don't know. But I'll share a bit of speculation and you can tell me if there are holes in my reasoning.

With out going into specifics there was a lot of effort at Google going into pushing 'popular' videos closer to the likely place they were going to be played. Any caching system or content delivery system, is by definition limited to the amount of edge cache it has and its ability to predict where that content will be.

One of things putting pressure on that cache is probably Google's efforts to put 'premium' content near possible subscribers. Clearly people who are paying money have a higher expection of people randomly watching the latest cat pratfall. So if you're Google and you're doing more "TV" like things, you may have reasons to reserve some of that cache space for your premium content whether or not folks have asked for it from that part of the network 'yet.'

Another interesting area is in transit networks. Generally the tier-1 networks who provide peering connections have done that on a quid pro quo basis (basically I'll peer your traffic if you'll peer mine). However as web 2.0 companies like Google or NetFlix start creating very large flows, suddenly if you are 'peering' and your peer is putting a LOT of traffic on your fiber but you have relatively little for their fiber, then maybe you start thinking you want to be compensated for that. We talked about Level 3 and Comcast going at it in: http://news.ycombinator.com/item?id=1954077

So an explanation might be that your transit network is 'throttling' its peering relationship. I could easily see someone agreeing to 'peering' terms where they take the traffic they've sent and use that as a 'credit' and provide full rate service for an equivalent amount of traffic coming from the peer, but excess traffic might get throttled rather than being 'free.'

This is a very active discussion in some circles. I would not be surprised to see it come up in any discussion of throughput to 'YouTube' vs local throughput.

> My ISP doesn't throttle

That doesn't mean that they have infinite pipes to Google's edge caches, though (you're not exactly the only one pulling data through those pipes :-). Or that they're even pulling their data through the right pipes. I suggest talking to your ISP. If enough users complain to them, and the problem isn't in their network, they'll complain to Google.

It's not just one ISP. I've seen the behavior described by the parent from multiple ISPs in separate countries. It's been driving me nuts for so long that I started collecting information about the connections I've seen it from. Years ago, on 512kbit DSL, it was MUCH better.

PAETEC DS3 (45Mbit/s) in San Diego, CA

Time Warner cable modem in San Diego, CA (20Mbit/s)

Colocation at AmericanIS.net (1Gbit/s via Level3.net) in San Diego, CA

Videotron DSL in Montreal, QC (10Mbit/s)

Teksavvy DSL in Montreal, QC (10Mbit/s)

AT&T 3G in San Diego, Detroit, Atlanta (varies)

Verizon 3G in San Diego (varies)

Rogers 3G in Montreal, QC (varies)

Ha! I was wondering that as well. It went from the best way to watch short clips to a very average way. The (big) competitors don't buffer for me, while Youtube does these days.

My Youtube connection has been strange for some weeks now. I'm in Brazil accessing Internet from very close to a core network (Terremark NAP). I started to notice the buffering after watching/listening to instant.fm by feross.

It's strange: sometimes a video downloads at 38Mbps - very fast - and some other it keeps buffering resulting in less than 1Mbps. I can see it with NetMeter (I love to always have a graph of my connection speed).

I analysed it a litte bit, all I could see was packet loss and retransmissions. It might be realted to the IP where I'm getting the youtube video, but I still have to dig it more.

Reply on news.ycombinator.com