YouTube’s Bandwidth Bill Is Low | Hacker News Reader

YouTube’s Bandwidth Bill Is Low

wired.com

98 pointsbensummers34 comments

This would be more accurately stated as "YouTube's marginal bandwidth cost may be close to 0". They're peering with enough providers that they don't pay for bandwidth like, say, Wired does. While Wired pays for the amount of data they send out (or expect to send out), Google mostly pays for the infrastructure, not the data.

Peering like this isn't all that new. Roughly 80% of our transit goes directly to our host's peers, which means they're probably not paying anything for that traffic (though they're charging us for it). We're happy with this because if it's going through a peer, it's much snappier for those particular users.

What you're missing here is that Google has built its own network with large global reach using dark fiber. Almost everybody will peer with a party like Google if they pop up at the local exchange.

By doing this Google can skip most of the usual transit bandwidth tax, which is where most of the money traditionally goes to if you're sending massive amounts of data to end-users.

transit bandwidth tax

What tax? Sure, Google doesn't have to pay for transit while us mere mortals do -- but that's just because Google provides its own transit.

The article glosses over the fact that maintaining that infrastructure costs money as well.

> Its costs for bandwidth are then amortized across the life of its fiber and routers.

It's basically the difference between 'renting' and 'buying', the costs can be (and probably are) significant though.

If you rent some resource you'll be paying bit-by-bit (pun intended), if you own it outright you pay it all in one big 'bite' .

So, their bandwidth bill is far from 0, it's just under a different heading on the balance sheet.

Exactly what I came in here to say. If I said I got free housing, you'd be quite disappointed to learn that by "free" I meant that I owned a home and hence don't pay rent.

Well, IIRC they were buying up 'dark fibre' before the YouTube purchase. So they had already bought the house before they were using it. And in the meantime analysts are trying to say how much rent they are paying... but they aren't paying rent because they bought a house years ago, and now they are making full use of it.

while the subject of the article is interesting, it's a rough read because of the author's seemingly ignorant suggestion that google abuse net neutrality, and the several typos. not really typical for wired.

So the actual cost is somewhere between $0 and $1,000,000 per day. tl;dr we have no idea.

From my experiences, if we assume that Youtube's bandwidth costs are similar to what Akamai charges for an extremely large video site, and their claim of 1 billion videos viewed per day is correct, they're probably paying about $5-10 million per month. Obviously, if their bandwidth is cheaper than Akamai, adjust those numbers accordingly. And it might be larger if 720p HD being the default mode sticks (those have 4 times the bitrate of their regular videos).

The whole point of the article is that they're not really paying for bandwidth like you would if you were buying from Akamai. When you peer with someone, you rarely pay by the mb. It's more like a direct crossover connection from your router to theirs in some peering facility.

The reason bandwidth costs money is not because ISPs and Akamai and other CDNs are greedy, it's because the infrastructure behind it costs money.

Building your own inftrastructure doesn't make bandwidth free, it just eliminates the cut that the provider normally takes.

Also, Akamai doesn't charge by the megabyte either, they charge by the megabit per second, as does most anyone once you start dealing with Facebook or Youtube-size loads.

Yes, that's what I meant when I said "mb", perhaps 95th percentile billing would have been more explicit. And when you start dealing with Facebook or Youtube sized loads, you generally start peering rather than just buying transit from various providers.

When you peer, you no longer pay a 95th. You work out a peering deal with some provider you send a lot of traffic to and effectively connect directly to them in some shared facility. Some providers will charge you a 95th rate when you peer, but even then it's vastly cheaper than what you'd be paying for normal transit.

Generally with CDNs you get charged per GB until a certain amount. Then it'll depend on your specific contract whether its GB or Mbps; there's no hard and fast rule.

The people who run that infrastructure can also help run other infrastructure, or the bandwidth can be used for more than just YouTube. That reduces the cost more.

From my experiences, if we assume that Youtube's bandwidth costs are similar to what Akamai charges for an extremely large video site

There's an "Akamai markup" which is a well known thing that Akamai charge for the heritage (somewhat analogous with the so-called "Mac-tax" except the Akamai "tax" is agreed to exist). A better comparison would be CDN pricing for Limelight or CDNetworks, the other two big players. It's worth bearing in mind that a lot of peering contracts for major carriers (but not necessarily Tier 1 - for example BT, Telefonica etc) depend on having a backbone - Akamai doesn't have a backbone, their backbone is the internet.

Also Google only do HTTP, so no Flash RTMP or Microsoft streaming which you effectively subsidise with an http product from a CDN.

They actually don't use HTTP, they use RTMP, try it yourself: Load up a Youtube video and scrub to the middle of the video. It will load to that very point even if it hasn't buffered that far yet.

They use HTTP with an addition in their servers similar to this plugin: http://h264.code-shop.com/trac to do HTTP scrubbing.

Google also used Akamai for its Google Live broadcast which indicated that their network couldn't do RTMP (at the time).

I was wondering what they were going to do with all that dark fiber that they purchased, and now we can see.

I guess it pays off to hire geniuses eh?

title feels a little misleading.

the best way to not have your cost scale is probably to do what spotify is doing - make every user into a peer in a bit-torrent network and have them share the vast majority of the bandwidth consumption

A Glance at Spotify's Peer-to-peer Streaming: http://www.google.com/url?sa=t&source=web&ct=res&...

I dunno if that's the best way. If you're large enough to make your own infrastructure worthwhile, I suspect that's way more robust than a relatively complex end user p2p setup.

Yup, Skype had that half a decade ago. Problem is that if there's a big Windows update that everyone installs, a large portion of your network goes down.

Come to think of it, large scale there could be a sort-of market in bandwith based on content. Between google and isp, who wants youtube to have bandwith more? You could say isp take data from youtube and sell it to the final customer. The only reason they historically get away with charging both the content provider and the content consumer is the balance of power: any single content provider is of little value to the isp, but the isp is of great value to that provider.

Now that google is a large enough force it could very well make money from offering bandwith, and isps will bid for it because not doing so would cost _them_ customers.

The key here is that Google bought up enough dark fiber to make itself in effect an ISP. Therefore, it's a negotiation between equals more than it is a content provider and an ISP. The fact that Google can make its services slower for users of a particular ISP if they cause too much trouble probably pushes the balance of power in its favor.

If an ISP hosts content, would analysts also say that they "have no bandwidth bill"?

This article states that "[Google] has purchased unused fiber optic cable known as 'dark fiber'. ... Its costs for bandwidth are then amortized across the life of its fiber and routers." Sounds logical to me -- it doesn't seem like Google is "getting away" with anything here.

If an ISP hosts content, would analysts also say that they "have no bandwidth bill"?

Yes, they very well might. The ISPs costs are largely fixed, it doesn't really cost them much to stream you a movie or whatever.

Everything costs nothing while you're using resources that you've already paid for, is a more accurate way of looking at it. At the point you exhaust those resources, the cost is the cost of adding more.

It's like the old memory/cpu/whatever is cheap argument. Yes it is, until you run out. Then you're in trouble.

More info:

http://blog.streamingmedia.com/the_business_of_online_vi/200...

I'd recommend Dan Rayburn's blog for a real-world perspective on CDNs, video etc.

Google has a lot of fiber and plenty dark fiber. But peering is not free bandwidth. A lot of people peer and depeer ask Cogent and Sprint :). Fiber is not free, power to routers is not free. But a headline like that will get you more clicks and use more bandwidth.

You can peer quite heavily without a fibre (or any) backbone. Akamai peers heavily but has no backbone.

Power is a major issue in datacentres here especially ones in large cities. I also wouldn't be surprised if Google use software routers instead of hardware beasts that cost as much as a house.

You're right you don't need to own fiber to do peering. But google has a ton of lit and dark fiber. http://www.voip-news.com/feature/google-dark-fiber-050707/

Forget about routers, Google has designed+built their own 10gb ethernet switches: http://www.nyquistcapital.com/2007/11/16/googles-secret-10gb...

When I traceroute youtube the last host my ISP owns is actually named google-10ge.bredband2.net. My ISP peers with google by AS29518 SKYNET-AS to AS15169 GOOGLE.

I remember a post on here a while back that suggested that YouTube costs Google about $1.6 million per day. :D

Reply on news.ycombinator.com