Google Video Quality Report | Hacker News Reader

Google Video Quality Report

google.com

276 pointsdudus174 comments

Now that it's fully legal for ISPs to throttle Netflix, Google, et al (thanks, Verizon) and extort money from them in return for giving customers the bandwidth they pay for, I think it's time for all the major players to track performance on the networks and call out the throttlers with a message right over the video stream on their website and in-app (Android, iOS, Xbox, etc). SO, when you use TWC and are getting only 60k as are all your neighbors, Google can pop a warning up on your screen that your connection is slow and the support number for your provider. This will result in lots of angry customers calling their providers about the issue, and get them to be angry at the right people.

Also net neutrality was a key barrier to ad hoc and distributed/self organizing networks. Once people perceive that they are not guaranteed to get what they paid for, there will be a race to the bottom to find alternatives. The ISPs killed the goose that laid the golden egg and don't even know it yet.

This doesn't ring true.

Mesh-type networks fail for many reasons. Mainly, nobody seems to have found a good use for them, especially with everybody walking around with 4G hotspots nowadays. Seems very doubtful that the FCC would have gotten involved regulating some fledgling community network.

Can you provide any evidence that an adhoc network was ever regulated under network neutrality rules?

I don't think that's what he's saying. He's saying that ad hoc networks never took off because normal ISPs were good enough. Normal ISPs were good enough because of net neutrality. Now that net neutrality no longer forces ISPs to provide equal service levels for all data someone may want to access, there's an opening for ad hoc networks to fill.

Just to play devils advocate here, but has anyone fully considered the impact on ISPs that this massive increase in demand is having on them. Especially the hated Cable companies who are seeing customers turn their backs on their own TV products whilst shouting and demanding more and more bandwidth so they can watch (and pay instead) NetFlix et al.

NetFlix and YouTube do not pay the ISPs anything. ISPs are not some cloud based company selling software, they are a utility company with Billions of $$ of investment in physical cables spanning every street in every country. In nearly every instance, the initial investment in this infrastructure was for a completely different technology (telephone or TV) from a time way before internet. The industry is now at a point where demand for bandwidth is simply outpacing technological innovation and these networks can't take the strain.

Think about the work involved in doubling Broadband speeds over say 2-3 years, in line with demand. In the past this involved installing some new equipment in the central office or exchange. Then this wasn't enough so the equipment had to be installed in the street, costly, especially as the number required was so much higher. Now we're at the point where the copper Coax or phone-line can't physically do the job anymore and it needs replacing, ideally with Fibre. Imagine the work and cost involved in doing this task. It's a hugely labour intensive task to dig up every street and there are many layers of red-tape and bureaucracy that simply weren't there during the initial build out. Once you've done that, you then need to buy vast amounts of brand new equipment and back-office software to run these new systems, plus hire in a load of new talent.

Now imagine doing this for every single home in the US or Europe (not just Kansas). Now imagine doing this without increasing your prices. Now imagine doing this whilst some key revenue streams are also being taken away from you.

Then think about the payback period on this investment... how long will all that Fibre even be used before wireless speeds increase and half your customer base decide they don't even want a fixed line anymore because they can get 1Gbps on their mobile? That 4K Video stream is still only 30mbps! 10 years perhaps?

I'm not surprised the ISPs are fighting back, they're putting in all the money and taking all of the risk for little to no reward.

Now... to speak my own mind. The sooner the old guard realise their time is up and adapt to fit the new demand the better, for customers and for the ISPs. Cable companies in particular have a very difficult time coming to terms with the fact that they're now just a data pipe. People don't want to buy their TV or their programming. People want internet and on the internet you can get everything, including TV and Movies. ISPs need to start investing in their networks even if it means we have to pay a bit more, because it won't come cheap, and stop pretending they're still relevant. But throttling the guy selling the better widget is just petty and benefits no-one.

The big difference is that each player is already paying for bandwidth at their end. Google, Netflix, myself (PortableApps.com), we all pay for bandwidth to host and serve videos, binary downloads, etc. And then the home user is paying on their end.

But, ISPs charge their customers for 'unlimited' bandwidth, a lie, and then have ghost caps, throttling, etc. The only person being dishonest here is the ISP. And having all the major content providers on-board with calling them out on throttling will keep them from doing it since it will 1. make their customers angry/consider switching and 2. cost them quite a bit in support hours. So, they'll be forced to switch to a more honest set of advertising, billing, and bandwidth counting based on individual usage.

Granted, we'll still have to put up with their monopolies, lack of common carrier rules (so your home VOIP phone is offline for weeks and you can't do anything about it), poor customer service, etc. Realistically, we'll need wireless to come up with better plans to force them to be competitive. Living in NYC and having only a single cable company as your option (unless you want incredibly slow DSL that was down 1 week of my 1 month trial) is kind of silly at this point.

> The big difference is that each player is already paying for bandwidth at their end

We are really only paying for bandwidth to/from our providers. Everything in-between is best effort, nebulous peering arrangements, etc.

> But, ISPs charge their customers for 'unlimited' bandwidth

I doubt you can find a single residential Internet provider who advertises unlimited bandwidth at this point. The marketing for 'unlimited' started to counter dial-up services like AOL who billed usage by connection duration not bandwidth usage.

> having all the major content providers on-board with calling them out on throttling will keep them from doing it

Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up. Your video is buffering because Netflix & Google want YOU to pay more for Internet service"

> Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up. Your video is buffering because Netflix & Google want YOU to pay more for Internet service"

Haha, good luck with that on increasingly HTTPS-only services.

"You need to configure your browser to trust our SSL certificate authority as part of your account creation and router setup process....oh by the way we are also going to use it to inspect your SSL connections."

Manipulating websites that way is likely illegal in a lot of countries.

I doubt you can find a single residential Internet provider who advertises unlimited bandwidth at this point.

Nation-wide? Maybe. But there are plenty of ISPs that offer unlimited bandwidth as a differentiating feature. See sonic.net, which covers most of CA[1].

1: http://www.sonic.net/sales/maps/

>We are really only paying for bandwidth to/from our providers. Everything in-between is best effort, nebulous peering arrangements, etc.

I don't know about you, but I'm certainly not. I fully intend to be provided connectivity to the wider Internet at the speed advertised. The money I'm giving to them pays for the peering arrangements that let that happen. Let me put it another way, I wouldn't pay anything for a connection that's just to/from Comcast with no wider connectivity.

>Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up.

Netflix and Google can just use HTTPS if that happens. I believe Google already does.

Interestingly, in Australia we have had Data Caps for a significant amount of time (2002 I believe was when Telstra introduced the 3gb Cable Cap).

I know it's not US based, but looking at ISP's and their offerings elsewhere, there are options available. For example I have an Unlimited ADSL2+ connection from TPG (www.tpg.com.au). The trend locally seems to be ISP's offering more data and more value for money (Unlimited in some cases, but 500GB / 1TB plans for marginal fees per month)

I'm also with TPG and can't fathom having a cap, much less the typically tiny ones offered. Recently I noticed that Optus is now offering an unlimited account for a reasonable price. I've heard anecdotally from people working there that it would be cheaper for Optus to do away with the caps because of the internal overhead that tracking and accounting requires.

Doubling broadband speeds every 2-3 years? if only they actually did that, i might have some sympathy for them. I get about 15Mbit. This is the same speed i have gotten for approximately the last 6 years. the only thing that is doubling every 2-3 years is the amount they bill me.

>NetFlix and YouTube do not pay the ISPs anything.

bullshit. You think google and netflix have teams of ninjas that go and splice wires into your ISPs lines? There are long standing agreements regarding how the exchange of data happens between Tier-1 providers, and how much it costs to send data to somebody else's network. Netflix is not getting a free ride. The consumer ISPs are trying to get paid twice.

> Doubling broadband speeds every 2-3 years?

Doubling the infrastructure required to deliver customer bandwidth demands -- not speeds. For example 2 years ago maybe the average customer had utilization between 8-11PM of 4Mbit/sec. Today it's maybe 8Mbit/sec. That requires your ISP to deliver 2x as much bandwidth during those hours regardless if the speed package has changed or not.

The ISP are already massively profitable.

The data caps are artificial scarcity to create even more profit or drive people to their own video providers.

Cities have decided to only allow 2 sets of cables to everyones house (and the FCC decided not to make them share between providers) so there is a defacto duopoly - competition is nearly non-existant.

Hence they know they can get away with double and triple dipping. Nexflix has to pay more - guess who they pass it on to?

There was some similar discussion in Germany last year. The major ISP announced to throttle monthly traffic above 70 GB in the future. Funnily enough they told that 70 GB are enough for every user while telling everyone that Internet traffic has a high increase every year.

Fact is, that their backbones aren't on any limit at all. The real bottleneck of the Internet connection is actually the last mile, the subscriber line. And that cable is exclusive per customer, so no shared medium and throttling wouldn't improve anything than making it possible to earn more money by selling premium contracts. To be fair, this is different for Internet over the TV cable which is indeed a shared medium. Still, throttling on the basis of a monthly limit doesn't make sense since it is reset for the beginning of every month.

Oh, and also that ISP was the only one not interested in doing a direct peering with Google's local CDNs for free. In France Google is actually paying for getting YouTube better routed.

The important thing is that these ISPs just try to make money on both ends of the cable while actually arguing with the power of the opponents. "Hey enduser, here you'll get YouTube! - Hey Google, wanna have more satisfied users?" And that they are often actively lying for achieving that.

Yeah, how much would it cost to lay new fiber and all the necessary equipment to support 1Gbps/house? Less than [https://www.google.com/search?q=google+fiber+profitable] $70/month/house [https://fiber.google.com/cities/kansascity/plans/] it turns out.

Now, admittedly wiring up all the suburbs and rural areas might cost more, but the incumbent players are really not so beleaguered [http://finance.yahoo.com/q/is?s=CMCSA+Income+Statement&annua...].

Funny story about that 1gbps needing fiber. In fact, a DOCSIS 3.1 modem at 24x8 will easily hit 1-1.5gbps with half the investment necessary as it would to roll out today under DOCSIS 3.0. So expect to magically see cable start competing with Google Fiber if it actually takes off! ;-)

It's not (only) that 1Gbps to the home, it also that fat pipe per street, the even fatter pipe per neighborhood, etc.

Especially the latter pipes need to get a lot fatter, the more people use that 1Gbps bandwidth.

Look at it this way: roads to houses haven't increased in bandwidth in the past century or so, but investments in highways have. Why? People started using 'their' bandwidth. They don't walk to work, but use a car that takes more room, and also travel over larger distances.

Any increase in average utilization is an insignificant cost compared to the elephant in the room: Broadband speeds have been essentially stagnant for 10+ years if you look at tech development vs what gets deployed. The ISPs have been reaping the savings that faster backbone technology brings and not passing them on to customers, while failing to deploy improvements in last mile connectivity tech.

Actually the tech would have gotten better much faster if its development hadn't stalled due to low demand. If you plot a Moore's law type from the 90's, we should be getting 10 gigabits at home by now.

Thanks for taking the ISP POV for one second.

Keep in mind that generating traffic is nowadays really easy.

For 50k$, you can buy 50 servers with 2x10Gb cards, and generate a constant 1 terabit per second of video streaming data. No monthly fee besides electric bill.

So the "content" provider offers the ISP to host 50 of its cache servers and now the ISP has 1 more terabit of data to distribute over its whole network.

Anyone sees the flaw here ?

Good. I swear youtube is throttled by Time Warner Cable for me. I stream Netflix, Hulu, etc and never have problems with them. I can confirm that I'm getting my speeds of 20MBit/s down, and yet almost every video on YouTube buffers. If I use youtube-dl to download the video, I can confirm that my actual connection speed is somewhere around 60KB/s. If I then use youtube-dl on my VPS with the same video, I get somewhere between 2MB/s and 8MB/s.

^this. Same with Netflix though for me. Where are you based? I'm in Austin, Tx.

Any of the speed tests online will report 30mbps/5mbps, but I'm unable to stream HD on Youtube or Netflix during peak times.

I'm in Cleveland, OH. Like I said, never have problems with other services like Netflix or even streaming videos by vimeo or skydrive, just Youtube. It's so frustrating I've considered setting up a network-wide VPN for my home network to get around it

Same for me on Time Warner in Austin. I can't wait for Google Fiber to roll into town.

At peak times it isn't out of the question that the video service itself is a bit under the weather. I've had some troubles with Amazon at peak time, and it's always been with fresh content, such as yesterday's television show that just became available on Amazon four hours ago. It wasn't obviously my network, as switching over to Netflix worked just fine at full quality. (That's not a proof it's not still somehow network related, but it strongly suggests it wasn't my ISP.)

I'd be happy if my TWC in Austin would just stay up. I've had many techs out and the service goes down daily for sometimes hours at a time. ugh...

I think Comcast is doing the same thing here in Colorado. Lately, I've been seeing quality drop in videos where it shouldn't. Running speed tests or downloading from newsgroups I'm hitting 60mbps.

Same here for Comcast in Seattle. From 6-8pm I can't stream youtube on anything more than 360p or it buffers constantly.

Other video streaming services seem to stream fine (Netflix, Hulu, Amazon Instant Video) and speedtest.net reports 56Mbps down, it's just youtube that suffers.

Ditto here re Comcast, but in Mountain View. I am a stone throw away from YouTube HQ and still get bad performance. Tried the 8.8.8.8 DNS and not much different. Comcast is definitely squeezing the pipes!

You'll be happy to know that Comcast's Denver ibone switching/routing has been over capacity for many months at this point...Same goes for their Oregon switches (which service SV and the PNW seaboard)

http://ispspeedindex.netflix.com/results/usa/graph Looking at comcast over the last two months looks like there is some data to confirm your notes about their backbone having issues. They are on a steady decline.

Same thing happened to me with AT&T in Tulsa with their fastest offering (I think it was 24/3 at the time). All speed tests, downloads, uploads were almost always the advertised speeds, but I couldn't watch Netflix or HBO GO, or Hulu. I could, but it was unbearable. To the point of kicking over magazine stands and throwing Nintendo controllers at the TV.

When I called support about this they claimed it was my setup. A Time Capsule connected to their modem/router. Whatever.

I switched to Cox (100/25) and get better than advertised speeds in all downloads, uploads, and speed tests, and have not had a single issue with buffering on any service. I have even been choosing the 1080p option on YouTube without any buffering.

The only problem I have now is whenever my Time Capsule is directly connected to my Motorola modem it causes the modem to restart, so I had to use a slower, older Linksys in-between the two until I figure out what's the cause of that.

Similar prob with AT&T in SoCal, bad Netflix, Youtube, and actually, almost all large file transfers of any kind, including linux isos, apple dmgs & system updates, etc. Switched to an independent ISP carried over the same AT&T DSL lines and I could actually get the advertised bandwidth consistently.

I suspect the only reason DSL is accessible by third-party ISP is that it carries some regulatory access rules with it because only U-Verse can get to the higher tiers of bandwidth.

My broadband choices are lower speeds with third-party DSL 6Mbps, or higher speeds with AT&T U-Verse, or TWC both of which seem to selectively throttle competing video services like YouTube or Netflix.

Yup. This article is a not so subtle jab at this and trying to educate consumers that their poor video experience isn't Google's fault. Both Google and Netflix are raising the bar here, customer demand/complaints are providing the necessary energy and the ISPs are maneuvering to get more money out of someone.

I feel like Verizon Fios does the same. I have 50 down and YouTube is painful most times

Are you by any chance using non-TWC DNS servers? I had this same frustrating problem on Comcast, and it turned out that their CDN works by using DNS to direct you to local servers; by using the Google 8.8.8.8 etc I wasn't getting that.

Even still, though, I run into certain Youtube videos that absolutely refuse to ever fully load -- it's always non-popular ones, and it's like they cache miss and just never get surfaced for me to watch the whole way through.

Have had the same experience on Comcast.

I am using different nameservers, but I've tried switching to the default nameservers and nothing appeared to change at the time, maybe I'll have to try it again

> it turned out that their CDN works by using DNS to direct you to local servers

Are these servers' IPs static? Can they be hardcoded in your hosts file?

I use TomatoUSB on my home router, which includes dnsmasq (under Advanced / DHCP/DNS, there's a textarea for "Dnsmasq custom configuration"). I have a line in there:

    server=/netflix.com/75.75.75.75

which tells dnsmasq to use my ISP's dns (75.75.75.75) instead of my default (8.8.8.8 and 8.8.4.4), only when resolving domains that look like netflix.com.

I don't actually know whether this is an anchored regex or what, maybe I'm spuriously using my ISP's dns for foonetflix.combar.info, but I don't really care.

All devices that use DHCP use the router (and thus the custom dnsmasq config) for dns by default. It improved my ability to stream netflix at the time, though I didn't measure it. TomatoUSB has neat-looking graphs, maybe I should run a test.

Edit as to your question:

> Are these servers' IPs static? Can they be hardcoded in your hosts file?

I suspect they use AWS for at least some layer in between you and the bits you want (if not the actual CDN, the server pool that generates the token that you need to get the bits from the CDN), so I don't think there would be a set of IP addresses to hard-code.

Kinda off-topic, but why do you use Google's DNS (8.8.8.8 and 8.8.4.4)? Unless I'm missing something, by doing so you are basically giving them a record of every domain you access. I don't really trust my ISP either, but why share that information with anyone else (especially a company with great interest in data mining everyone)?

Assuming you don't believe they are lying:

We built Google Public DNS to make the web faster and to retain as little information about usage as we could, while still being able to detect and fix problems. Google Public DNS does not permanently store personally identifiable information.[1]

More details about exactly what is logged at [1]. As for why, speed and security are the main benefits touted [2].

[1] https://developers.google.com/speed/public-dns/privacy

[2] https://developers.google.com/speed/public-dns/docs/intro

Despite being a (bit of a) privacy enthusiast, I still use Google's DNS servers, for a few reasons:

1. I use chrome. If I didn't use Google DNS, they'd be able to track 90-99% of my internet usage just by watching what domains I visit in chrome. The remaining portion is e.g. irc, usenet, xmpp, ssh, etc, that doesn't go through chrome. I could use firefox or opera or w3m or telnet or whatever, but I like chrome.

2. I became disinterested in using comcast's dns because of previous internet-breaking shenanigans like http://corporate.comcast.com/comcast-voices/domain-helper-se... .

3. I work at YouTube (owned by Google), so I get to see the internal temperament and discussions regarding privacy, data safety, PII, and so on. I'm comfortable letting Google have this data.

  > I use TomatoUSB on my home router, which includes dnsmasq (under Advanced / DHCP/DNS, there's a textarea for "Dnsmasq custom configuration"). I have a line in there:

      server=/netflix.com/75.75.75.75

  > which tells dnsmasq to use my ISP's dns (75.75.75.75) instead of my default (8.8.8.8 and 8.8.4.4), only when resolving domains that look like netflix.com.

You are my hero, good sir! You've just inadvertently provided the solution I didn't know existed to fix my Netflix and Hulu issues.

To be fair, I'm just repeating something I saw on HN before. Here, I dug up the original post, give that person your upvotes: https://news.ycombinator.com/item?id=6393829

Also, please consider sending the author / maintainer of dnsmasq (Simon Kelley, simon@thekelleys.org.uk) a thank-you email. Same for the maintainer of your router's after-market firmware, if you use such a thing.

I found the same thing. I reports it to TW and (unsurprisingly) they refused to admi anything. Switched my DNS to OpenDNS and YouTube works perfectly.

I have TWC and have had a lot of success in speeding up Youtube by blocking access to the caching servers that TWC uses.

If I block 206.111.0.0/16, I can stream at full speed.

See: http://www.reddit.com/r/technology/comments/13kmvd/have_time...

I tried blocking some of the relevant IP ranges, but it seemed like it wouldn't fallback or something. Instead of getting perfect streaming quality, I would not be able to load the youtube video at all

AFAIK ISPs have special hardware from/for these big streaming services to efficiently handle the traffic. Every-so-often that peering/mirroring equipment cannot support the demand and they have to upgrade the capacity.

They probably do it, but Google is at fault, too. Their new DASH thing has made the experience so much poorer. They never should've done it to begin with, and I don't think whatever savings in bandwidth they might have from it are worth the compromise.

Youtube was perfect before, because it allowed you to cache videos, and go back on the videos whenever you wanted, with no need for reloading or whatever. DASH made it a much worse experience, at it only caches like a few seconds at most in front of the streaming, and it won't even allow you to go back in the video without reloading it again.

All of that is required by their advertising business model. The more YouTube emulates the pre-VCR TV world, the more they can charge for time and visibility.

> because it allowed you to cache videos,

Youtube has to pay for that bandwidth usage even if you dont watch the cached video later.

> go back in the video whenever you wanted without reloading

That's even more bandwidth that they have to pay for unnecessarily.

I wouldn't know. YouTube is actually not so bad now if you block Flash, save the videos via the HTML5 interface, and watch them at your leisure.

I wonder if proxying just those sites through a $5/month Digital Ocean server would work...

Worked for me when I had streaming problems with AT&T, but AWS micro was the cheapest at the time.

Same here! Youtube is faster for me from China through a VPN connection than in the US via Time Warner.

I'm really surprised nobody's mentioned peering/transit in this thread. Ars has a good set of write-ups on this topic, the latest being http://arstechnica.com/information-technology/2013/07/why-yo...

Whether it's the fault of (1) the residential ISPs for letting their ports run so hot, or (2) the transit providers for trying to stiff them on pricing for additional capacity, or (3) the content provider for choosing a cheap upstream who is known to engage in this activity is up for debate!

As someone else noted on another thread, it's remarkable how often the name 'Cogent' comes up in these kinds of issues...

Brighthouse (TWC subsidiary) throttles quite a few services these days, Youtube being the biggest one. The throttling is so intense 24 hours a day that Google's streaming algorithm (download a little, play a little, repeat) can't continue downloading after the first initial burst. I can easily break it by VPNing into Work or School.

They very recently started throttling Netflix. I have a 90mbps down connection, and I can hit those speeds even at peak times.

ISPs are trying to strangle companies like Netflix into paying them for use of their network...you know...because Netflix doesn't already pay for an internet connection...they get free internet...because they're Netflix.

This is like the Mob walking into your laundromat and saying:

Mobster: "We own this whole block, and we want you to pay protection".

Laundromat Owner: "But we already pay rent, and protection!"

Mobster: "No, Clothing Protection. You're making a lot of money off dirty clothes, dirty clothes that get cleaned in our building. We want a cut. It would be a shame if someone left a sharpie in every washing machine wouldn't it? Then no one would come to your laundromat."

Netflix: "What if we don't let anyone in who has a sharpie?"

Google: "What if I just tell everyone the Mob ruined their clothes?"

Google is smart to Name and Shame the bad ISPs. I just think they should build this into the Youtube player, whenever the buffering spinner lags for more than 5 seconds have a a little message fade in that explains what the hold up is.

Interesting. Couldn't they just run a tracert and, through broad data comparisons, prove quite elegantly what the holdups in the chain are?

Worth mentioning that Netflix has a similar effort: http://blog.netflix.com/2014/01/new-isp-performance-data-for... http://ispspeedindex.netflix.com/

+1 that was useful! Took 20 seconds to see how my local ISP does relative to other providers.

Great, but I can't see my country (Brazil) in there, even though Netflix operates here.

I can't get it to work in Washington, DC. I smell a conspiracy...

Google, please don't break my Back button. I actually try to use it once in a while…

There is nothing to go back on...

Anybody else see "Results from your location are not yet available." when you hover over "Your Results"?

Yes. Am in San Diego on Cox Cable.

Same here: Centurylink DSL in Indiana

I have this as well, and I'm in Los Angeles, which seems like one of the earlier markets they'd nail down. Can someone who does have results for their area post a screenshot of what it looks like?

Same in Austin, TX, where Google Fiber is coming next.

Same in downtown SF -_-

I see it, however when I was on YouTube while uploading a video and trying to watch a video at the same time I had a little blue message pop up below my video letting me know I can find out more about why load times are slow. I clicked it and it brought me to the page rating my ISP.

It's funny an ISP that uses the same lines as my ISP got an HD-Verified status, and then mine is only SD. Hmmm, throttling much.

Is it me, or is this website subtly paying homage to Sen. Ted Stevens' famous "Series of Tubes" anti-net-neutrality speech.

Yeah, I immediately thought series of tubes.

Maybe they used that imagery because a series of tubes is an apt metaphor for internet bandwidth.

People talked about Internet "pipes" for many years, but somehow Stevens was a moron for saying "tubes".

He wasn't a moron for saying "tubes". We was a moron for the extended, stammering, rambling and incomprehensible context he used that word in.

The rest all seems reasonable to me, but you think he's a moron for stammering? I have a noticeable problem with stammering myself, and so do a few of the cleverest people I know. Stammering is an involuntary speech disorder, not a sign of low intelligence. That's simply not accurate.

Stevens' whole quote:

"I just the other day got, an internet was sent by my staff at 10 o'clock in the morning on Friday and I just got it yesterday. Why? Because it got tangled up with all these things going on the internet commercially...

They want to deliver vast amounts of information over the internet. And again, the internet is not something you just dump something on. It's not a truck.

It's a series of tubes.

And if you don't understand those tubes can be filled and if they are filled, when you put your message in, it gets in line and its going to be delayed by anyone that puts into that tube enormous amounts of material, enormous amounts of material."

This doesn't strike me as someone who knows what they're talking about.

> I just the other day got, an internet was sent by my staff at 10 o'clock in the morning on Friday and I just got it yesterday.

See, this is the funny part, not the "tubes". His staff sent him an internet? Yeah, I can see why that might take a while to deliver.

I'm pretty sure he meant "email" and misspoke. The overall point that the people regulating things often don't know as much about them as a specialist would, but it seems rude to laugh at a small mistake like that.

The point is that he is trying to convince people to regulate the internet. His rhetoric includes nonsense analogies that he doesn't even get straight. It's ridiculous.

Far better to simply say "I've consulted with experts so-and-so, and their conclusion was [reads a quote from expert]"

The underappreciated brilliance of this quote is that it's not clear what regulated industry he doesn't understand from his quote. Is it the Internet? Transportation? Waste removal? Plumbing?

Also worth pointing out that if his staff (house.gov) sent him an email (house.gov) and they all presumably use the same mail server (they do, House IT is all in house), then the tubes he's referring to were literal wires going from the basement of the capitol building into his office. I suspect it'd be pretty hard for the "commercial" stuff to get caught up in that.

Sounds like House IT was bad at administering a mailserver (they are -- when I was at the Sunlight Foundation, we linked to them and they called us accusing us of a DDoS), and decided to blame it on "the commercial internet."

It's a not-terrible but very basic metaphor applied to an area of engineering which about 99% of his audience did not need explained to them with a very basic metaphor. People assumed that it was he who needed the metaphor, and... hilarity ensued.

Curious. For me the 30-240 seconds long ads YouTube shows are always 1080p (or at very least high quality) and never choppy, yet streaming YouTube content videos at the same resolution often results in terrible buffering issues. Are the crappy ISP (ComCast) cahce servers only caching the non-ad videos? Does YouTube serve these through an AdSense / non-YouTube source that bypasses the cache? There are times it seems the only part of YouTube that works are the ads.

The ads are cached because the same ones are shown to so many people. Most videos are served to any particular neighborhood much less frequently than ads, and so are usually not in the edge cache.

Yes, which in my case is an indication of the system working properly. Leads me to wonder if the issues I have with Comcast + YouTube are on the Comcast side, the Google side, or are just the nature of the streaming-video beast.

How many ads do you think there are, and how many videos? 100 different ads, and about a billion videos people are interested in watching.

Good step but in the absence of true competition between ISPs this does little for the end user. I can discover that my connection is bad, and then what?

If you have the largest online advertising and marketing company in the world telling everyone that an ISP sucks... well, then guess what the ISP will do?

Either nothing, or lose customers.

In the absense of meaningful competition, they'll likely just laugh. Still, it's good to have someone call them out.

Which suggests that it would be in Google's interest to start fostering some competition.

Hrm....

They've been playing with that in the fiber-to-home space. But running cable is expensive.

I'm familiar with some wireless Internet providers in a few markets. The cool thing is that they can poach high-value, high-concentration subscribers first. Since the model is based on transmitters, typically rooftop installations, and the like, if they can hop onto a building, run Cat-6 throughout, and plug it in, they've got the whole building, plus can run local connections throughout the neighborhood, and beam line-of-site to other tall points. It's a pretty cool gig.

Thinking maybe you set that up in a few major markets as a competition play.

At least you will know whos fault it is and direct your hate when the posibility arises, (new videosite or new isp).

Then you go out and vote. The absence of true competition is because the average Joe isn't quite clued into the underlying problem of why things are slow. I suspect this is Google's way of raising awareness.

Interestingly, this also puts ISPs at a severe disadvantage for ratings if they decide to do net neutrality violating throttling on youtube or netflix. I hope every bandwidth-heavy service starts publishing reports like this

Or, even better, a neutral 3rd party that reports throttling for many services/domains.

Like the FCC?

It's not that simple. Comcast/Time Warner are monopolies in their area because we tax payers subsidized the laying of the lines. Verizon tried to expand FIOS to compete and took huge losses. My wife works for a fiber company, and seeing the numbers, it's basically impossible to make the investment worth it unsubsidized at at a price competitive with Comcast.

Agreed that fixing the root problem isn't simple. But citizens being pissed off en masse about their internet connectivity is a necessary precondition to any real change, and Google certainly seems to be trying to fan those flames.

Well, if I were in the ISP business and looking to expand, areas with poorly rated ISPs would be interesting.

Frontier mysteriously got better after the FCC publicly shamed them, although it could have been a coincidence.

I didn't this happened elsewhere. Here in France it's a huge problem, providers even talk about youtube in their ads to get people's attention. Free is the most famous case in France because they constantly refuse to do their jobs thinking that it's google's job to do (I don't know who's right but I left them because of their problem with youtube). Few months ago (or was it a year?) they upgraded their modem to block Google advertisement by default. A pretty bold move.

I was really hoping this was going to address the seeking problem. It seems like whenever I want to seek, even backwards to already buffered content, there's a 50/50 chance the video player will stop working entirely and force me to do a refresh and lose my entire video buffer.

The past few times I've had this happen, refreshing the page just dropped me right where I'd left off before trying to seek backwards.

I'm a netadm for my network (a /16) and "Results for [my] location are not currently available". Any idea how I can alleviate this? I'm curious how we're doing.

This part was particularly interesting "We often deploy servers within your ISP's network, vastly reducing the distance the video has to travel and minimizing the chance for congestion."

Google Cache Servers. It also saves the ISP money by keeping popular traffic within it's own network/reducing carrier interconnect bandwidth.

I'm not totally buying this - why is it when I have this extension[1] enabled, YouTube FLIES and works just as I remember it to work back in 2007?

[1] - https://github.com/YePpHa/YouTubeCenter

I haven't used this extension but my guess is that it's defeating the host redirection that ISPs do to point you at their extremely shitty and underprovisioned edge caches. Most ISPs run them to cache popular youtube content and save on out of network bandwidth, however as with most things they are terrible at it and extremely cheap. The solution I've been most happy with is to run all traffic over VPN to an endpoint not under Comcast - keeps everything nice and speedy, albeit with slightly more latency.

I'm unfamiliar with ISP scale networks - where is the actual bottleneck for them? One of the large ISPs in the UK is Virgin, and they have their own fibre network. They also throttle Youtube.

Is there only so much data that can go down a fibre line? Is it the routers (I'm not sure if calling them routers is correct?) that lie between the main backbone and the network connecting a street/apartment? Is it the hardware in a handful of large data-centers?

Usually it's the peering between google/youtube and the ISP that is the bottleneck, internal ISP network and google CDN are usually fine there's just an unwillingness to upgrade the connection between them (usually because parties can't agree on terms, some ISP would like to be paid when peering).

Traffic inside your network come free of cost of course except for your own bandwidth constraints, but once you need to communicate with other networks that's where you start to pay.

Google offers free interconnections, but sometimes you can't just connect directly to Google and have to go through backbones that are owned by private companies.

So ISPs need to create as many interconnections as possible with other networks to decrease their costs, invest in their own infrastructure and pay for everything their network can't reach directly.

Quite often videos are buffering on my 80 Mbit connection at home - however when I switch to tethering with my phones connection, there is no problem. I always had the impression that providers try to cache or throttle Youtube video downloads. So I really appreciate that Google is doing something about it.

On a sidenote, the biggest german provider was trying to get money from services like Youtube for delivering their content. This rating turns it upside down :-)

Well, I'm on a resold Telekom VDSL connection, and so far all I see is "Results from your location are not yet available."

I've had major problems with YouTube videos in the past (despite having a very reliable 50Mbit down), but recently it's been fine.

I can't even work on my regular job or projects without seeing at least 4% packet loss, intermittently spread out across the day. Not only that, I clearly see pings taking too long even with wired connection. Comcast has told me they cannot do anything about it. They say that the 2 MacBooks and a couple of iPads is a lot of traffic for the router+modem they have given me.

Not only this, I know that the moment I start Youtube or Amazon instant (I do not have Netflix), the packet loss starts to go up or at least the ping return times sometimes go up to 4000ms. This also happens when I RDP into my Windows PC at work when I telework.

I have tried resetting the router, use it with other channels, use it with and without all possibilities of encryption (WPA, WPA2 etc.) and I really cannot think of doing anything anymore. The Comcast representation said they will reimburse $30 bucks if I got my own router because the issue is not their fault.

I am not sure how to approach this issue anymore. My apartment building does not have Verizon FiOS or any other service. I have given up all hope. Sometimes I feel like adding unlimited LTE/4G on my iPhone from T-mobile and use that as the main Internet hub.

I'd absolutely try as much of your own equipment as possible. Comcast may well be up to other tricks, but they also have no incentive to give you anything other than a "good enough for most people" modem/router.

It can be surprising how much of a difference better networking gear can make. I've seen a VoIP router throttle a 50 Mbps to 5 Mbps for no apparent reason (over-aggressive QoS logic?). I've also seen issues with modems that don't have the latest-and-greatest DOCSIS support (you'd think Comcast would notice that sort of thing when you complained, but...).

I'd also avoid combined modem/wireless router boxes (which it sounds like you have) like the plague. The bundled wireless is usually much worse than what you can get separately. Plus wireless technology tends to move faster than modem technology anyway.

There are still a lot of people in rural America that don't have internet speeds above 2MB/s, and that should be our top priority.

I agree, let's move them to the city where the bandwidth is better.

Kansas City has Google Fiber. That's a start.

Probably very relevant for this discussion: http://mitchribar.com/2013/02/time-warner-cable-sucks-for-yo...

I have FiOS and have the same problem and following these instructions transformed my youtube experience.

> We pick the shortest, most direct route.

Eh, no. I'm sure they try but more often than not my YouTube is streaming from Amsterdam according to http://redirector.c.youtube.com/report_mapping, I live in Stockholm and have a 250 MBit connection. So instead of blazing fast speeds I get buffering like I'm on a modem.

I've found no way of reporting this issue, my ISP did not know either. I worked around the issue myself with hundreds of lines like this "74.125.163.38 r1---sn-5hn7ym76.googlevideo.com r1.sn-5hn7ym76.googlevideo.com" in my router hosts file. This finally made videos buffer pretty much instantly.

I find it amusing that their big support button[1] ends up at an error page[2], given the bad reputation of Google's consumer customer support.

[1] Rightmost, here: http://www.google.com/get/videoqualityreport/#a_faster_web

[2] https://support.google.com/youtube/checklist/3480866?p=video...

This is not an error page for me.

That link seems to only work if you are signed in to Google...

It seems like only a matter of time before Google starts informing people where, along the pipe between the Youtube servers and their computer, their traffic is being throttled.

Not that I have the time to do this, but it would be ideal if something like this existed for multiple services and was run by a third-party. Kind of like http://speakeasy.net/speedtest that speaks the APIs of various streaming video publishers. If it's crowd-sourced and non-profit, then it could be more credible and less subject to allegations of impartiality by the ISPs themselves.

Who can one test throttling by an ISP because I think Comcast is throttling youtube for the last 3 months. Its been very slow for me but my other downloads are 7MBs.

It is sad that this has to exist. Net neutrality is pretty much in the trash if companies like Google and Netflix have to issue reports like this.

I've noticed a trick when watching long HD youtube videos that I need to pause and re-wind several times. If I'm signed in to my web browser the video quality is demoted to lower quality after a while of watching, pausing and re-winding. To avoid this degradation of quality, I use a browser in incognito mode, and the quality stays HD. Does anyone who why this occurs?

Interestingly, one of the early Nest investors is the CEO of an ISP who is in a battle with youtube over network traffic.

I also experience 'buffering' or pausing/hanging videos almost any time I try to watch a youtube video. The video ads of course work perfectly fine, so I don't think I'm buying this 'it's your ISP' argument. For what it's worth, I'm sitting on 100M up/down fibre...

That is because the ads are cached since a lot of people are going to watch that ad.

For Google to build their own fibre optic networks, that will be able to accommodate a lot of the USA and Europe, what's the costs/difficulties?

I am aware they were trialling some sort of netwrok locally to CA, how difficult/expensive is it to roll this out worldwide?

YES! Deutsche Telekom needs a public shaming. They are the former state monopolist in Germany and use their large customer base als bargaining chip, refusing to peer at internet exchanges and so on.

Needless to say that YouTube loading speeds have been sub-par with that ISP.

Anybody knows why recently Youtube re-buffers every video when I hit replay? It might make sense for long videos but for relative short videos (music videos and the like), it's really frustrating.

It's because Youtube uses DASH[1] by default now. It's also the reason why it never buffers more than a few seconds of video anymore.

I installed YoutubeCenter[2] and disabled DASH from the options and it seems to work well every since.

[1] https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_ove...

[2] https://github.com/YePpHa/YouTubeCenter

Does DASH actually require you not to utilize the browser cache for some reason? It seems to me that all you have to do is keep the request URLs around that you previously requested and perhaps the byte range that you fetched if you abort and the browser should be able to pull it straight from the cache. No?

Of course you'd get the same quality as the previous play, but with a little work you could splice in some higher quality segments if you wanted.

That's confusing. It listed the 9 ISP's in my area... and the local power company. The power company's graph looked completely different from the ISP graphs, of course.

I get great streaming from netflix, and I find youtube frequently has trouble. Maybe my ISP (Verizon FIOS) is throttling youtube and not netflix, but that would be strange.

In other words, Google thinks a PR campaign and angry customers is cheaper for the same bandwidth to edge networks than just paying the owners of those networks.

Under the net neutrality paradigm, the entities "paying" for the last-network bandwidth are the ISP customers, not the content providers. The content they choose to receive has traditionally been expected to be treated "neutrally" without requiring payment by the provider. This argument has been had, and in virtually all cases our community has agreed (contra the positions of many regulators it seems) that it's a good thing.

Have you flipped on net neutrality or are you just confused because Google is the party harmed in this case?

Yes, because the customers of the networks have already paid for the bandwidth!

I buy internet access and that includes access to youtube!

You might, for just a brief moment, consider a less cynical, equally plausible explanation.

What does the Y axis mean on these graphs?

Google Fibre will soon top the list.

Well played Google .. very well played.

Youtube videos are terribly slow for everyone, regardless of ISP. Every other video service works much more reliably than Youtube. It's your goddamn fault, Google. Fix it.

ha! they need not worry about my ISP. Because youtube is totally blocked for more than year now. hehe.

How will they technically do it?

It's likely they'll just start doing analytics on the streaming speeds as we go about streaming youtube videos. They'll extract the ISP from our IP.

That's how I'm guessing Netflix does it: http://ispspeedindex.netflix.com/usa

Im liking the series of tubes

Regarding the graphics shown, so it's a series of youtubes...

So basically...it's a series of tubes.

well played, Google.

Google, would you please stop making these pretty advertisements and actually fix the problems? Thanks, Internet.

Wow, angry googlers strike again ;)

or maybe you just have no clue about what "the problems" actually are, and who needs to fix them.

Telcos will have to invest in capacity where Google wants in order to get a good rating. Since Google is a de-facto monopoly, I think we'll see an Antitrust case soon.

How is youtube a monopoly?

There are many free-to-upload/host videosistes.

Monopoly is defined by market share or market power. In the strict sense, it means "single provider", but a sufficiently large company can exercise an effective monopoly.

Having a monopoly is not of itself illegal. Using it in certain ways is.

Google are effectively a monopoly in search space. They're a massive presence in online advertising and video, as well as email.

If you look at the actual requirements they're talking about, they aren't that crazy. Even for HD video, they're talking > 2.5Mbps. So for really good quality, it's probably 5-6Mbps.

I have 85Mbps FiOS at home, and I still sometimes have issues streaming Youtube videos.

If my ISP can't even guarantee 10% of what I'm paying for over a fiber optic line, then yes, they need to be called out and publicly shamed.

What about if youtube.com's CDNs are overloaded and slow? Will you blame your ISP?

This is the problem with such ratings systems -- they're inherently one-sided.

I actually have much more faith in Google's capacity and CDN than I do in Verizon's history of sleaze.

After all, Google doesn't have a conflict of interest, Verizon does. The better Netflix and YouTube get, the closer I am to canceling my Verizon TV service.

Why would this be the case? It is only in Google's best interests to provide the best streaming service possible. It's not like they have any shortage of servers, bandwidth, or funds with which to do so.

5mbps per user, times tens or hundreds of thousands of users in a metro area.

Video is the thing that will finally get us faster IP networks, as it slowly replaces TV as the modern opiate of the masses. Even BitTorrent wasn't enough.

$60-80 a month for each one of those users (our bill is about $120 counting TV as well).

Not counting the fact that local governments subsidize the last mile. Also that routers and networking equipment keeps getting cheaper and faster.

It'll take a lot before I start to feel sorry for my ISP.

The thing is, I get amazing speeds from other sites and very good speeds from my EC2 boxes. Downloading MS/Ubuntu ISOs are just beautiful.

But those just aren't mainstream uses so I'm sure Verizon doesn't throttle those servers.

For the record I hate Centurylink. But I have 12 Mbps DSL with them and have never had a problem with youtube.

Google isn't close to a monopoly in the legal sense - especially in the online video delivery market. This will shine a light on ISP data handling and throttling.

I don't know how this could be antitrust. On the contrary, it seems like it would be protected as free speech.

Reply on news.ycombinator.com