Netflix performance on Verizon and Comcast has been dropping for months | Hacker News Reader

Netflix performance on Verizon and Comcast has been dropping for months

arstechnica.com

169 pointssamsolomon85 comments

Hint: never use Comcast for residential service. Sign up for business service, even if you're just one person in an apartment. You get better support, less BS, and (likely) improved immunity to copyright-trolling law firms.

They will still route about 10% of your packets via the Mars Reconnaissance Orbiter, of course, but that's just the nature of the beast.

This also prevents you from using your own modem and requiring you to lease one of theirs (not to mention requiring a year contract), doesn't it?

Dunno. I've never tried any other modems besides the one they gave (leased?) me.

When I signed up for their business class service about a year ago, it was not required that I use their modem, but they did suggest it (noting that any failure of equipment would be taken care of on their end). It was the bosses dime and my time if a failure happened, so I just went with their leasing options.

You can use your own modem with Comcast business service (I do) but it took about two weeks of calling all the tech support numbers I could find until I finally got a tech who was able to do it. Not because it's hard to do, but because most of the people I talked to were lying or incompetent.

The guy seemed a bit weirded out at how profusely I thanked him -- after all, all I'd done was tell him the model and MAC address of my generic Motorola modem.

Don't waste your time explaining to them why you want to use a real modem (I have my own router/firewall, their modem had NAT I could not disable) or that it's the same modem that works on the residential service (which is the same physical network). It won't help, just say "I bought a <VENDOR> <MODEL> modem, can you set it up on my account?" and hang up and try again if they act like your request is bizarre.

Thanks for this. I've been wondering, so tomorrow. I'm going to start the process.

It saddens me that I feel like every interaction with Comcast is a "process" at best and an outer circle of hell on average.

They have a list of approved modems for the both the residential and business services, with the business one being a small subset of the residential one. But I was able to get a modem approved only for the residential service on my business account by calling around. What worked for me after only a few tries was calling into the residential tech support and having them add it. It's apparently all the same infrastructure on the back end.

Not if you have a static IP. I talked to 5-6 reps to try and ended up returning the equipment to Amazon.

It requires at least a year long contract and a hefty set up fee ($200 for a one year contract and $100 for a 2 year if I remember correctly).

If you don't want a static IP, you can use anthing on their approved list:

http://mydeviceinfo.comcast.net/

(Which has all the usual SB* suspects).

If you want a static IP, AFAIK, you still have to use what they give you.

I am currently running on Comcast Business with 5 static IPs, and while I do have to use their modem to terminate the coax connection, I have my TomatoUSB'd Asus RT-66u router playing just fine with it. Their modem has some sort of weird 'automatic bridge mode'; I configured the external IPs on my Asus router and plugged the WAN port on the Asus into one of the four internal ports on their modem, everything works just fine.

Granted I still think Comcast is the scum of the universe for myriad and sundry reasons, but at least this aspect of their service was fairly easy to set up. I still miss the FiOS I had back in Boston though :-(

Be very wary of signing up for Comcast business service. They have an auto-renewal policy that renews your contract every year even though you thought you'd signed up for a single year (with the expectation of month-to-month). When you come to leave Comcast Business they will charge you 75% of the outstanding balance you have on the current year ($485 for 10 months in my case) and they'll fight tooth-and-nail to get that money. They don't apply this policy for residential customers because consumer protections don't apply for businesses.

During the four years that I had Comcast Service I found it to be very reliable and well run - well, except they wired the wrong apartment by mistake the first time they came out.

It took me about 2 months of fighting until they relented. There was a magic cocktail of proposed complaints involving the BBB, the PUC in my state, social media and others that finally liberated me from their contract.

I don't get it, what would happen if you simply hadn't paid the $485?

Lawyers, I imagine.

They sell your debt to collection agencies who harass you until the end of time.

True, but they would also settle for a lot less than $485.

Wouldn't this hurt your credit score if it went all the way to settling with a collections agency?

I had a 50/10 Comcast Business account that was active in the south bay area from 2009 until last year.

The terms were an auto-renewing 1-year contract, but I could (and did) kill it at any time on 60 days' notice.

I don't recall doing any negotiation on this point when I signed up at all.

A sad thing is there used to be a YC Comcast Business deal which was pretty awesome, but somehow this got killed (it was done through a local office, but Comcast National apparently decided it didn't fuck the users enough to meet with normal Comcast standards.)

So a fair number of YC founders and offices have Comcast, and while you can relocate the service address, if you close the service, you can't get that deal again. And you can't upgrade it (I think I get 30/10?). Going from 30/10 with a static IP to the next tier of service (50/20) would basically 3-4x my cost.

Last week we try to install comcast business in our church. They would not give us a wireless router unless we accepted her installation of a publics hot spot too. This is against the church policy because it will allow someone watching porn in the sanctuary with no knowledge for us. They would not relent.

We end up putting our own wireless router. Speed is ok, but I didn't like that comcast pushed her policies to the clients disregarding the client policy. It felt like a bully.

Couldn't someone just use a cellular network? Or a magazine?

Sure, but at least it's not on the church's own equipment.

It sounds like it'd be mostly on Comcast's equipment, in this case...

Still coming out the wire assigned to the church address, and that what the ip will be traced to, hence authorized by the church; as in "the church agreed to have this hot spot here"

Fair enough.

What if they used the church bathroom? Maybe consider getting rid of those or at least installing cameras in them so you can be sure no one is doing anything unholy. Hmm, actually then who could view the footage to monitor things? I suppose you could only permit someone that was totally trustworthy, like the pastor? But what if he/she secretly enjoyed that? Maybe have a two person team view all bathroom footage? Make sure they pray first (and maybe put a camera on them, just in case. Can't be too safe).

You're missing the point. If someone brings the internet with them into the church and does X, the church is not involved. If someone comes to the church, connects to their internet connection and does X, the church enabled it.

> This is against the church policy because it will allow someone watching porn in the sanctuary with no knowledge for us.

Sounds like a feature.

Why not wrap their supplied router in aluminium foil? For a few dollars the problem is solved (although I agree with everyone else....people could just as easily use their data connection on a phone)

Ah, the juxtaposition of this could be delicious. They wonder why there are no connections at that public hotspot. When they come out to check it and nothing's wrong (just remove the foil before hand - it's not like they could access the building without your knowledge) . They leave, you wrap it back up and they come out again with the same results but this time they replace the equipment... and so on.

The only thing that could be better is if you gave them a window of time in which you will randomly show up at the facility to unlock the office or whatever and give them access. Say, between 8am and 11am. "No I'm not sure when I'll be there exactly. I have several appointments that morning. Is that an issue?"

Here is the kicker... the rep over the phone told me that his suggestion was for us to disconnect the hot spot after the tech leaves. But I had read the service contract before calling, and it say that we are obligated to follow all Comcast policies, and in the last paragraph it say that any modification to the contract must be in written and signed by a Senior VP from Comcast.

So I ask the tech to give me his suggestion in writing, and he refused, and gave me a different reason every time I ask for a printed copy of his suggestion. When after all that denial I mention the last paragraph in the contract, hid answer was:

- I'm sorry I cannot help you, but I don't think you will cancel the service because we provide the fastest access and that is what you care about

- I was voting against you during Church Council! I don't want you here, I was happy with ATT because they don't force their policies against their customers

- Sir, I don't know why they would do that. Do you want me to send our hot spot and your FREE wireless router?

- No, cancel the order of those, and I'll bring this to the Chairman. They will decide if they want to take your liability.

After that, someone donated a wifi router (wasn't me)

The problem here is that Comcast and Verizon don't want to pay for the upstream bandwidth to the Netflix Open Connect CDN which means that as Netflix increases quality and gains additional subscribers, the time-weighted bitrate (TWBR, the metric they use to compare ISP performance) will continue to plummet. This isn't to say that all Comcast and Verizon customers will feel this pain, some markets have better performance than others (e.g. SF Bay being an example of poor performance).

Edit: Other CDNs such as Akamai, Level 3 and Limelight pay to peer with Comcast; whereas Netflix refuses to since Comcast will simply gouge them for the opportunity. Netflix is increasingly moving away from third-party CDNs and onto their Open Connect CDN which will escalate this to a fever pitch.

Thanks for the great explanation. Moved to the Bay Area a year ago and have gotten to the point of debating canceling my Netflix account. The quality is terrible over comcast. No problem with Prime Videos. Next move I will hopefully go for someone else, maybe Monkey Brains.

It should be noted that having a Netflix OpenConnect Hardware Appliance is 100% free of charge for ISPs. The device itself is free, but I suppose you could factor in the cost to power the 4U server and the rack space used by it but those costs are negligible compared to the additional peering bandwidth used by not having one.

From the FAQ[0]

  >What does the appliance cost my organization?
  > The appliances (and any necessary replacements) are provided
  > to participating ISPs free of charge when used within the
  > terms of the license agreement.

[0](PDF) https://secure.netflix.com/us/layout/signup/deviceinfo/OpenC...

It's also worth noting that a 10ge flatrate port to Cogent isn't going to cost them that much either -- and then they don't have to deal with any additional hardware, just a crossconnect. But for some reason, paying an additional ~$6K a month so that millions of your subscribers have better service is too high a price to pay

10 gigabit can't come close to covering the growing demand, no?

During primetime hours, Netflix represents some substantial fraction of total U.S. internet traffic. If you're Comcast, running 25% of the U.S. broadband market, you're seeing a big chunk of that traffic. What if Netflix decides to double their max streaming rate? Or quadruple it? We can argue again about what it means that Comcast sells you a 20Mbit-max connection, but as a practical matter, it will cost them a lot more than $6K/month to keep up with Netflix's growth in subscribers, average daily viewership and stream quality.

10Gbps isn't a lot in terms of total capacity or utilization but even a 100 extra kbps of burstable bandwidth/user can greatly improve the experience. It's the difference between buffering every 30sec and watching with only minor stuttering. Also, bandwidth costs only go down/mbps as you get bigger. $6K/month is what you pay if you're a small business; Comcast or Verizon would get high-level pricing (puts their cost somewhere around ~$3.5K/mn). A 50Gbps commit with another 50Gbps at 95th percentile pricing would would cost them fractions of a cent per customer but would make people much happier.

Now they'll never to it because they'd rather keep that ~$25K.

100Kbps/subscriber is around 200Gbps for Comcast. And that's just 100Kbps! Netflix is going to keep telling their customers they're getting a suboptimal HD experience (and they'll be right!) until the average stream gets somewhere near Blu-ray's ~20Mbps. Maybe half that if you think codecs are going to get better, but by then 4K will be winding down the pike.

The point is that for Comcast and Verizon, there's no end in sight: A few more ports connected in a few more data centers isn't going to do more than make a few subscribers slightly happier for a few months.

It doesn't have to be dedicated, just bustable to a reasonably large number of subscribers at a time. Netflix will prefretch data and a small amount of burstable bandwidth goes a long way in viewing quality. This isn't about letting everyone stream SuperHD content at the same time, this is about relieving some pressure around the average bitrate (somewhere around 1900Kbps)

Other CDNs (Akamai, Level3, etc.) pay Comcast/TimeWarner to host boxes in their datacenters, and to support interconnects to their private CDN networks.

If they start giving it to Netflix for free, how do they continue to charge other companies for the same service? The thing to note is that the established precedent is for companies to pay, because they see value in it. Netflix is trying to get something for free that other companies pay for. Oh, but they slapped the word "Open" on the front of it, so it's about free speech!

The ISPs don't give a shit about downstream bandwidth consumption because they get paid by the CDNs for that. What they do care about is last-mile bandwidth; i.e. the bandwidth between their datacenter and the customers' homes. It's not unlimited and there are constraints. That problem doesn't get solved (in fact, it gets worse) with OpenConnect.

Could there be an opportunity for small ISPs who peer well with Netflix?

Or does the fact that they usually operate using the Big Boys cables remove any advantages this may bring?

Instead of Comcast's TV + Phone + Internet,

I would be willing to buy Small ISPs NetFlix + AmazonPrime + Internet package if I knew I could get high quality streams without slowdowns.

There definitely seems to be – my local ISP runs ads like this: http://rcn.com/j/rcn-supercharges-your-netflix-experience

Since I live in a neighborhood where FIOS, Comcast and RCN are all competing this is particularly attractive – we get rock-solid HD quality on Netflix and the Comcast customers are usually complaining about quality. I'm hoping this strategy proves successful…

How is that consistent with the sudden drop in bandwidth shown in the article's graph http://cdn.arstechnica.net/wp-content/uploads/2014/02/netfli... - which is in mbps so if netflix increased their quality, you'd expect it to remain level.

Which do you think is more likely, Verizon gained 15% more users after demand was level for months, or Verizon acted in their own self-interest, doing something they have publicly said they want to do?

How is that consistent with the sudden drop in bandwidth shown in the article's graph ... which is in mbps so if netflix increased their quality, you'd expect it to remain level.

To me, that looks an awful lot like our observed Cogent <-> Verizon/Comcast performance starting around October. It also looks like what happens when a network connection becomes saturated: Performance doesn't drop off linearly, it drops off much faster.

Netflix and Comcast/Verizon don't have enough bandwidth between them to support subscriber demand and Netflix's high bit rate streams at peak hours. There are three obstacles to improving that:

- Somebody's got to pay for bigger interconnect. This is actually the cheapest part, and Netflix would be happy to forgo the interconnect by putting a few boxes loaded with video cache on Comcast's or Verizon's networks.

- Somebody's got to pay for the added bandwidth on Comcast and Verizon's networks to push these video packets to subscribers. Well, duh, Comcast and Verizon should, right? Except now they're at the mercy of Netflix, who can keep loading on subscribers and upping the bit rate until a substantial fraction of home broadband subscribers are trying to pull 10Mbit constantly from 7pm-12am EST. Comcast and Verizon are not thrilled to have to pay for this, especially when it's unclear how they benefit.

- And of course: Netflix is in direct competition with Verizon and Comcast's own "OTA" TV and video on demand services. Every dollar Comcast and Verizon spend to make Netflix more appealing for their subscribers is a dollar they're spending encouraging those same subscribers to cancel or scale back their expensive TV packages.

Minor point, but if I'm paying for 10Mbit, shouldn't I be able to actually USE that fully?

Mind you, I'm not paying for a 1Mbit with 10Mbit bursting.

I am the revenue source for my ISP and they should have a vested interest in providing me the service level I'm paying for.

EDIT: Also, the conflict of interest with an ISP also being a content provider should be legislated away. We have a horrible situation right now and it's only going to get worse.

I don't think your provider has guaranteed you 10Mbit constant bandwidth to every host on the internet. Would you demand 10Mbit constant bandwidth to the Galapagos islands? Or to some site getting slashdotted?

If you check your contract, I suspect 10Mbit or whatever is described as a maximum possible speed which can be affected by a variety of conditions and is not guaranteed.

If you read the post I was replying to, he was alluding to customers pulling a sustained 10Mbit on the connection during certain hours and that being justification for shenanigans. I was simply saying if I pay for a full 10Mbit (not bursted 10Mbit) then I expect to be able to saturate that and not have the ISP try to justify throttling some of the traffic or charging a downstream source for the 'right' to send me data over a connection I'm already paying for.

I have a theory... (based on personal experience). The issue for many is (was) with Apple TV, and not Verizon.

I updated my Apple TV in november (shortly after Netflix introduced its SuperHD mode). For a long time after that, Netflix performance was abysmal (dropping down to sub 480p at times, especially during the night time).

I complained to netflix and verizon. The latter suggested I try connecting to netflix on a PC connected to the modem. I did and experienced no problems. No problems on my Android device or chromecast either. Only my Apple TV box had issues. Ultimately I found a thread on the forums which suggested a hard boot (power down) of the Apple TV device. Once I did that, the problems went away. If you have Verizon and use Apple TV to watch netflix, give it a shot.

I would love to see ars/netflix display a bandwidth graph by device.

Interesting, but what percentage of people have Apple TV? Is it enough to have that much of an impact?

Note: Edited to remove part that was actually addressed in original post.

Not sure (Apple TV and Verizon I assume).

This article might implies quite a few might: http://www.tuaw.com/2013/07/16/report-half-of-streaming-boxe...

For what it's worth, I've had the same type of issues with my Roku on Comcast. I have multiple Roku boxes in my house and all have the same issue. For the past two or three months I've had buffering issues out of the blue.

Our Rokus all worked great, until one day around 1.5 years ago, something changed. They are only barely functional at this point. Charter cable here.

I have FiOS and use google tv (underrated, but great) and I have been having issues with Netflix recently.

Netflix is unusable on my Apple TV. I can't even get through a 20-minute TV episode without it dropping to sub-480p quality and/or stopping completely. I've never been able to figure it out. I have the latest generation Apple TV updated with the latest software. I have tried rebooting it, etc.

It's only a problem on my Apple TV. Netflix works flawlessly on my Playstation 3, iPad, iPhone, and PC. My ISP is a regional provider called Mediacom.

This is what did the trick for me. Update Apple TV to the latest version, unplug. Unplug the modem. Plug it back in, wait for it to come online. Plug in the Apple TV again. Wait for it to go online. Enjoy.

PS Amazon Prime Instant Video and Vudu have never given me any problems.

We have a 40Mb link from XFinity (Comcast), and we only have problems with Netflix. Amazon prime streaming works great, Hulu worked great when we had it (I couldn't stand paying to watch the same commercial over and over again), HBOGO, XBLN movies, and the assorted Xfinity services all work perfect. It's only Netflix that we have trouble getting a HD stream from.

We use these services on an Xbox, Roku, and Kindle/PC - and Netflix is consistently the slowest of them all, no matter the platform -- sometimes not getting an HD stream at all during playback, but usually only getting HD stream for about 30-40% of playback.

Wow I didn't realize Hulu Plus still showed ads. That's quite annoying.

I had the opposite experience: Netflix was fine on Comcast, Amazon Prime was surprisingly poor.

Lesson: don't get your internet service from a company that also tries to sell you entertainment content. They're going to fuck you over both ways.

Yep. The disadvantage of consolidation in that industry is these big providers have a conflict of interest. In an ideal world, platform providers would be broken up and prevented from selling their own content. That goes for Google too.

Too bad there are no other reasonable options. Not sure what the lesson is there - move, I suppose

Part of this is seasonal traffic trends. Here in the North East when the weather starts getting hostile around November we see a big uptick in bandwidth usage among our customers. Utilization tends to peak in January and starts breaking as soon as the weather warms up. You can see a less dramatic dip on this chart from the same period last year. With the long stretches of cold weather all over the US this year I suspect ISPs all over the country got hit hard. It's also the season for many millions of new devices to get unwrapped and turned on.

It really annoys me when people present data on charts and the lower range is not zero. This article shows a speed comparison for Verizon and Comcast. The Verizon chart ranges from 1.8Mbps to 2.3Mbps. The Comcast chart goes from 1.4Mbps to 2.2Mbps. This is lazy at best, disingenuous at worst.

People, stop doing this! Start a chart like this at zero. Only if that is meaningless, do something different (it's not here). But is really that hard to at least be consistent?

I'm on Verizon FiOS in Pennsylvania, with the standard residential 15/5 service, it's 10:44PM, and Netflix is streaming at a steady 3-4Mb/s. I guess I'm lucky?

http://i.imgur.com/akih9HA.png

You're on FIOS and you're getting 3-4MB/s and you're feeling lucky?

How sad.

It's higher than Google Fiber's average speed per the article. Netflix doesn't saturate connections, it doesn't take that much to stream an HD movie. They also don't stream "Super HD" to browsers which is the only way to get a higher bitrate than that.

He only has 15 Mbit service. He's actually getting ~2x what he's paying for.

3Mbps is 0.20x (of 15Mbps) what he is paying for.

Bits != Bytes.

Mbps == Megabits/sec

MBps == Megabytes/sec

>I'm on Verizon FiOS in Pennsylvania, with the standard residential 15/5 service, it's 10:44PM, and Netflix is streaming at a steady 3-4Mb/s. I guess I'm lucky?

He wrote bits, he's only using a 5th of his pipe.

Corroborated by the png he linked.

dangrossman said Mb/s samstave mistakenly read that as MB/s

Yeah, I thought he was referring to his whole connection...

And while his link is 15/5 -- I still thin thats slow for FiOS -- but I had misread his comment..... I may have also been drinking.

Just because someone has a 15Mbps connection doesn't mean Netflix will eat it all up - Netflix will use the minimum bandwidth needed to stream the quality of video being watched. Sounds like it is working great in this case.

Are we assuming one Netflix stream requires the full pipe?

I'm on FIOS; while the kids are watching Netflix in the living room, the wife is watching Netflix in the bedroom, and I'm playing a multiplayer game with no lag on the desktop. Seems everything works sharing the pipe just fine.

What if we started netneutrality.js? Something that shows a banner on sites for people using Verizon/Comcast. Perhaps that will raise more awareness for the issue?

It's not an awareness problem, it's a monopoly problem.

For example, I live in an affluent suburb of Philadelphia. The only ISPs we have are Verizon and Comcast.

The banner might as well read "Pick Your Poison".

It would be interesting if such a script used WebRTC to collect data from many users in a distributed manner and stored the information in a DHT that researchers could query for data to determine which providers are and are not being a good netizens.

It's worth nothing that the FCC has a speed test app which is trying to gather exactly that kind of data:

http://www.fcc.gov/document/fcc-unveils-mobile-broadband-spe...

(source available here: https://github.com/FCC/mobile-mba-androidapp ).

It's currently more focused on mobile networks, but it also gathers Wi-Fi performance data as well. And given recent developments, I wouldn't be surprised to see the FCC broaden that focus.

This sort of exists in the form of a hardware box which you add to your network: http://www.measuringbroadbandamerica.com/fixed-broadband/how...

I have residential fiber from a local isp @100mbps up & down, and have definitely noticed netflix has been horrible for a few weeks (appletv).

I took a look at my airport stats, and it seems the atv had connected at 1mbps to the AP, which would certainly cause problems. A reboot of the ap & the atv resolved the issue.

If you have apple gear, you can see the data rate for your device by option-clicking the AP in airport assistant.

Is there any way to (do these comparisons) differentiate between Comcast Residential & Business?

I cancelled Netflix as a Charter customer because performance was terrible, while Amazon Prime works fine.

Don't Amazon and Netflix both stream from the AWS cloud?

Reply on news.ycombinator.com