Netflix replies to Verizon cease and desist letter
cnbc.com
cnbc.com
"This is about consumers not getting what they paid for from their broadband provider. We are trying to provide more transparency, just like we do with the ISP Speed Index, and Verizon is trying to shut down that discussion."
If Netflix is 35% of global internet traffic at peak capacity (as per the Akamai CEO's comments at a number of events), is it really fair to treat them like every other company? That is to say, if Netflix is really the sole driver of Network upgrades, why does Verizon have to subsidize their costs of business?
I'm not taking the stance that this is correct, only querying as to why the dynamics here are such that we automatically assume Netflix is in the right. Guilt tripping Verizon into adding more routers is a major net positive to Netflix's business.
Yes, ISP customers are paying for access but the business reality is that the ARPU per subscriber is decreasing every year at a rate lower than additional subscriber acquisition can sate. As Wall Street demands growth, it has to come from somewhere, and since Carriers are not able to sell consumers additional services (no matter how hard they try), they need to find another set of wallets (content providers). They're not utilities and have a profit motive, right?
Again, I'm not saying this is correct, simply trying to add another viewpoint to the conversation.
Customers are requesting that data, through a pipe which they have already paid for.
It is unfair for Verizon, or any other ISP, to ask to be paid twice for the same bits. What bits I am asking for, as a customer, shouldn't make a damn bit of difference.
To my understanding, the upstream ends are linked largely thanks to peering, which is based on mutual benefit. At the upstream level, nobody charges other providers because everybody is interacting with everybody- sort of like, "I bought this round, but I know you'll be buying the next" so you don't worry about balancing the books.
Then Netflix comes along, chews up 35% of internet traffic, and basically takes a disproportionate piece of the pie. If I've got everything in my head right, this is against the spirit of how peering is supposed to work, and so it doesn't surprise me if spats start to develop upstream.
If Verizon is unhappy because they have to develop their last-mile network to meet promised speeds, that's Verizon's problem. But if Verizon is unhappy because the upstream is becoming distorted and they are becoming beholden to just one peer (Netflix) instead of an environment of free, balanced exchange- maybe that's not Verizon's problem.
A thought experiment: Let's pretend it's not Netflix for a moment. Let's pretend there's a bunch of competing streaming media services which happen to take up, combined, the same amount of bandwidth that Netflix does.
In this case, Verizon would still have the same problem (if you buy that they have a problem at all, which I personally do not), except they couldn't play at being mafiosos, holding one in particular to what would be a protection racket in any other context since the load is the same amount, just distributed among many much smaller players.
What would they do in this case?
The fact that it's one large company gives them the ease of threate^H^H^H^H^H^H leveragi^H^H^H^H^H^H^H^H negotiating with one entity instead of upgrading their damn network like they're expected to do.
If Comcast wants to offer their customers video over a traditional cable network, they pay content providers.
Why should the Internet be different? Shouldn't ISPs be paying people who provide the content their customers are buying?
The Internet is different in many ways, but notably because cable companies aren't doing peering and don't have to deal with the complex settlement of international data trafficking.
The reason ISPs can charge both the customers and the content providers is because we have no other choice. Where I live I have a choice of two providers: Comcast or Verizon. My choices are: use one of these two, or have no Internet. Not much of a choice.
It's possible that if Netflix didn't exist someone would fill that vacancy but it's also possible that the aggregate total volume of bits transferred during streaming would be lower if Netflix did not exist.
ISPs typically possess less total bandwidth then they sell. The problem is that the typical users usage is rising faster than the ISPs are willing to handle. If I pay for 50Mb/s, and the other end of my communication has enough bandwidth to provide it, then I should be able to get to that speed.
As far as I am aware, there has been no accusations that Netflix does not have enough bandwidth to supply their customers, but that the intermediaries have oversold their networks to the point where they cannot deliver on their promises.
If they choose to reduce that speed for any reason, or throttle any kind of connection they are entitled to do that but they can't complain when someone else says that is what they are doing.
Personally, I think given how much public money has gone into infrastructure and the amount of money I pay in taxes it seems insane for me to first pay for the ISP's infrastructure with taxes then have to pay them again. So while they have a right to throttle, they don't have a right to throttle my connection on infrastructure that they did not pay 100% for. Which probably accounts for the majority of their infrastructure.
It's also very hard to prove that Verizon is throttling, versus, say, simply physically underprovisioning the connection.
With respect to public money, I think you've hit the nail on the head. The crux of the Net Neutrality debate, to my mind, is whether these networks are public or private. If they're private, Net Neutrality is bullshit. If they're public, it's obvious.
It is in consumer interest for the networks to be public, but the same could be said about the socialization of any number of industries. I'm constantly fascinated by the way folks consider socializing the telecom industry to be incredibly reasonable and socializing, say, the healthcare industry to be insanity.
Edit: Maybe I need to tag all of these posts with #DevilsAdvocate to avoid being downvoted into oblivion.
Can't I want both to be socialized?
* Water
* Sewer
* Power
* Internet
* Healthcare
* Fire
* Police
* Roads
* Education (added per /u/CodeMage)
Feel free to chime in if you can add anything I forgot.
Cheers.
I'll note that we should have explicit public subsidies for Internet and that I would much prefer to have public networks with private operators (to my mind that's heaven for competitive markets but politically impossible in the US).
Orphanages
Mental institutions
Defense
Still a good list to have around so we can argue about where the line should be drawn. Kudos!
Healthcare is arguably a public good, at least as far as vaccinations and contagious diseases go, since herd immunity is a large way to fight them (especially for contagious diseases for which we have 80%-90% effective vaccines). Not so sure about other illnesses (like cancer) though. I'm personally for socialized medicine for other reasons (humanitarian), but I agree that the natural monopoly argument is not applicable there.
Education is tricky, and honestly, I don't know any better than the next guy how to solve the issue. What's important to one parent may not be important to another. Second language? Art? Music? Sports? Recess? Lectures? History? Religion? Critical thinking? What outcomes are important? Good test scores? Writing?
Health care has become a perverse industry because its market properties were destroyed not by government, but by malicious corporate conglomerations out to maximize their own profit at the expense of human life. People forget that one of the purposes is government is to prevent private organizations from growing too powerful and interfering with the general welfare of the people, because the narrative is rarely framed this way, but it's a very important governmental function. I don't think socialization is the answer, but something has to be done.
I think health care needs to be brought back down to earth, and we should outlaw all non-catastrophic health insurance products. The insurance racket has completely destroyed that market. There are other elements that have caused the health market to get completely out of whack, but insurance is at the very core of the issue, and it has to go away.
When almost half of the country's bankruptcies are due to health care debt, simply sick people trying to stay alive, we have a real social problem that we need to address.
Of course, we did the opposite of this and made it illegal not to have health insurance, only further inflating the size of the monstrosity that has leeched and destroyed the system in the first place. I don't believe writing a blank check, like the other countries have done, is the way out. I believe the way out is going back to a what-you-see-is-what-you-get approach, and forcing medical providers to charge rates people can actually afford, as their business will immediately plummet if they don't seriously rework their billing structure to become affordable again since people will actually be paying it now.
Most European countries offer health insurance, and total healthcare costs are much lower (compared to the GDP), so I don't think insurance is the problem (also, AFAIK most countries are bankrupt because of bad bank loans, falling GDP/taxes, and social payments (pensions), not health care costs).
The real problem is that doctors get paid when people are sick, so they are not incentivized to heal them, long-term (without regulation). If doctors were paid when the patients were healthy, you wouldn't have this problem. But I don't know how to implement that.
Insurance, classically understood, is risk sharing, not cost sharing. In other words taking it from a "not everyone will use this service" risk sharing model to "everyone will use this at some point" cost sharing model is what causes the problem of hidden pricing.
You can have it both ways (risk sharing and visible prices) because most healthcare costs aren't unforeseen. In other words, the number of accidents where the hospital needs to be relatively sure insurance is involved pales in comparison to the cost of routine visits/care which could be paid for out of pocket and prices would then be visible allowing for competition.
However, there are also many medical costs that can be foreseen (even planned for) but are either "beneficial" for the society (e.g. childbirth) or still risky and essentially no-one's fault (e.g. cancer - most people will get it, but some will get it at 50, others at 95). Incidentally, it's cancer and the catastrophic treatments (e.g. heart surgery) that are most expensive. Possibly also because it's in the big-pharma's interest to keep it this way (at least for cancer).
Insurance is absolutely the wrong model for something with this kind of usage pattern. The insurance industry is notorious for its antics precisely because they're one of the only industries that can only make money when they don't provide the service you're paying them for. The fact that medicine is so widely used makes this especially egregious with regard to medical insurances. They'll do anything they can to deny your claim because aside from basically accidentally having a healthy group of people in the insurance pool (due to the way corporate policies work, and the new health legislation), denying claims is how they make money.
Firemen are even more heterogeneous, in that they have arrangements for big fires and catastrophes like tornadoes where adjacent units come to help. After my city got hit by an EF-5 tornado, as I was exiting my trashed apartment complex, there were fire units varying from a small Indian tribe that's right across a state border from us (and the city e.g. treats the sewage of their casino) to one 40 miles away. Next day, the markings on my apartment door were from a fire unit the came from 160 miles away.
What is needed is something like unity of command/hierarchy. Having two or more units responsible for the same territory leads to the sorts of problems you cite, e.g. a friend told me the NYC city police and subway police once got into a big firefight. This would have been in the '70s or earlier, now they've been unified.
For example, here in the UK, we have the formerly nationalised telephone network provider required to allow individual ISPs access to key parts of its infrastructure via local loop unbundling, so those ISPs compete for the contracts with individual customers. And while that infrastructure is indeed common and so in a sense a monopoly at that level, there are also multiple cable companies and mobile networks with completely independent infrastructure all the way down. There have even been a couple of impressive success stories where installing high speed broadband infrastructure was too expensive or risky in a certain geographic area for any of the big commercial providers to take it on but the local community devised their own solution, sometimes even finishing up with a better connection than what most of us get in big cities.
In other news, prices for Internet access in the UK seem to be dramatically cheaper than most places in the US.
This is what I'm advocating for. A public last mile that any service provider can use.
These rents comprise something like 5-10% of city revenue. And so everybody gets what they want:
* The city gets a free 7% budget bump, and doesn't care who pays for it as long as it's too complex to be an election issue
* ISPs get a monopoly, and don't care that they play the scapegoat because consumers have no other choices anyway
* Citizens get better government services at lower tax rates. Sure, they pay for it via a backdoor tax on internet access but nobody really notices that and they direct their anger at the ISP rather than the elected government
Really the only way to throw a wrench in this system is something like Google Fiber. Nobody wants to be the elected official that turned Google away, because that is actually a sound-bytey enough issue that could come up at re-election. And Google, as a company that doesn't really care about being a profitable ISP, will only move in to a city if it gets a pretty good deal on the rents.
Part of my point was that there is no physical monopoly here, even on the wires.
BT (formerly the national telephone provider) has one set of infrastructure.
Cable companies have their own, literally installing a different box in your home with an independent physical connection to their own network.
If you use one of the mobile providers, which is viable for a home connection in the 3-4G era if you're only using a modest amount of data, there is no physical wiring to the home at all.
There are even a few special cases where people have literally built their own infrastructure as a community, only hooking in to some big provider's network higher up the hierarchy, presumably on some sort of customised, one-off deal.
So while we do have local loop unbundling and a shared-last-mile kind of deal, and we do have ISPs competing for that same physical connection, even that is not a monopoly provision.
IMO non-catastrophic insurance should be abolished, the government should overcome the supply chain cycle ("Sorry, I have to charge $500 for this MRI because this machine cost me $4 million") by producing its own functional medical equipment and selling it at or under cost, and health providers need to get back to reality and charge face rates that real people can pay directly. And we all need to come to terms with the fact that some of the extravagance probably needs to be toned down.
However people are not willing to accept the consequences of hospitals that can't surge in catastrophes. When an EF-5 tornado hit my city 3 years ago, it took out one of our two hospital complexes. Things got ... rather intense at the other (http://stormdoctor.blogspot.com/2011/06/first-response-mode-...) but it being big, and stocked with otherwise excess inventory of all sorts of things, kept the death toll at ~160 instead of a likely 1,000 or so.
My list would include: non-elective healthcare, food, housing (same as for food), electricity, water, local/regional transportation (e.g. for commuting to work), internet access and possibly basic internet-capable mobile phones.
Basically "natural monopolies" are certainly all candidates for complete socialization.
It doesn't matter either way, in the end the experience is the same.
Is it really the same people arguing for that? First and foremost, the telecom industry is nothing like the healthcare industry, and they shouldn't be reasoned about in the same way. Common carrier status has precise laws and legal meaning surrounding it. Please don't confuse it with socialized medicine.
You lost me there.
"There is only one valid definition of a business purpose: to create a customer." ~Peter Drucker (whose writings contributed to the philosophical and practical foundations of the modern business corporation)
“On the face of it, shareholder value is the dumbest idea in the world. Shareholder value is a result, not a strategy… your main constituencies are your employees, your customers and your products. Managers and investors should not set share price increases as their overarching goal. … Short-term profits should be allied with an increase in the long-term value of a company.” ~Jack Welch (former very successful CEO of GE)
Jack Welch was hugely successful in creating and competing in new markets specifically because competing in saturated markets is not profitable.
Those models are great for industries in the 0-80% saturation growth period, but aren't really applicable to high saturation markets.
The negative shareholder value argument depends on a summary characterization of all investors as people who only care about money. Not all investors are like that. You can create shareholder value out of anything that your shareholders value, which is not to say that "shareholder value" should even be a serious metric. It's just a narrative that anti-corporate people have drummed up.
And yet financial statements are reported in dollar units, not in customer units.
A business is a balancing act of many "should"s and "could"s.
But when your top priority, or god forbid, sole priority, becomes "maximizing" money, versus contributing in a meaningful way to society, your business no longer deserves to exist.
2) Agreed.
3) All of the big Telcos are public, but your point stands.
To be explicit, I wholeheartedly agree with you about humans making pseudo-random decisions. I'm trying to explain the dynamics of the Telco position, from their vantage point because I think it helps the discussion to view things from other angles.
Also, intentionally physically underprovisioning the connection is throttling, plain and simple. You are limiting the amount of traffic that can traverse that network boundary to 10G/40G/Whatever your port speed is. What difference does it make if I do it in software or simply refuse to bind more ports to the team?
Hard to prove in a court of law, perhaps, but the speeds of access to the businesses being extorted can tell a pretty convincing story.
This chart: http://appledailyreport.com/backend/wp-content/uploads/2014/... paints a nice picture of how ISPs want to extort businesses that are in more profitable lines of work: notice how speeds change from before Comcast started trying to extort from Netflix, to during the negotiations, and after Netflix payed their protection money.
Level3's article also called this out: the ISPs are purposefully, physically, underprovisioning their connections, because it gives them leverage over companies like Netflix.
As to why the internet is different: the internet is not like healthcare. Socialized or not-socialized medicine is obviously not that big of a deal to the future of a nation; both the US and Britain were economically competitive while having very different systems.
In contrast, the Internet obviously matters a great deal to the future. And if various players (hollywood; last-mile ISPs) can manage to appoint themselves gatekeepers and engage in rent-seeking behavior on the internet (in the United States, at any rate), well, HN is acutely aware of what that will mean for the future of innovation, and the future of nations as well.
I understand that it would be a negative impactor on the future of nations and innovation; I'm asking if that's enough to actually drive change. My sense is that, historically, with respect to ISP behavior, over time they get what they want. I'm not saying this is right, I'm simply noticing a trend, just as many have noted that less-private social networks win over more privacy focused businesses.
Sounds sufficiently sophisticated.
http://en.wikipedia.org/wiki/Last_mile
"Last Mile" is a common telco industry term. It's not about framing, it's the correct term for the connection. It does not connote any specific value to the connection relative to others, nor is it derogatory. It's simply saying where in the network the connection lives. A "last mile" connection is not an inter-network connection, it's a specific link within the network that your ISP has built from the rest of their network to you, the individual customer.
Last mile is from the POV of the carrier, and implies that something is being delivered to you.
First mile is from the POV of the customer, and implies that this is the first mile of your internet connection.
The direction data is flowing is irrelevant.
How is that viewpoint even remotely considered valid ?
I've yet to meet any carrier who wants competition, for anything. Their ideal world would be that they would decide which phone the customer can have, what limited service the customer can buy, and how much it is, and take-it-or-leave it. That's how AT&T worked, and every major carrier today is a spin-off from AT&T and still thinks pretty much the same way.
The word "promise" is not an accurate reading of the legal concepts at play in bandwidth purchase. Dedicated bandwidth for consumers is expensive and completely unavailable on cellular networks.
Verizon also doesn't have to let iPhones connect to their network and only does so because they benefit financially from the relationship. Trust me, putting a device onto a major cellular network in the US is highly political, full of peril and financial alchemy.
Source: I do lots of telecom stuff specifically around wireless.
Wouldn't intentionally throttling and/or not keeping up your capacity intentionally be a violation of this "best effort" doctrine? Not arguing, honest question.
Ideally, you're right -- if netflix (and internet video in general) drives growth in internet traffic, maybe it isn't fair to require service providers to upgrade their service out of hand.
But then I think that rule primarily applies for a free market. I'd argue that internet providers are not currently participating in a free market and it causes real problems. Without a free market... companies need more supervision.
I'll put it this way: 2 competing providers. Average internet usage keeps rising exponentially. People want faster internet. The providers either lower prices or upgrade infrastructure. Netflix is not forcing them to do anything. But then it's probably reasonable for Netflix to say "Your connection is crappy because of your internet bandwidth, it's not us..." But hey, consumers can choose.
Problem here is... what are their choices? Even just one other company participating in the market that can make a similar offering (eg cable speeds) would change the whole conversation I think. Suddenly they'd be working harder to keep your business (or take you from the competitor) and they'd stop talking about more and more ways of screwing their customer and taking advantage of de facto monopoly status.
The incentives are all messed up and it encourages gouging instead of innovation.
Are the networks public or private and why can't we decide?
I don't see that argument going very far. Netflix has an ISP and pays for their own bandwidth. Why should they pay twice for their bandwidth?
If verizon has a problem with the amount of data their customers are requesting from netflix they could always disallow that or institute data caps etc.
If the ISP's customers are making requests to Netflix' servers, then yes.
ISPs have been miss-selling Internet connectivity for many years - "UNLIMITED!!" "FAST!!" "CHEAP!!" - and now they ( and their customers) are reaping that harvest.
Interesting question. I guess from my perspective I buy broadband from AT&T and they tell me under no uncertain terms that I have a cap at x amount of gigabytes. As a customer I expect to use all of that.
>Guilt tripping Verizon into adding more routers is a major net positive to Netflix's business.
As a customer it shocks me they aren't doing so. Yesterday it was Netflix, today its every kid with an Xbox1 or PS4 buying games digitally as buying physical media is out of style. What's a typical game nowadays? 20gb? That's the equivalant of many netflix movies right there, all at once.
What does tomorrow hold? Why aren't they prepared for this?
Customer revenue is decreasing and because the market is saturated they cannot add enough customers to combat the atrophy of revenue. As shareholders expect growth every year, ISPs are forced to look for new revenue streams.
If you don't think they should be doing this as a private company, reclassify them as a utility. Otherwise shareholder demand is what gets satisfied.
Carriers have very specific equipment amortization patterns in their heads and they are financial models that banks accept. If carriers have to change their amortization patterns (more routers, faster) they have to change their financing which damages the value of the stock.
Sure we can do this, but what does "fair" mean here? I'm trying to say that the argument is not as black and white as people make it out to be.
Welcome to the world of business. Change is constant, you can't just tell your customers, "Whoops we didn't plan for that."
I also disagree that its as hopeless as you make it out to be (in many nations 100mbps or even 1gbps unlimited internet is trivial to get). They can raise prices on customers, lower caps, throttle netflix users, etc if they're in such financial dire straits. Their competitors will love this as customers will migrate to them. Their competitors may have a more nimble management structure and can provide the data at this level.
What I don't get, assuming they really are in dire straits and not trying to monetize artificial scarcity, is why don't they just push out a notice, "Due to network limitations, we are throttling streaming video users to 4mbps which is enough for medium quality 720p streaming. The investment to support 1080p streaming is impossible for us. Love Verizon."
Actually, I know why they won't do this, because Comcast will pay for the router upgrades and steal all their customers once they find out. So they go around their customers' backs and try dirty dealings with Netflix and others. These guys being duopolies in most markets (substitute AT&T with Verizon in some states) just decided to not compete and simply change the rules at the FCC, which is pretty much a textbook description of collusion.
This is the biggest part I think people are missing- they are not thinking about the upstream end of the pipe. Maybe it's the same story even when you do consider the upstream, but I'd at least like people to acknowledge that aspect.
Yes it is.
If you solve their problems in the general case, then that solution will also work for the next big user, and for the small users as well. If you special-case handling for them, then it will be harder for the next new entrant (who could even be a Netflix-upending upstart) to become established.
They do use more resources, there is no question about that, but they also have higher costs to their ISPs. This cost could be mitigated by them running their own internal network and peering directly with other backbone providers (they already do this to some extent). But in an architectural sense that option is available to anyone (in practice, of course, nobody is doing BGP for their home network).
If you believe in the original promise of the end to end network (a promise which is currently lying in the street gasping for breath) then you don't want netflix special cased.
It's no surprise that Verizon (with POTS roots) and Comcast (hub and spoke cable provider) don't have end-to-end in their DNA, but rather a tollbooth model of the world. It doesn't mean we have to go along with it.
The ISPs are punishing consumers for their choices - it's just ridiculous.
There's a conflict of interest: Netflix has a vested interested in provided lightly-compressed, very-high-resolution, bandwidth-hungry movies to its subscribers, especially if it obtains 100% free peering that it demands, as bandwidth would be free for them. On the other hand, ISPs are forced to upgrade network any time Netflix unilaterally decides to increase the bitrate of the movies, without this bringing any benefit to their bottom line.
For instance, if Netflix halved the bitrate of all their movies, subscribers will complain, but surely they would be able to deliver them without any buffering to everybody, and at the same time to keep free peering agreement and whatnot; they would go from 35% of total US bandwidth to 17%, and ISPs would be satisfying their customers' choice.
So where do you draw the line? Surely Netflix can increase the bitrate at will, at any time; should they be "allowed" to do it any time they want, while keeping a free peering agreement? Should the ISPs be forced to upgrade their network every time?
Come on, they have been doing this since the internet started. And in the beginning of internet some services were hugely popular as well. Keep in mind that upgrading the network costs money, but once the infrastructure is in place, moving bits is really cheap.
If Verizon tripled its total bandwidth today, how long before Netflix begins raising the bitrate because "the Internet has evolved and we can now deliver bluray-quality streaming at the same $9/mo?". And then they fill up pipes again.
The whole issue here is cause by the impedance between the cost structure (pay by the gig transferred) and the price structure (pay by the peak speed, and I approximate the average gig transferred).
In a perfect utopic world where everybody was willing to pay by the gig transferred as that is the real cost factor, we wouldn't be here having this discussion.
> moving bits is really cheap.
Moving bits is cheap WITHIN the ISP network, but once you go outside they have a cost (depending on the peering agreements).
Forced to upgrade? Maybe not.
OTOH, should people be made aware that the ISPs failure to do so is a contributing factor to the unsatisfactory results they are getting so that they can make informed market (and political, insofar as government policy impacts the competitive landscape) decisions? Absolutely.
This is an excellent point. Why doesn't NetFlix at very least offer the ability for their users to queue content for download / local caching? If customers had a slow Internet connection at least they could plan ahead to avoid suffering streaming problems.
Ironically isn't streaming-only perhaps a violation of net neutrality? NetFlix artificially limits the speeds my ISP can deliver by only offering streaming. I have a 100Mbit/sec connection yet NetFlix chooses to limit me to streaming speeds.
No, not at all.
Because this would almost certainly result in the content industry screaming bloody murder.
As others have mentioned, the ISPs woefully miscalculated the growth of the internet and consumer bandwidth usage and as is the case with capitalism, those mistakes should cost them.
When you install a server in a colo, you don't get free transit. You find a transit provider (like Level3, Cogent, or NTT) and you pay them for bandwidth. You can understand this very intuitively as most hosting services (like AWS) will charge you for bandwidth.
As a service provider (like Weebly), you are paying your own transit provider for a certain amount of bandwidth, and they are covering the costs of delivering that bandwidth to (usually) anybody globally and the costs of maintaining their network.
If a network gets large enough, they may often negotiate peering agreements with other networks. So Level3 and Cogent may have a peering agreement, or Level3 and Comcast may have a peering agreement. These are generally set up when two networks deliver roughly the same amount of traffic to each other, or it's considered beneficial to both parties to get the networks closer, and it simplifies the working relationship.
These peering agreements can often become contentious when they become imbalanced. Very early on, I remember a fight Cogent and France Telecom had over peering -- FT felt like they were getting the raw end of the deal, and de-peered with Cogent. Cogent refused to pay another transit provider for access to FT's network and so traffic from the Cogent network to FT would drop. They resolved about a week later, with lots of unhappy customers caught in the middle.
As you can see, it's simply not the case that (a) service providers never pay for bandwidth because the end-user already pays for bandwidth and (b) peering / free transit is guaranteed.
Maybe I'm not understanding the nuances here, but it seems to me like Netflix is taking an ideological stance to hide behind the fact that they are trying to negotiate a better business agreement for themselves.
The latter is something that more people ought to be thinking about, consumers using Netflix are not just consuming more peak bandwidth, they're competing with the video offerings of all of the big last mile providers.
Although. AT&T doesn't actually offer U-verse video service to very many of their subscribers; don't know about Verizon's FiOS vs. plain DSL offerings (except that they're shedding small population states and markets as fast as they can), but the others are cable companies so they obviously do.
Peering agreements should be about saving costs on both sides, but if Netflix is willing to pay Level3 for sufficient bandwidth then Verizon should have the ability to handle that for their users.
That's how it's always worked. These fights and skirmishes over peering agreements happen often. That's the reality of how things have always been up until today, it's not a change in the status quo.
I also think that there these peerage agreement create an untruthful dynamic with their customers, the point about transparency is valid. People think they are buying "internet" access at a certain speed, not different speeds for different sites. If Verizon thinks it's OK for the speed of their customers' connection to be dependant upon "interconnect agreements" or whatever, then letting their customers know about this isn't defamation. If they do have a problem with their customers knowing, then maybe they should be making sure their customers get what they expect (or expect what they get).
In any case, I don't see how this is foul play on Netflix' part. As long as the message they display is true, why should't they be able to display it? Even if the message openly advised their customers to switch away from Verizon, I don't see a foul.
I don't see why not. I think it might make more sense to think of it as streaming video, the category of goods, rather than Netflix, the company. So 35% of global internet traffic is people streaming videos. And who cares that it's video; from Comcast's or Verizon's perspective, it's just an awful lot of bytes. So if the issue is the amount of traffic, it's just Verizon complaining that we're using too much bandwidth (yes, us; we're the ones watching Netflix). I get that's expensive to deliver all those bytes to my home, but that's what precisely what I'm paying for. That a huge number of those bytes comes from the same originator shouldn't enter into it. It's just a site on the internet.
EDIT: Also, I wanted to add, that all of the traffic comes from the same source actually helps Verizon from a practical standpoint, since it can solve a lot of it with only a small number of interconnects. Compare to a more fragmented market, where Verizon might have to make specific peering provisions for a bunch of different 5%-market-share content providers to fulfill its customer obligations.
> Guilt tripping Verizon into adding more routers is a major net positive to Netflix's business.
All sorts of businesses benefit from the ecosystem of other businesses around them. If I run a store in rural Iowa and Acme Co opens a huge factory there, then I stand to make a bunch of money from all these newly employed customers. Should I subsidize Acme? [1] After all, hiring people is expensive. If I open a hardware store near a bunch of planned construction, should I pay those real estate developers? You can expand this to the whole economy being basically just a set of interdependent positive externalities. Maybe Verizon should pay Netflix for making their service more valuable?
Or we can use the simpler model, in which you get to charge your customers. Verizon charges us for delivering the content we request. Netflix charges us for the content itself. Cogent (or whomever) charges Netflix for its bandwidth. Everything works.
> They're not utilities and have a profit motive, right?
They could charge more, and they could even charge per unit bandwidth (or tiered bandwidth) if they wanted. "My business isn't working that well" is not a good excuse to hold your subscribers hostage. You could equally use "Wall Street demands growth!" to defend Comcast's new practice of robbing banks.
Imagine I bought a popular TV set in 1980 to watch broadcast content, but after I got it home, the TV maker remotely activated a previously undisclosed filter that would only show certain channels. Then they went to ABC, NBC, and FOX and said, "So guys, you want your content seen? My margins on these TVs keep going down, so somebody's gotta pay." Not cool, right?
EDIT: Another point, probably the most important. The "problem" here is ridiculous: it's that demand for Verizon's product (i.e. internet access for end-users) is increasing a lot because it is becoming more useful to its customers. That's what every business wants! "We sell internet access and everyone is now using so much more internet! Whatever shall we do?" The idea that Verizon is up against some sort of profitability wall because of how popular its product is becoming and thus needs to charge the companies responsible for that increased popularity...it's simply not credible. Not sure why I didn't think of this point earlier.
Edit: s/Comcast/Verizon, since that's specifically who we're talking about.
[1] This actually does happen in the form of tax incentives for companies opening factories. It's a bit messed up.
Then Netflix comes along, and suddenly 35% of Comcast's customer traffic is all going to a handful of Netflix DSNs? Suddenly Comcast has to redo the whole connection. They have to buy top-shelf hardware just for Netflix.
I don't know if this has happened, but it's an interesting thought experiment to see how "just one company" could place disproportionate burden/cost on the ISP, as compared to the case where traffic travels to & from many different entities.
The Tier 1 always wants to ask for more bits than a lower tier operator. If a lower tier operator asks for more bits and tilts the traffic balance, things get interesting. Cogent is not a tier 1 but slings 35% of the internet around at will.
It does cost a lot of money to route Netflix's bits and the narrative is hugely helpful to Netflix. I'm not saying this is correct, I'm simply trying to point out that Netflix has a ton to gain from carriers upgrading their network connections and routers.
Another way to view this: Netflix is demanding huge network upgrades without any additional revenue source.
Edit: not sure why I'm getting downvotes since silverstorm clarified exactly what I wrote
I am not talking about imbalance in U/D, but rather imbalance in destination (or source). If tremendous amounts of traffic is going to or from just a small handful of nodes, you need tremendous routers & pipes going to those nodes.
You're missing a key element of the situation: Comcast made a deal with Netflix precisely to address this problem. Netflix now has direct connections to Comcast's network, so Netflix traffic does not have to go through a third party. In other words, Comcast now directly controls how they connect to Netflix, how many connection points there are, what the average network distance is between the closest Netflix connection point and their customers, etc. So they have a much cheaper way to even out Netflix traffic: open up more connection points.
(Which, btw, AFAIK they already did as part of the deal; they certainly did not come out of the deal having "just a few spots" where they are connected to Netflix. Netflix basically is their own CDN; they have servers with their content on them all over the Internet. So Comcast has all kinds of ways to shorten the average network distance between each of their customers and the closest Netflix endpoint.)
Sounds to me like just another way of rephrasing "ISP Customers are demanding huge network upgrades without any additional revenue source."
Netflix does not send packets to you unless you request them.
I think a lot of us would have more sympathy with the ISPs' situation if they hadn't consistently and for a very long time been promoting their services as ever more valuable by offering ever higher bandwidth, yet keeping quiet about little things like massive over-subscription and high contention ratios that meant if all their users actually tried to use the service that was purportedly being offered then no-one would get anywhere near the advertised speeds they were paying for.
With the rise of things like streaming video services and Internet telephony, some people are actually trying to pull significant volumes of data down the pipes relative to the advertised size of pipe available. Everyone's known this was coming for a long time, and no-one more so than the ISPs. The way I see it, if the current situation hurts ISPs that failed to invest or set a realistic pricing model, they'll just have to take their medicine and offer a more realistic deal to customers next time (or fail because they can't compete effectively).
Funny, I actually just added an edit to my post above that discusses this. Comcast adds interconnect capacity as needed. So in at least one way, it's actually easier this way, right? Just one peer to make sure they have capacity for.
But either way, it's Comcast's job to run a network that's optimized for its customers needs, not some abstract concept of fairness. And in this hypothetical, it's not on Netflix that streaming lots of content is not compatible with how Comcast happened to build its network. Comcast needs to fix its network so that its customers can get their data.
You can even ask them for a freeish box to run inside your network to reduce requests to their CDNs
But Netflix is not "just a small number of nodes". They're basically their own CDN; they have servers all over the Internet, precisely in order to not have the problem you describe.
What's more, Comcast has a deal with Netflix to directly connect Netflix nodes to Comcast's network; in other words, Comcast directly controls how many paths there are for Netflix content to get into its network. If there aren't enough such paths, they can just open up more.
I guess the core issue here is that, from a customer's point of view, the ISP is selling me inet access, regardless of where the data actually goes, so it is the ISP's responsibility to keep their end of the bargain (to provide inet access). As a customer I should not care how is their topology planned, what kind of equipment they use, etc.
So basically the ISP though that they could offer T amount of bandwidth, advertised it as U (with U > T) because most customer won't use even close to U, and now that more and more users are actually using U amount of bandwidth (or closer to it) the ISP's infrastructure is suffering because they simply where not prepared for it, i.e. they oversold their capacity.
OT: By the way, why downvote the parent? I really thought downvoting is for when the post is off-topic or offensive or something... but more and more I see HN modders downvote because they disagree... really? so you make the post go into an ureadable color (grey) just because you disagree? I don't think it's supposed to be that way, is it?
An airline for example; now everyone actually caches their flight because Google's self-driving cars always make it on time, and then they demand that Google stops bringing passengers on time?
Or a buffet restaurant, since now everyone goes to the gym, they can actually eat a lot more, thus restaurants complain about, either the gym or the pill maker?
I don't know, it would be a strange world to live in if this happened in other industries. Maybe prices would rise too much?
Fast forward 3 years later, you come out with your pill and throw a wrench in the works. Now gyms realize they have to go back to the old pricing model because the economics have changed. Consumers will no doubt become upset at the prospec of having to pay more. Some gyms take a wait and see approach to see what other gyms will do. Your pill started it but really, it's not your fault or your problem.
What should the gyms do?
Be Honest. They can invent a new business model based on transparency. Imagine if price is not fixed but you are charged per use based on the traffic at the moment. It could be similar to what Uber does, price based on demand.
In terms of ISPs, I think there should be a micro transaction exchange where I can buy X minutes of data transfer. If Verizon is clogged I switch my router to connect with ATT etc. All networks should create a single broker device thats given to each consumer.
And then there's this: http://arstechnica.com/information-technology/2014/04/bittor...
"[W]hen we ask [ISPs] if we too would qualify for no-fee interconnect if we changed our service to upload as much data as we download—thus filling their upstream networks and nearly doubling our total traffic—there is an uncomfortable silence," Netflix CEO Reed Hastings wrote last month. "That's because the ISP argument isn't sensible. Big ISPs aren't paying money to services like online backup that generate more upstream than downstream traffic. Data direction, in other words, has nothing to do with costs."
That's because around the time Comcast was building up their networks customer demand was wildly asymmetrical. People simply download far more than they upload and Comcast wisely dedicated more of it's bandwidth to downloads.
At the time all cable companies were were building up their data network, they were building that network over top of their existing video distribution network. That network was designed [ages prior to this point] to have tiny, tiny, upstream channels for use by system control and maintenance devices, and huge downstream channels to shove 100+ analog TV channels. Because it would have cost a metric shitload to replace all of that hardware, the cable companies were forced to give their data subscribers pretty large downstream links and relatively meager upstream links.[0]
Frankly, the nerds of the day (remember, this is the mid-to-late 1990's) would all have been running home servers, and would have been demanding a symmetric connection so that they would have a substantially cheaper alternative to ISDN.
(Note that nerds that remember those days consider asymmetric home Internet connections to have been more harmful than the Eternal September.)
[0] Note that this property is pretty meaningless in an HFC network where the cable company could run fiber directly to the home, but chooses not to. [See also ATT U-Verse.]
Consumer ISPs have never expected symmetric bandwidth. If they had, they would sell us 50/50 connections instead of 50/10 or 50/2.
The only way to incentivize a company to provide the best service possible is to tie the use of the service to its cost. I wouldn't mind paying, say $2/GB for service if it meant that Comcast's incentives were in line.
That would even have an external benefit on creating competition among streaming companies to have the best compression algorithms. Efficiency and speed become directly correlated to the service and its value.
That's a ridiculously high price. You'd pay $2 for every hour of streaming-quality HD video watched, quite a bit more for download-quality HD.
My main point was that this charging scheme incentivizes companies to provide better service, not set up monopolies or short change you service.
100 gb. / month (included) for 47,5 € monthly (65 $). Extra bandwidth @ 0,5 € (0,7 $) / GB, so it's an additional 1,5 € extra for your ISP, if you want to watch a netflix HD movie (2,3 GB).
Don't worry, i know i'm getting bend over :)
Personally, I'd really love to see Netflix start using encrypted peer-to-peer to distribute the bandwidth. See if Verizon would be willing to cripple all VPN and HTTPS users to try and shake down Netflix.
However, I don't think it follows that Netflix must be treated any differently. The customers are the ones requesting the data. To the extent that anyone (once at capacity) should bear a cost, it should be the requester -- either through a higher fee or a smaller fraction of the pipe.
In practice, that will mean "the more Netflix [or anything] you use, the slower your service". But you shouldn't "short-circuit" the process and just start punishing Netflix's packets directly just for being Netflix. Rather, it should be "this customer endures this cost [broadly defined] for using this much, regardless of what it was for".
People getting their video from Netflix threatens this business, and it's pretty obvious to me that's one of the reasons we're seeing all this now, especially with Netflix getting big enough in streaming to a) pose a greater threat b) potentially have a lot of voters on their side (particularly relevant because the last mile providers are all government enabled monopolies, cable companies at the municipality level, telcos at the state and Federal level).
I'd really like Netflix to offer their own speedtest site that looks to the network almost exactly like streaming video, so the ISPs couldn't game the measurements.
Because I, a Verizon customer, have paid to access Netflix.
I pay monthly for bandwidth to use as I see fit. It doesn't matter if all of the customers are using their bandwidth to go to one source. It doesn't matter: it's our bandwidth to use how we see fit.
It is Verizon's responsibility to provide us with the service that we are paying for. They advertise a service, they market this service, and they charge us for the service: but then fail to deliver on their promises because they're charging someone else too?
They sell us capped packages and tell us we have a set amount of data. We cannot exceed the data, because the infrastructure can't take it and we have to pay for our usage. We have to pay for our demand, that's the point of caps. Make the users pay for using.
That to me implies that we are paying for a set amount of data! That we are paying for it, not anyone else.
We pay for the upgrades that are necessary to deliver data to our homes. That is the point of a subscription.
This is like saying: all of the telephone customers are calling Pizza Hut, so Pizza Hut needs to pay the phone company to upgrade the switches at the Phone Company. What???
We've already paid for the infrastructure required to deliver the data we're paying for.
Content services should not be harmed for being successful. If the bandwidth customers are desiring certain content providers it is the bandwidth providers responsibility to make sure it is available and fast, per their marketing and products.
And that's in direct competition with these big last mile providers. Cablecos by definition, and the two big telcos have their own FiOS and U-verse "TV"/"cable" video offerings (even if AT&T's coverage is meager, and both stopped their build outs except for contractual requirements when the Great Recession started).
Which provides additional strong motivation to provide less than stellar service when handling Netflix packets.
Instead of fulfilling their obligations to provide advertised bandwidth to their paying customers (who only pay them in order to access the content provided by someone else), the last-mile providers are using their customers as hostages while they negotiate ransom from the content providers. While they also report record profits (sending their customer's payments back to stockholders instead of into the network), because the economics of their business are compelling on both the consumer side (low expectations, minimal support costs, essentially no competition) and on the technology side (network technologies increase capacity-per-$ as fast or faster than traffic grows).
Oh, also, 99% of the last-mile providers operate a sanctioned monopoly that has let them build and grow their networks with essentially guaranteed income from rate-payers and essentially no competition. Hence adding Internet is relatively low-cost and highly-profitable, but apparently not profitable enough.
Because that's Verizon's business: to provide internet. That's not up to Netflix, who is an internet-based content provider. If Verizon wants to charge the fuck out of their customers to beef up their network so they are the best Netflix streamer out there, then they're free to do that.
People want Netflix because Netflix is doing its job. People can't get Netflix because Verizon's not doing its job. Simple as that.
To put it another way, if Netflix was the bottleneck (and thus, the problem), Verizon would only have to show that their bandwidth from Netflix is bottlenecked by Netflix' lack of serving capacity. Of course they can't show that, because that's not the case, and thus not the problem.
It's not that Netflix is 35% of global internet traffic. 100% of my ISPs "global bandwidth" should be content of the end users choosing (usually requests). The user is paying for the right to request this content, and have it delivered. Wether its Netflix, gaming, news, poltiics or Zombo.com doesn't enter into it. At 35% though, it might make sense for Netflix to enter the ISP business.
This framing is a tool that ISPs and their supporters use to excuse charging twice for the same service, and more importantly, controlling users access to content through discrimination.
Thanks again for the writing prompt though.
But to answer your question: I am already paying my ISP to deliver all packets I request to me, up to certain limits that my ISP agreed to when they started my service. If my ISP is failing to live up to that bargain, I'd like to know.
Also, the "poor Verizon, helpless in the hands of Wall Street" thing verges on obscene. That they are in a position to shake down people for more money does not justify them doing so. If they don't want to be in the ISP business, they can get out of it. They are also welcome to find new ways to create value. But running a protection racket is not in any way morally defensible.
Verizon isn't some kind of customer owned co-op. It is a profitable for-profit business that has (over)sold a service. They need to deliver that service, not complain when people use it.
I would entertain your argument only if there was consumer choice. In such case we could perhaps experiment with different business models where content provider subsidizes last mile. Without it, consumers are essentially hostage to a monopoly, which is unacceptable.
Well, actually, none of my money goes to Verizon. My provider manages to avoid all warnings from Netflix and the graphs from Google show basically 100% of users getting the HD stream. They give me a legitimate 100mbps symmetric (well, almost: http://www.speedtest.net/my-result/3546047591) and I'm paying under $40/mo.
If my niche provider can offer that service, Verizon can offer that at scale. The fact that their service is a) worse and b) more expensive means either a) they're less competent in running the business or b) they're overcharging their customers. Either way, I have zero sympathy for them. They're a network provider and people are paying them to connect to the internet. People aren't paying them to connect to the subset of the internet that's willing to pay Verizon. It's already bad enough that TV viewers are seen as the product rather than the customer. Verizon's stance here is, once again, treating their subscribers as a product to be leveraged rather than a customer who's interests matter. That they can do it is due to their mono/duo-poly that keeps customers from having a choice of providers. But that doesn't make it any less offensive.
Basically, it boils down to this: if the service costs you more to provide, then charge more to your customers. You can even have a streaming plan that charges customers who stream more than customers who don't. But treating your subscribers as a resource that can be pimped out to all the services on the internet is offensive.
They have subscribers because those subscribers want to use stuff like NetFlix.
Take away all the good 'content' and why bother having an ISP? Mobile phone can check emails too.
Verizon is paying to provide its own customers with what they want (speed that enables access to Netflix and other entertainment), not to subsidize Netflix.
Netflix pays its for its own high-speed internet access already. Verizon offers high-speed access and then doesn't pay for it. Therein lies the conflict.
In other words, the ISPs are doing a bad job of running their businesses, but rather than do a better job, they want others to pay them more without offering any additional value.
Netflix isn't the driver, the Verizon customers who want to use netflix are the driver. They're paying Verizon to get them access to cool stuff like Netflix.
It sounds to me like those customers are not getting the service they expect.
However, I feel that this issue is one of misrepresentation. Netflix wants Verizon to be a common carrier, and have its revenue increase linearly with the amount of data it transfers. In that case, Verizon would happily increase their revenue by providing actually faster Internet, because customers would pay more per day if they downloaded more per day, since they'd then pay a fixed amount per GigaBit.
On the other hand, I don't understand what Verizon wants to be. I feel like the type of company it wants to be is only sustainable in the very short run, highly dependent on other companies such as Netflix, and incapable of good PR because what clients want, and what they subscribe for, is impossible to deliver (the amount of data clients want to transfer is always above whatever they can promise to offer). That market positioning is tolerable for instant Wall Street growth, but it is detrimental to customers, to website owners, to peering partners, and to the long-term future of the company, were a reasonable competitor to enter the market.
I also feel there could be a middle ground, such as Verizon asking customers from every city to tip in for a massive city-wide network upgrade. That would be good for PR and customers would happily donate. It seems fitting, because from what I understand, the real money eater is a one-off upgrade; it doesn't increase maintenance costs much, if at all.
You buy an Internet connection from an Internet service provider to consume the Internet. Until the word Internet is dropped from “ISP”, that's the service they sell.
They have no more rationale to charge Netflix to “access” Internet service purchasers than they have to charge your grandmother to deliver you her email.
And about that profit motive, cable already raises prices at 4x consumer price inflation:
“Since the FCC began compiling these reports 19 years ago, the cost of basic, expanded basic and the next-most-popular tier of service have risen at average annual clips of 4.3 percent, 6.1 percent and 5.1 percent, respectively. Inflation has grown at an average annual pace of just 2.4 percent over that time period.”
http://www.siliconbeat.com/2014/05/19/pay-tv-prices-continue...
Because Netflix has a higher brand affinity and customers of Verizon will be outraged due to what they perceive is shitty customer service buy a quasi monopoly.
The consequences is damage to Verizon's business because customers would switch if they could, but they can't.
Verizon doesn't HAVE to subsidize Netflix's business but they should if they don't want a legion of pissed off Netflix customers who feel trapped by their ISP.
This is supremely dangerous to Verizon.
Taking the US political area I know best, gun owning households are a majority in the US, but of course those who vote, vote first and foremost on this issue, let alone be politically active are the usual successively smaller fractions of that huge number.
And starting in the mid-80s we've been able to move mountains (details on request). The last thing the big incumbent landline last mile providers want is a large percentage of the population getting pissed off and doing the above, voting when they might not otherwise bother, especially in state and local elections, voting on this issue, and becoming politically active on it.
I think the incumbents' behavior is so outrageous that this is eventually inevitable, Netflix's PR attack, in part enabled by their being so big, has the potential to substantially move that day of reckoning forward, and is therefore probably the biggest thing to happen in the net neutrality fight since AT&T's Edward Whitacre opened his foolish mouth and really got the issue started 8 years ago.
ARPU is going up while Verizon's bandwidth costs are going down (peer with Netflix/Google/YouTube/Facebook and a huge chunk of your traffic is taken care of).
http://newscenter.verizon.com/corporate/news-articles/2013/0...
"FiOS revenues grew 14.7 percent, to $2.7 billion in second-quarter 2013, compared with $2.4 billion in second-quarter 2012."
vs.
"Verizon added 161,000 net new FiOS Internet connections and 140,000 net new FiOS Video connections in second-quarter 2013. Verizon had a total of 5.8 million FiOS Internet and 5.0 million FiOS Video connections at the end of the quarter, representing year-over-year increases of 12.2 percent and 12.6 percent, respectively."
(Revenue growing faster than new customer growth and a very high ARPU.)
Also, I find it interesting that netflix charges ~10-15 bucks a month, while the average internet only service plan in the 40-50$ range per month. This is a guess, but I would also say there are more Comcast/Verizon/Time Warner customers than netflix customers. The amount of money these large service providers rake in must be an order of magnitude higher than Netflix.
To me, this money that the big ISP's are trying to get from Netflix looks like some sort of fee applied for forcing them to actually provide the service they promise to their customers.
The message is "The Verizon Network is Crowded Right Now." not "Verizon is Deliberately Slowing This Right Now." (or some less loaded version of that)
It's bad PR for the ISPs, but combined with the fact that most people can hardly spell ISP, and that Comcast and their FCC homies keep describing the new toll booth as a fast lane, if I were a nontechnical consumer my first response would be, "What the hell are they waiting for? Put in the fast lane! I wanna watch my damn movies!"
Considering that customer satisfaction with the average telecom company is somewhere between that of the DMV and a crooked tax collector, it plays into the confirmation bias people have that they don't love their ISP, they tolerate their stupidity because its the only way to get online.
Telling a customer they're not getting what they paid for from their service provider is about as plain English as you can get. I daresay we need this sort of rhetoric, and I doubt their response will be levied toward Netflix unless they start getting letters from their ISPs saying "We're throttling you because you're watching Netflix and they won't pay up."
This situation could get ugly, though.
Verizon doesn't want more "cord cutters" of a sort to drop their FiOS video accounts in favor of a la carte offerings from Netflix et. al. AT&T's U-verse isn't as big a thing for them, but they use vicious caps to protect it, and of course all the cable companies are by definition video service providers.
If I pay for a 100Mbit/sec connection from my ISP why doesn't NetFlix send me 100Mbit/sec of data? They choose to limit my speeds (via streaming) and do not offer any options to download or pre-cache video content I plan to watch. Is this a violation of network neutrality? If I ask NetFlix for 4GB of data why are they allowed to limit the speeds?
The average joe isn't outraged enough about net neutrality. If only they'd start doing this to other known bad actors coughcomcastcough, that might just be what the doctor ordered.
I wonder why they ponied up the money to the protection rackets first, and only then started pointing fingers. Maybe the agreement gave them access to some better data? If not, this should have been done months ago!
By the way, don't bother with the comments on the article page unless you want to lose all faith in humanity :(
Verizon: Netflix performance sucks because their traffic is congesting our network!
Netflix: Our performance sucks because their network is congested!
Verizon: OMG WTF you can't say that!
They also control a fair chunk of the overseas connections.
Could also be that the agreement had an anti-defamation clause, and Verizon was counting on that to avoid upgrading their internal network.
> NetFlix will lose huge. Obama is protecting the Taliban now. That's the only reason Silicon Valley soared. Democrats (from Jan 2007) and Obama 'til his Taliban kiss.
If the problem with Comcast is that their network is congested and users can't switch to another provider then it doesn't matter whose fault it is, those people are going to leave regardless.
Note that Netflix is only showing these notices in areas where Verizon has competition to which their customers can switch, which lends a lot of credence to the idea that 'ISP switchability' is a key factor guiding Netflix's behaviour.
One possible reason is that it makes it easy to quantify their economic injury.
Yet when another entity does something as simple as showing an error message that casts them in a negative light, they are willing and able to threaten legal action.
They stretch the truth as far as they can, yet give not an inch when confronted with truths they don't like.
Remove government so we can add customers! But we need government so we can slap Nextflix when they say something we don't like!
I don't have a good answer how to fix it, it's just very plain to see, and frustrating.
Reminds me of the late 90's when my ISP told me I couldn't get cable Internet because I didn't have a 486. I lied and told them I did to have the modem sent to me to hook up myself. It's a computer network, once it's in my house I can expose it as I like.
They probably mean the WiFi built into the cable modem.
No, but all of their features can be enabled/disabled by the ISP remotely.
Similarly to how you only get the ability to tether your data with more expensive plans. It's a completely artificial client-side restriction that can be circumvented by rooting, etc. Not unlike DRM.
People just don't know that you can have wifi for free, especially when you've got adverts from O2 and such that 2gb 3G and " unlimited wifi".
I dunno if it's the same in the US but apparently it's some typically restricted resource that many providers in the UK are lifting the limits off.
No, this isn't too typical. Most people have their own router offering WiFi in front of the cable modem. It wasn't until recently that cable companies started bundling wi-fi and NAT into the modem.
I just felt a disturbance in the force. Damn, that sentence triggers a whole range of evil ideas I hadn't yet thought of.
Hmm, hold on while I check with my ISP whether I can downgrade my WiFi data limit for cheaper service... or would that just be giving them ideas? :p
My general sense is that these types of conversations happen daily at Comcast and such...I've heard too many similar stories over the years.
Tech: "Could you please unplug your computer for 20 seconds and restart it?"
Me: "Okay. <does nothing for 60 seconds>. Okay, it's back"
Tech: "What browser are you using?"
Me: "<Looks at Ubuntu desktop> Internet Explorer 6".I now refuse to subscribe to any ISP who does not offer Linux support, and will in fact check to make sure that Teksavvy (or an equivalent as described above) is available in an area before moving to it.
Full Disclosure, I am a Teksavvy customer but not a Teksavvy employee.
However, over 90% of American homes have at most 2 ISP choices. At most! So we are in a small but happy minority.
I lay most of the blame here on the FCC and the FTC, who have totally failed to regulate in a way that maintains a healthy market.
But I also have some blame left over for all the people who signed up with major ISPs while the small ones went out of business. It has always cost me more money to go with a local ISP rather than whatever the megacorp's latest intro rate would be. Now everybody is shocked that Comcast, Verizon, et al are giving shitty service and being total dicks about it. But it turns out I can't go down to the corner and shout, "WELL DUH! WHAT DID YOU EXPECT?!? TO THEM YOU ARE THE VEAL!" because nobody likes hearing I told you so. That or they don't like crazy people shouting on the corner.
Tech: "Looks like we're about done. Now I just need to
use your computer to activate the connection."
Me: <looks at my Debian laptop> "I don't have a computer,
I only use my internet for tablets and my roku"
Tech: <looks at my laptop> "I need to use your computer to
activate your connection..."
Me: "I don't have a computer, and of course you do not."
Tech: "Okay, just a second, I have to make a call..."
[one minute later]
Tech: "Okay, your connection is activated, you should be
good to go."
Me: "Thanks!"
I feel comfortable lying to their techs about a computer that is in plain sight because they feel comfortable lying to my face that they must use my computer to complete the installation.
I just played it dumb and said it was just me. We had about 10+ wireless devices connected.
A few years ago I was setting up a new internet provider for my mother, so I called to get the info to get email: Support: "You're using Outlook, right?" Me: "No, Thunderbird." Support: "We only support Outlook, so I can't help you." Me: "It's email, it won't matter if I use Thunderbird." Support: "Our email does not work with anything other than Outlook." Me: "Okay, I'll get rid of Thunderbird and use Outlook." I entered the information into Thunderbird as he gave it to me and then hung up the phone.
But, of course, it only run on Windows, and I was lying telling that I had Windows, as every ISP would refuse to serve somebody that uses Linux...
I had one of their built in modem/router combos that provides internet and wifi, me and a room mate had a minecraft server set up on a box wired into the router, and we played locally, but for some reason every 20 minutes, our connections to the server, the internet, and eachother would just die.
leaving an active traceroute on a spare computer suggested that the router just stopped routing for a period of 20-30 seconds, every 20-30 minutes.
Angry and annoyed, I called in a tech from Shaw to look at and replace the hardware. Standard operating procedure here: "We will show up some time between 8am-1pm". Of course he showed up at 3pm, with no call that he was going to be late. I had to call them.
Unbelievably, he logged into the router (with no permissions given, so those routers are backdoored... goody) and actually identified that the router WAS resetting every 20 minutes or so. Yes, the problem was identified! The problem was that my internet plan was not sufficient enough to support online games, and that if I didn't get another better one, the problem couldn't be fixed. (For the record, I had a 30 mbps down, 3 mbps up, cable connection.)
Jaw agape, I tried to explain to him that A. The game server he was referring to was literally right next to his feet, and B. that even if it wasn't, game data is minuscule compared to video data, something I am a prolific consumer of with no obvious problem there. All that combined with the subtle hint that the router literally just told him it was resetting, the conclusion clearly was that the router is broken, and my internet is fine.
His response to that was:
...
He didn't want to say anything because I had the audacity to say something that went contrary to the marketing line, and the only option he had was to either say I was wrong, or accept it.
He wouldn't replace the router regardless.
So I told him to take his modem/router, put in my old modem, get it registered, and buzz off (though maybe not so colourfully).
When I talked to a Shaw customer representative about having a Tech show up late, lie to me, and not fix the problem without an upgrade, I got this response:
uh, ok?
Fun part though: that transgression was actually considered a transgression by Shaw's regulatory agency, So I reported it to them after I got piddly squat from customer service.
As much as I'd love to see all of that information come to light in a court battle, somehow, I don't think Verizon would...
.@Verizon if it weren't for that C&D you sent to @netflix I would never have known how shitty you've been to your customers. THANKS!
They've got us right where they want us.
I've seen no change in download/upload performance myself.
So I suspect they prioritize the "business class" customers above the "residential" customers.
[EDIT]: fix a spelling typo.
Mostly just because I 100% hate all the cable companies.
Come to think of it, what really separates cable and telephone companies these days? Most of them sell the same things as the others (TV, internet, phone), and the distinction is largely historical and technology based.
There are several nations around the world where this is the case. It reduces the barriers to entry for ISPs, creating an environment where there are dozens of ISPs to choose from with differing levels of customer service and pricing plans.
The primary problem with Comcast and Verizon is that they can leverage their customer base as a negotiation tactic rather than solely on the state of the network.
[0] I wish I was joking about that.
Sue their pants off for the copyright infringement of their customers (and all the other illegal things customers can do). This why common carriers exist. They trade ultimate control over the transfer of content and treat it equally for the protections of being indemnified from the content itself. This is why the U.S. Postal Service can't be prosecuted for transporting a death threat letter. Make it about money and they'll jump on the common carrier bandwagon before you can blink.
They must register as an ISP, can't modify data, and can't directly profit from infringement or promote it. Certain ISPs already have done things that should get them deregistered though, such as compressing images, replacing ads or injecting content. For example, comcast injects their copyright warnings into html pages transfered.
http://recode.net/2014/06/05/verizon-threatens-netflix-with-...
Verizon is a walled city. Verizon entices people to live in it's city and pay its rent because it says you can get deliveries from all over the world in the blink of an eye if you live in our city. It has high speed trains, and wide thoroughfares.
It also has a small gate that can let a few cars in at a time.
Outside the gates is another series of superhighways and supersonic trains that connect the whole world.
Netflix says "We can't deliver our movies because the gate to Verizon is too crowded."
Verizon says "You can't claim that, that's unfair. We offered to let you operate inside our walls if you wanted to pay our rent, but you didn't want to."
The problem is that when Verizon got all of its citizens, it promised that it would give them fast INTERNET access. It didn't promise them that it would give them fast access to things on the Verizon network, or things with special arrangements that bypass the internet and use a special toll entrance to the city.
There's no reason they can't widen the gate. The highways outside are absolutely fast enough to send as much traffic as they need into Verizon. But Verizon doesn't want to widen that gate, because they know if they don't, Netflix will have to pay them in order to deliver to their residents at a reasonable rate.
But they absolutely don't want their residents to think that they aren't getting their advertised fast access to the super-highways outside.
They're also doing this in concert with AWS, right? Another huge player.
It depends on how fancy they get with their detection routines. They can certainly see which networks the traffic is flowing over, but then there are multiple devices (and networks) in play to deliver that traffic to them.
For instance, just a one-way trip showing the devices that could slow down a packet of data can give you an idea of the number of potential bottlenecks:
-> Chromecast CPU
-> Chromecast WiFi chipset
-> User's home network router (usually, but not always, provided by ISP)
-> Cable/DSL/FiOS modem
-> [[ ISP devices they directly control ]]
-> Transit provider's network
-> Netflix's network.
I'm not sure they really need to anyway. The hockey stick graph they have showing throughput before and after payment looks like a classic mafia shakedown.
It seems to me that they're stopping one foot shy of calling Netflix out for libel. It's not libel if it's true, so it might be better for Netflix to only show the message when they can prove that the specific connection is being slowed because of the interconnect (or lack thereof).
At the bottom of this item you can see in a lot of detail how Netflix is making this judgement: http://blog.streamingmedia.com/2014/06/netflixs-network-cong...
Sounds sufficient to me.
1) ISP's insisted on selling home bandwidth as functionally unlimited. They did this primarily so they would only have to offer expensive plans. Grandma can't get the 300meg email plan for $8, only the $80 plan. And then everybody tried to use it like it really was unlimited.
2) DRM. Drm makes it so that the net can't cache. 90% of netfilx bandwidth is likely the same 10% of videos. They all have to make the full trip through the wires. A local cache at the ISP to improve efficiency is impossible.
The ISP need to quit whining. They need to quit trying to get laws passed in every state that keep competition out.
Verizon 2014 1st quarter. $30.8 billion revenue $7.2 billion operating income http://www.verizon.com/investor/news_verizon_reports_fifth_c...
no sympathy
I guess you could add mobile providers to the list if you count mobile hotspots as broadband. That would be an additional 1-4 providers in most areas.
1) cable internet service from TWC (fortunately - they have decent service in my area so Netflix won't be displaying the "your provider sucks here is a better one" link for me)
2) 1.5mbits/s DSL from AT&T
Some might argue that satellite and wireless providers like Clear are viable alternatives. They are not as wireless has spotty connectivity and satellite broadband has unacceptable latency.
So, my choices are either broadband through the local cable monopoly or almost-broadband DSL through the telco that refuses to upgrade their infrastructure in the area that I live.
Free market indeed. :-/
2) This makes me want to get a Netflix subscription just to support them, knowing I would never watch a single video from them. I'm signing up right now. Go Netflix!
Edit: interesting, this seems to be the only screenshot anyone has?
http://arstechnica.com/information-technology/2014/06/netfli...
http://knowmore.washingtonpost.com/2014/04/25/this-hilarious...
I can see that ping times increases and it's always reoccurring around 9-10pm Fridays. Around 9:30 buffering speeds up, meaning most of the people giving up and switching to regular TV. One could argue that this is the time that Netflix/Youtube/Hbo/ViaPlay or any other service is used the most, but my suspicion is that capping occurs at those times, to validate the Fastlane/slowlane argument when the it arrives here in Sweden too.
I am currently collecting statistics to validate this. However, its insane that a ISP can cap and provide lesser service's on certain ports and argue that the fault is at the entertainment provider. Its also insane that when going from a "torrent way of life" to becoming a paying stream customer, I cant even watch my favourite show, when i want, on 100/100Mbit line.
ISP's are in the business of providing fast connections and high bandwidth, and that's what I'm paying for, however I want to use it. It shouldn't matter if I'm streaming from YouTube or downloaded illegal porn. Though it seems it looks like they want to sell me bad gasoline that doesn't take me anywhere, and say my engine is at fault.
Is Netflix not paying Verizon for the bandwidth and data transfer?
If Netflix is not paying Verizon then why does my hosting provider charge me for bandwidth and data transfer?
Netflix's ISP and Verizon pay eachother nothing to exchange traffic (called peering).
Verizon wakes up one day and says, "Shit out of all the ISPs we peer with, Netflix and their ISP is responsible for 35% of our traffic, we have to do huge upgrades on the link to them, and they aren't paying us a dime"
That's my understanding of what is going on.
Thanks for replying (Linode is just an example)
Linode isn't an ISP, they still have to deal with ISPs for the end user, which is why it is different.
As an ISP, I think if you are offering 100Mbps unlimited in/out, you should Expect full thrust from every connection, or at least be capable of delivering that. I know it's not really realistic, they need to assume that some people, sometimes are disconnected but if I am paying for a connection and a certain level of throughput, I expect to get that.
Just my point of view, correct me if my assumptions are incorrect!
Of course there would be a massive revolt of their customers if they did so. Because their customers WANT Netflix. So instead, Verizon will try very hard to make it "sorta" work and blame Netflix while trying to get money from them.
I'm in the same position with AT&T, in a residence that's at the edge of two cities, but is unincorporated (not part of any city, just the county, state, and the USA itself). No cable service is available, the deals they cut with the two cities don't cover me. Those deals make them local monopolies, long ago the original AT&T (which got split up, and then recombined into AT&T, Verizon and Century) arranged a Federal and state level monopoly, except of course for many rural and otherwise expensive to serve markets, which are served by smaller telcos.
I have at least one option for a WISP, but it's awful; at least AT&T's ADSL here is reliable. For now, AT&T really doesn't want it's landline business, wants to drop it for more profitable wireless. Ditto Verizon.
http://www.washingtonpost.com/blogs/the-switch/files/2014/06...
Do they think Netflix will comply? "Oh Jeez we hadn't thought this through but now with this letter and all, we understand! We'll just start blaming our own servers instead. Thanks."
Or is it simply a necessary step before they sue? But if they sue, isn't it likely they'll lose? (Sue over what anyway?)
Not that I think Verizon actually has a strong case for defamation. Netflix should have little difficulty arguing that they had good reason to believe that their claims are true. I don't think this is going to end well for Verizon.
Netflix should try to explain that in the error messages too.
(please look up irony on wikipedia before downvote, thanks)
Because the former is wrong and demanding more money from a single company hosted on a service provider you have specific peering agreements with is extortion.
The latter however, well, no one said there were unlimited pipes between every transit provider on the planet and if you're someone large like Netflix, sometimes you need to pay for transit on more than one provider to get the performance you need when your own provider can't or won't do so themselves. It has been like that for decades.
Netflix will find that most other people also do not care to hear their whining. I can understand why Netflix does it though...it's $8 a month. Between production costs, licensing costs, marketing, and profits, all $8 a month leaves room for is crappy movies, poor investment, and a continuous drive to shift costs to ISPs and whine about it.
It's not like Netflix is causing you to magically download more bytes. You chose to do this, and that's fine, because that's what you're paying your ISP for. Comcast, et al want to double-dip on this transaction and you should be extremely not OK with that. Why wouldn't the ISPs do the exact same thing to Amazon that they did to Netflix as soon as they become a big enough target and the legal precedent has been established? This is something that needs to be nipped in the bud, like NOW, before this becomes an established way for ISPs to do business.
Also, it's not like all of your favorite services that will someday be paying Comcast protection money not to throttle them are just going to swallow those costs. Those costs will get passed on to you one way or another. No good can possibly come out of any of this.