I just turn on my VPN (VyperVPN) which has great bandwidth to the YouTube servers and I get crystal clear 4K streaming. It's kind of pathetic. I don't think they are deliberately throttling, its just that Verizon is being cheap and there is plenty of local bandwidth between VyperVPN's servers and my home, but not enough bandwidth between YouTube's servers and FIOS network in NYC.
You don't pay "for the bandwidth", you pay for a link between two ISPs (Google and Verizon). The bill between then should be split, otherwise the traffic will have to pass through somewhere else, and that will cause congestion problems.
the router ports on both sides, let's say you are using 4 ports across 4 fully loaded nexus 9508 (let's call those $500k per incl. optics, so $2M / 384 x 4 ports / 36 months = $600/port/month x 4 = $2400/4-ports/month)
Depending on how the connection between the two worked or was paid for, it could be $1000-$20000 for 2 pairs of fiber per month depending on distance. Let's take a middle of the road $10k average cost.
So for 10Gbps peak redundant capacity, you are at $12,400 per month. Netflix 4k stream is about 16 megabits, so you can fit 625 4k streams in $12,400 of cross-connect capacity. If you don't share costs, that's about $20 per user to support Netflix's business model that people seem to think it's Comcast's responsibility to pay. That's not even considering Comcast's last mile distribution cost or paying any salaries.
Streaming video services soak up a lot of network resources and putting all of the cost on ISPs is going to increase your ISP's cost and price. How many low-income families do you think would lose broadband if it went from $50/mo to $100/mo?
That, alone, tells me that this is not a congestion or a cost problem, it is a bull-headedness for the purpose of rent seeking problem.
And as to the "last mile" costs, those are what Comcast subscribers are paying Comcast for. They act like this traffic is unsolicited noise, when in reality it's why they're being paid anything by anyone. I say don't charge people for x megabits down if you don't intend on them using it.
I agree that Comcast marketing "up to X megabits" is misleading and those chickens have come home to roost with people thinking that their last mile means they should get that speed to every point on the Internet 24 hours a day. I have been negotiating commercial bandwidth agreements for years and even from a tier 1 you can't get that guarantee in a contract.
Which is still less than the cost of doing a proper interconnect - they chose instead to let it degrade and play semantic games instead. It also puts the lie to their complaint that it was ever about congestion. The correct response to people requesting a lot of traffic from X is to ensure that traffic is delivered efficiently. That is why their customers pay them.
Your numbers are like historical fiction based on reality but not an accurate portrayal. Do you work for verizon or comcast?
since you INSISTED, I have no personal interest in any ISP
If you're going to refute my numbers please provide your own.
If you think this is unfair, then the answer is to get rid of bullshit "unlimited [but not really]" connections and charge people based on what they use. Stop making low-impact users subsidize people who stream HD movies 24/7.
You're getting a 100Mbps connection to the edge of their network and they have peering agreements, what else should they be doing?
They should be peering freely with anyone that is responding to requests from the ISP's end users.
So what is my $80/mo Comcast Internet Service supposed to be buying me?
Seriously, though, could someone do a traceroute and tells me who pays for each hop?
Alphabet spends more on lobbying than Comcast.
As for the "distance" - they are sitting next to each other in a data-center. The distance is likely measured in tens of feet and the cost is likely a fixed cost of a couple hundred dollars. The optics on either end if they aren't close enough for twinax would add a couple more grand.
It's also ridiculous to use 36 months as your payoff date, they are running equipment a heck of a lot longer than that. The line cards and supervisor modules might get swapped out, but I'm willing to bet Verizon keeps their chassis level switches for 7-10 years on average.
So now you're at about 4$ per user to support Netflix traffic.
Netflix in 1080p uses 4.22Mbps or sometimes 7.15Mbps, so we're now at 15¢ per user.
1) The peering happens in Internet exchanges, there's no way a peering connection in an Internet Exchange costs 1000$, your range is simply ridiculous. Usually, it is fixed cost 1K or less depending on the locations of the router
2) Yes, you have to pay for bandwidth between the Exchange and your users. But guess what? That's what your users are paying for.
3) Not everyone stream movie in 4K at the same time. So you could optimize your bandwidth usage out of the exchanges. It is called "multiplexing".
Yes, the rest of the chain could be expensive, but that's what your clients are paying for. Net neutrality is about transparency. I have no problem in understanding that my 40$ connection will perform differently than an 80$ one. But you should compete in giving me the best service (where best is best for me, not universally) at the lowest price point in a way I could easily compare prices.
The problem, with my statement, is that implies Broadband Internet is a commoditized service. And that is all the battle about Net Neutrality, ISP doesn't want to act like commodities, they don't want to be your P&G or your ConEd. They don't have the structure to compete and instead of changing for the new market structure, they are fighting back. What they are doing is rent seeking. Good for them, bad for the economy...
My numbers have been called ridiculous by a couple of people now but they are based in personal experience, albeit a few years old, and nobody has posted any other cost breakdowns that demonstrate an understanding of the industry and the costs involved, just "those numbers seem really high!"
The difference between bandwidth and power is that data is NOT a commodity like power is, power is fungible and can be drawn and combined from a number of sources to fulfill the demand, but the dilemma with ISP bandwidth is that in order to satisfy customers the ISP must ensure adequate bandwidth to each individual content provider, and this is a much harder problem.
It's also not Netflix's responsibility to support Comcast's business model if they can't afford to provide bandwidth that customers desire and are allowed.
It's Comcast's responsibility to support their advertised usage sufficiently. If they say that they support 100mbps then I should be able to get 100mbps from wherever I like. They're going to have to pay for that uplink somewhere, whether it's Netflix or Google or Zayo or Level3 or whoever.
Precisely. Not "from wherever we've got a 'preferred' agreement or kickback only".
Funny, the upwards of $50 to $200 I pay Comcast per month should go somewhat to that.
Besides, how many users are streaming 4K content, on what little 4K content Netflix has (I'm going to wager that we're looking at about 5% or less).
At 5mbps for 1080p, peak, now we're at 2,000 users, and $6/month.
Is it really that onerous for most users of Comcast to expect that $6 of their say $60/mo cable bill goes to Netflix?
I'm also not sure why you're populating 4 ports and then only talking about 1 10Gbps connection, when in reality (though I'm no expert), that's probably 4 10Gbps connections. Accounting for the cost of 4 populated running ports and then only talking about the capacity of 1 of those ports when calculating cost/megabit seems misleading.
The recent changes are more around who is involved in the peering disputes. It used to be smaller ISPs trying to get peering with larger ISPs, such as Cogent trying to get peering with [name your favorite, or PSINet vs Cable and Wireless; and most often the ISP refusing to peer didn't have residential customers themselves. Now it's more often the content providers themselves trying to peer with the residential isps directly. A major factor here is the huge consolidation of residential ISPs, but also the consolidation of content providers.
Consolidation of residential ISPs means each ISP is big enough to run their own backbone, and as a result they can credibly have strict peering requirements. Smaller, regional ISPs will tend to want to peer, because otherwise the traffic will come through on paid transit connections. Large ISPs may not care; because of their size, they may not be paying anyone for transit, and because of the common asymmetric nature of residential connections, there's not likely to be many networks where the large ISP is on the wrong side of the ratio.
If I were one of these content providers, I would spend a lot more time messing with the large ISPs. Figure out how to make the traffic cost them money, so they'll want to peer. Provide transit to data backup services to try to make the ratios less unbalanced. Run campaigns suggesting that residential ISPs should be paying their customers, given that the traffic is unbalanced. Etc.
> Consolidation of residential ISPs means each ISP is big enough to run their own backbone, and as a result they can credibly have strict peering requirements. Smaller, regional ISPs will tend to want to peer, because otherwise the traffic will come through on paid transit connections. Large ISPs may not care; because of their size, they may not be paying anyone for transit, and because of the common asymmetric nature of residential connections, there's not likely to be many networks where the large ISP is on the wrong side of the ratio.
> If I were one of these content providers, I would spend a lot more time messing with the large ISPs. Figure out how to make the traffic cost them money, so they'll want to peer. Provide transit to data backup services to try to make the ratios less unbalanced. Run campaigns suggesting that residential ISPs should be paying their customers, given that the traffic is unbalanced. Etc.
I can upload all my files to Google Drive and all my photos and videos to Google Photos if you think it helps the ratio...
Exactly. If the Internet worked on this model—where the recipient of the net imbalance was paid money for it—I'd get a check from Verizon each month instead of a bill!
The sole and only thing they are being paid for on the ISP side of Verizon's operation is to quickly and reliably deliver the content people want to consume over the pipe people have paid to have installed.
YouTube by hosting content people want to consume is driving demand for Verizon's services.
Pretending that this is a cost is a bazaar inversion of reality.
Do they? Who do they pay? I thought that maybe one of the reasons Google got into the ISP business was to be a peer with the other big networks and not pay any peerage fees.
Does Verizon pay for network services?
s/bazaar/bizarre/*I should clarify when I say "the video space" I mean original AAA content like Netflix is doing. I could be underestimating them but that's a LONG way from their market sweet spot.
https://arstechnica.com/information-technology/2016/11/space...
I don't think that's right.
At its most basic, an interconnection agreement says “You carry some traffic for me, in return for which I’ll do something—either carry traffic for you, or pay you, or some combination of the two.”
With Netflix and Google, both were basically saying, "we're not going to pay you (Comcast/Verizon) for access to your network, because we're important enough that we shouldn't have to." They don't have their own ISP networks to exchange traffic at the same rate, all they have is their services, so they don't have anything to offer the ISP in return.
So it's disingenuous to say that service providers (Google, Netflix), are extorting anyone. They don't see themselves as ISPs, but they are setting up their own interconnects to provide faster access to customers, so they assume they are exempt from what were traditionally informal interconnect rules.
I don't think anyone was saying that.
> With Netflix and Google, both were basically saying, "we're not going to pay you (Comcast/Verizon) for access to your network, because we're important enough that we shouldn't have to." They don't have their own ISP networks to exchange traffic at the same rate, all they have is their services, so they don't have anything to offer the ISP in return.
Generally peering agreements work on total bandwidth and generally you try to make it as balanced as possible, so there's no cost to either side, as traffic may traverse your network from the peer but not terminate there (it continues through another peer), and that's just a load you bear, but the other side has the same risk.
For an end service peering, that's not really as much of a risk, to my knowledge, so what you have is purely a win-win, where Netflix delivers content directly to your network so it's quicker, and you aren't using up peer bandwidth and backbone connections to serve that same content. In any market where Comcast/Verizon didn't have near monopolies over large areas, Netflix could easily charge for this access given their size and ubiquity. That some large ISPs are actually throttling the content only highlights the perverse incentives at play.
But they still want to enforce those "agreements"... when it suits them.
https://www.peeringdb.com/asn/15169
Youtube is available via public peering at the Equinix Internet Exchange New York:
https://www.peeringdb.com/ix/12
Purchasing a port on public peering fabric isn't the same as buying bandwidth from a transit provider. It's a great way to get traffic to your users much cheaper than you could via transit which is why people do it. The issue is eyeball networks - Comcast, FIOS, Time Warner etc. may choose not to do so because they went to seek rent in the form of "paid peering" from these same content providers that their users paid them to get access to in the first place.
Once they know you're connected to Youtube, they just limit the bandwidth on that connection. The Youtube video player will detect this and automatically downgrade to a lower quality until it finds one that fits within the bandwidth profile. Netflix does the same thing, which is why after buffering sometimes the video looks like crap and then gets better after 15 seconds or so. It buffered a lower quality until it figured out it could send higher quality again.
EDIT: I should add, this is why VPNs are sometimes used as a "solution" to such bandwidth shaping. You are hiding the true recipient by purposefully man-in-the-middling the connection with a peer you (hopefully) trust. VPNs are also popular for connecting with business networks, so it's generally not "safe" for the ISP to shape bandwidth to any VPN as aggressively.
Ahh, this is the key point I missed, thanks. So it's a problem with the video player itself trying to "decide" for you the quality you want. Couldn't this be worked around with things like youtube-dl?
I'd imagine this is due to politics and AT&T betting net neutrality will go away. Once YouTube forks over money a peering point will be upgraded.
[1]https://forums.att.com/t5/AT-T-Fiber-Equipment/Gigapower-You...
Youtube-dl would probably work, because it will would download the high quality video as a file, and your ISP would make the download take longer than the video.
It's an extremely annoying thing actually, especially when it changes after I manually set it to 1080p. This is why I use mpv to watch stuff from youtube nowadays.
Now Verizon gets the blame if something's slow.
It was the first time in my life that I'd been wishing for Comcast. 320p videos could barely load, and the lag in certain MMORPGs was unplayable (we're talking 500-1000ms delays)
It got better after a while, but google had some stats by ISP for Youtube, and FiOS in my area was like 1/10th of the speed of Comcast on average.
Google needs to take gloves off.
Or maybe they are throttling it...
Reality is a bit more complicated than this simple example but such an issue is certainly not unheard of (and inspired mitigations such as RFC7871, for example).