Why We're Having to End Our Direct Peering Relationship with Deutsche Telekom
about.fb.com
about.fb.com
DT is saying they had an agreement of interconnection for payment, Facebook stopped paying, a court said it has to keep paying, and Facebook decided to end the interconnection instead of continuing to pay. DT, in typical German fashion, insists on continuing to receive payment. Because it's Germany, they'll probably go back to court and be able to win another case and forcibly extract payment even though the service is no longer provided.
This (and the DT article) seems to imply Meta stopped paying without ending the interconnect. That seems like a reasonable reason to want to be paid?
It takes two to keep a BGP peering session up: if DT didn't want the traffic without being paid they could have shut things down on their end too:
* https://www.juniper.net/documentation/us/en/software/junos/c...
* https://www.cisco.com/en/US/docs/ios-xml/ios/iproute_bgp/con...
> the company feeds a gigantic 3.5 terabytes into Deutsche Telekom's network [...] Deutsche Telekom has done everything in its power to ensure smooth data traffic. [...] The start of the rerouting of data traffic in the night from Tuesday to Wednesday went smoothly.
It sounds to me like DT was not prepared to handle the volume of this traffic along the alternate paths it would have taken if Meta's paid transit sessions were removed, so in the interest of avoiding disruption of their network, kept them running temporarily, settlement-free, until mitigation was completed. Great deal for Meta while it lasted!
The word "deal" implies some kind of (commercial) transaction that has a price tag associated with it (deal = good price).
But DT keeping this peering connection up should be (IMHO) DT just doing the job they're paid for.
DT's customers pay for "Internet access", and since Meta is on the Internet, the customers should be able to connect to it. Customer's send requests to Meta, DT passes those requests along, and it is DT's job to pass along Meta's responses.
Why should Meta pay DT anything? DT is claiming to offer a service, and part of that service is transporting the responses to DT's customer's requests. If those responses are high bandwidth it is part of DT's job to deal with it. If 'dealing with it' entails special connections to particular corners of the Internet to better serve their customer's then it is DT's job to do that.
The value that Meta received is equivalent to what Meta is now paying their third-party transit provider to transit this same volume of traffic to DT. My napkin math suggests that this is tens to hundreds of thousands of dollars per month, at 3.5Tbps and pennies per megabit.
> DT's customers pay for "Internet access", and since Meta is on the Internet, the customers should be able to connect to it.
What do you mean by "on the Internet"? Meta is "on the Internet" by virtue of paying transit providers to propagate their IP prefixes within the DFZ.
You can't just show up at an IXP or other peering location, announce your routes, and expect them to propagate globally. You need to convince at least one transit-free network to carry them before you're truly "on the Internet". Meta used to pay DT for transit, and now they don't. Meta is now "on the Internet" via third-party transit, rather than directly via DT.
> Why should Meta pay DT anything?
Meta doesn't need to pay DT anything, but if they're not paying, they shouldn't expect free connectivity services from DT. This traffic can now flow between DT and Meta via transit, instead of directly between DT and Meta.
> If 'dealing with it' entails special connections to particular corners of the Internet to better serve their customer's then it is DT's job to do that.
Which is exactly what DT did by providing settlement-free peering, temporarily, while their links with other ISPs, who they do have a business relationship with, could be beefed up. Now those links can handle the traffic, and now the traffic is flowing over them rather than over direct peering with Meta. No traffic was dropped in the process; neither DT's customers nor Meta were or are being harmed.
Sounds like Meta threw its toys out of the pram because German courts didn't bow to their demands. I'd love to hear more details on the "real" story, and not Meta's PR bullshit.
For a very long time, there were substantial issues with getting traffic from Hetzner to Deutsche Telekom for similar reasons. At the time, when you rented a server at Hetzner you could actually pay extra to get 'premium' routing to Deutsche Telekom though Core Backbone (AS33891), which did make the ransom payments (unlike Hetzner itself).
They were also hard at work undermining net neutrality for a while by zero-rating some companies on their mobile network until they were stopped by courts.
I'm not sure what's going on in this specific case, but I'm not inclined to give them the benefit of the doubt.
This would go against Net neutrality principles but I don't see any facts which support those claims by FB. So it's not clear what this is all about when it seems it broke down during contract renegotians.
This seems dubious though given the German market is extremely well regulated. Also I've never heard of a provider requesting payments for delivering externally hosted services. Maybe some special cases might exists for things like Netflix bandwidth tunnels with ISP's but that's usually a request from the service provider to have special traffic lanes.
Edit : DT customers are being limited.
It ran this same shakedown routine on an academic network in the middle of the pandemic.
It's run this against Hetzner.
It's the same playbook that Comcast/Verizon/Time Warner used vs Netflix/Cogent/League of Legends in 2014.
https://cyberlaw.stanford.edu/blog/2024/09/a-deutsche-teleko..."
* https://web.archive.org/web/20151208135240/http://wiki.hetzn...
The reputation of DT(AG) is not good:
* https://old.reddit.com/r/networking/comments/3uuulg/this_is_...
* https://old.reddit.com/r/networking/comments/1fpe8p9/meta_de...
Meta is ending its direct peering relationship with Deutsche Telekom due to the company's actions that undermine net neutrality and put the open internet at risk.
Deutsche Telekom is using its market power to potentially restrict its subscribers' access to internet services that do not pay the company additional fees, creating a de facto paywall.
Importantly: This is Facebook complaining, hoping to pressure Deutche Telekom through public humiliation. They have peering agreements elsewhere where strict access to Facebook without open Internet access is allowed. So if shutting down open Internet access benefits Facebook, they're happy.
This isn't quite correct, and thats why I was complaining about the article missing essential information because it's really misleading and makes you think that that is what is happening.
It turns out what happens is Meta wanted is a price cut of 40% on a contract with DT, DT was only open for 16% and decided to move to a different provider. Also (and here other sources where also a bit unclear) there was some curt ruling which boils down to yes the prices from DT are legal and Meta has to pay if they continue to use it (which they did while trying to get a 40% price cut). What is unclear here is who sued for what. FB trying to force lower prices by ab(?) using net neutrality laws? DT suing because FB not paying for continous usage of the service while discussing a new contract? Something else? Did they sue or was it a declaratory action? etc.
Most important this might not have as much to do with net neutrality as it seems.
Net neutrality is about not unfairly prioritizing some traffic over other traffic. It's also somewhat about you not having to pay every network provider your traffic goes through. But it is _not_ about content providers not paying at all for network traffic, they still pay the provider which connects them to the internet for traffic the same way you as a private person do (just probably much less).
And that is where they wanted a 40% price cut as far as I can tell. And that is also where they then moved to a different provider by now and where they don't have to pay DT for traffic flowing from their new provider through the DT network to customers (as far as I can tell).
So if you look at it from that perspective it's FB trying to abuse net neutrality by labeling traffic from their servers as transit traffic (but also some articles say the judges said it doesn't matter weather it's transit traffic or not in that case, which makes it even more unclear what exactly the curt case was about, probably some contractual clauses in their previous contract in which case it probably really doesn't matter)
* Contract law: Meta lost a court case about a contract for IP-Transit with DT. This is legit and correct - they continued using the private interconnect and by german law this requires you to pay. This is clear cut.
* Telekom peering policy / brigandage: DT is the only major provider that is not present at any IX in Europe. They don't do any public peering. My local small ISP has better IX connectivity than DT while having 1/100th of the userbase. They require from hosters/content providers to sign contracts with them that are very overpriced (about 10x more than regulary private peering deals). They basically use their market power in germany (40% of internet users using their network) to force any internet service into buying an expensive contract with them.
This is a thing that should be regulated via the german regulator or EU wide. If other ISPs act like DT any server hosting provider needs to pay every bigger ISP for access to their customers and this would be the end to the internet as we know it. That's why it's a big topic. But legally DT can do it at the moment and they act like this for over 10 years now. There were older disputes with a big german hoster Hetzner, Init7, Level3... now Meta is playing on another level and hopefully that bully behavoir from DT stops and others follow suit and these overpriced contracts stop.
then I would agree that the combination of unusual high prices for direct commit + absence on IX or in general not enough bandwidth for "fair" cross network pairing is indeed effectively undermining net neutrality and needs to be stopped
through IMHO it's a bit of an oversight of Meta to not be very clear about it, missing information in their article and other articles starting their reason section with 40% price cut doesn't make them look good
1. DT no longer peers with Facebook for free, and Facebook isn’t willing to pay? A bit more latency for Facebook users using DT, probably a small drop in Facebook visits from DT due to that, but overall, big whoop.
2. That + DT throttles Facebook traffic aggressively in retaliation? That would count as “undermining net neutrality principles” and bad in general. But it’s the kind of thing that already happens in a lot of places generally speaking (don’t know about Germany specifically), and we should push telecom and/or antitrust regulators to prohibit it, but I’d say there’s no urgency here.
3. That + DT sells customers some sort of “Internet except without Facebook, pay more to get Facebook” or “Internet except with slow Facebook, pay more to get fast Facebook”? That would officially be a Very Bad Thing, worth boycotts and other kinds of shouting from the rooftops.
The post is very very unclear about what’s actually happened. The actual court decision is about something like 1, but the talk of a “de facto paywall” would imply 3. Overall, I feel trampled by a Gish-galloping cavalry regiment here.
The problem is DT keeps its transit routes congested or artificially limits how much traffic a transit provider can deliver.
So now when FB moves to transit, will DT widen those connections or will it keep them narrow to try to make Facebook's applications get terrible enough performance that Facebook will pay DT.
It's a showdown. DT wants to get paid twice for the same service, once by its subscribers and once by every app and website on the planet.
https://cyberlaw.stanford.edu/blog/2024/09/a-deutsche-teleko..."
They have both reasons and they are both wrong.
Ten words. That's as far as anti-EU techbros will read before jumping to conclusions.
I believe they forgot to mention "innovation" though?