Verizon network ALG blocks SIP | Hacker News Reader

Verizon network ALG blocks SIP

onsip.com

63 pointsgz527 comments

Verizon appears to be using CGN (Carrier Grade NAT) for IPv4 on their LTE network. Having seen many, many SIP ALGs in the field before this looks less like intentional blocking and more like typical broken (but somewhat well intentioned) SIP ALG behavior. Unfortunately SIP ALGs only work (well) when combined with properly configured far-end equipment that doesn't perform the various SIP checks to detect endpoints behind NAT devices. My suggestion for any provider in these situations is to use SIP over TLS. SIP ALGs can't modify packets they can't inspect. Often times SIP ALGs can be defeated simply by changing SIP port numbers.

I'd expect onsip to know better and not write a sensationalist article implying that there is some anti-competitive behavior from Verizon at play here. Hanlon's razor at it again (Verizon).

I agree that this may not be intentional. ALG is the cancer of SIP. I never saw one that actually works. Petty much every documentation that describes home network configuration for VoIP will start with "if your router advertises SIP ALG capabilities, turn it off". They're at best behaving in silly ways with simple traffic and mess up pretty much everything in more interesting scenarios (hold+transfer can be broken in so many ways...)

God, I've lost count of the times where the SIP ALG on my WNR3500L's stock firmware just caused my softphone for work to behave in the strangest ways, normally causing one-way audio. The worst part is we have well defined ways for handling NAT traversal for SIP without need for an ALG, but router manufacturers still include them?

Switch to a combined channel. If everything's in one TCP connection, it doesn't matter as much what the NAT does. Better yet, make it work over "HTTPS" (ie. what looks like HTTPS).

Easiest possible way to do this : switch to IAX.

This is confusing... Iax doesn't work over TCP and doesn't look like https. It does use a single UDP stream for both control and data, but that makes it really difficult to scale on server side and makes attended transfers tricky.

You got me on the TCP/UDP part. Obviously you don't want to run voice over a (real) TCP stream, so I got that wrong.

I would argue however, that the "feature" of SIP to send the audio over an out-of-band channel has caused me so much misery over time that I wish it didn't exist. I have managed a VOIP provider and at the edge we terminated SIP, terminated the audio on the incoming system, where it was sent to a central "cluster" for routing, and then terminated again on the outbound. Yes this "wastes" bandwidth, both on the (internal) network and memory/cpu bandwidth too. How much ? Well, about 3*48 kbit per call, on a 10Gbe network. And the machines were needed anyway : inbound customer lines termination needed 2 machines for redundancy. The routing "cluster" needed to be 2 machines so as to be separate from the inbounds (due to much more complicated configs), and again 2 for redundancy in different datacenters (and then 4, and 8 when customer base grew). And the upstream (towards providers) machines weren't even machines, but routers. Again doubled for redundancy.

We didn't actually run IAX (mostly because the routers didn't support it), so I do not know much about it. We explicitly avoided using the out-of-band audio feature, and we knew why. It's savings are negligeable, and there's no end to the trouble it causes in operations.

Exactly. SIP ALGs get turned on often (who knows why). There was one cloud computing vendor using NAT that had a SIP ALG "accidentally" statically compiled into the kernels on their routers. They weren't sure why, but thought "it might help", but of course, it ended up just breaking things.

This could totally have been a move by Verizon to help enable SIP that just backfired.

With my iPhone 5 on VZW, I can SIP REGISTER just fine with Bria (a SIP client) running on the phone itself. I can make a call, too (using FreeSwitch). I cannot, however, do any SIP REGISTERs with my MBP tethered to my phone.

So the headline would more accurately be:

  "Verizon blocks SIP on tethered devices"

very interesting. and Bria is not using TLS or encapsulating SIP in https? would also be interesting to see how selective VZW is being, e.g. letting some SIP through (even tethered), depending on the app.

A lot of networks do this, it's not just Verizon. We've seen invites disappear at many different networks like hotels and different mifis. As well as registrations never go through on some.

That said, this and poor implementations of sipinspect on routers appear to be the Achilles heal of SIP.

This is super interesting.

In full blown 4g networks, voice is going to be handled via sip traffic via an IMS core.

Some things to investigate: Is Verizon using separate APNs for their voice and data.

Based on what is presented, I am guessing no... And that they jhave routing/QoS rules that are rather ungranular. They assume all sip traffic must be their own IMS traffic and then route it to their ims core, where a proxy is damaging the flow.

Anyone who thinks that an operator will allow you to use SIP traffic over their LTE network forever is incomprehensibly naive. Paying customer voice traffic should not have to compete for resources with your data traffic. Net neutrality is not a suicide pact.

With Verizon, there's an issue of open access. Verizon bid on 700MHz spectrum with special "open access" rules that require them to allow any application. In the 2008 700MHz auction, there were five blocks up for auction and Verizon went for the one band that had these restrictions: that customers be allowed to use any compatible device they like and use any software, content, or services they like.

Paying customer voice traffic is today, and can be in the future, carried on other spectrum.

Spectrum is only part of the equation... And if they didn't design things right, they won't be able to take advantage of spectrum differently like you propose.

Yeah. . .but to be a bit crass, that's totally Verizon's problem. I mean, at some point they have to have had meetings, "so, how will these open access rules affect our future?" And yea, I'm being a bit silly when I say "they have to have" (this is going to be a bit of a silly comment). Companies do dumb things all the time. But if the response was, "eh, who cares. I'm sure people will forget about it," then it's kinda their own fault. Without these restrictions, that spectrum may have raised more money. It was auctioned with these restrictions with the idea that the restrictions are in the public interest and that it might cause the winner to undergo additional costs or lose certain pricing options. If the result is, "c'mon dude, you can't really expect us to do that," I don't have a lot of pity for it. Net neutrality isn't meant to be a suicide pact, but to be frank, almost all of the wireless spectrum that the FCC has made available for mobile doesn't carry this restriction. Verizon knew this restriction going in and could have chosen not to bid on that spectrum. I think with Verizon having gone in with full knowledge of the restrictions involved, it's reasonable to hold them accountable.

The big question is, does this only affect IPv4? Last time I had a Verizon phone, the IMS interface was IPv6-only, so VoLTE will probably use IPv6 as well.

Yeah, it's because VZW uses carrier grade NAT for IPv4 traffic, when using IPv6 you get a publicly addressable IP and it's a non-issue.

Nobody is running VoLTE, and there are no VoLTE handsets in the wild, and if they were running VoLTE, there is no way that SIP traffic could harm it any more than any other kind of traffic. Nor is SIP used solely for voice calls.

C'mon, a "suicide pact?" That's way over the top. So to speak.

IMS/QOS on LTE is actually a very complicated topic, as it goes beyond routing/QoS rules. I'll try and comment on some of the points you hit on.

So to clear up a few things: 1. They assume all sip traffic must by their own, and route to their IMS Core IMS SIP proxy information is relayed to the device when it attached to the network if it's IMS capable. It's up to the devices stack to use the proxy provided by the network, which as far as I'm aware no 3rd party software does at the moment.

2. Is Verizon using separate APNs for their voice and data. Based on http://support.verizonwireless.com/clc/devices/knowledge_bas... it appears they may use vzwims I believe by convention, for roaming the "ims" APN is to be used, however, I'm having trouble locating the recommendation by either the GSMA or 3GPP, so now i'm not sure where I heard that.

3. routing/QoS rules that are rather nongranular Actually, LTE has spent immense effort in providing capabilities and controls around prioritizing very specific traffic. In high level concept, when you use the IMS core to generate a SIP call, the local SIP proxies will have links back to the PCRF (policy server).

The PCRF is capable of generating IP Source/Destination rules that are sent to the network, to prioritize the specific RTP conversation. This is taken to the level in LTE network's, where the LTE network depending on the rule will "Guarantee" the bitrate is available, and the rule is pushed to all aspects of the network include the eNodeB (Cell Site). The over the air scheduler in the eNodeB is to respect the QoS rules to ensure traffic such as conversational-voice does not receive jitter/packet loss from background traffic (such as file transfer). Obviously, if you don't use the IMS core, you lose this capability, since it's not based on packet inspection, but by the proxy server itself knowing a call is in progress, and creating dynamic QoS rules just for that call.

Even with the way net neutrality is going, from my view it looks like the standards body is providing the capabilities for a fairly QoS rich environment. Personally I don't have a problem with, the network understanding that your doing web browsing, and to deliver a better experience, you get higher network priority for you're basic web viewing, then the person down the street who's doing background file transfer, and saturating the cell site. The net neutrality problem is, if the carrier tries to leverage this not for experience, but profiteering, or to block certain types of traffic. However, that's a much larger debate that I don't think belongs here.

4. Paying customer voice traffic should not have to compete for resources with your data traffic This is effectively true already (see number 3 above). However, your point is more around that this is reason to block non-carrier SIP, which isn't the case. As an operator, I can just lop it in with the rest of the background data traffic, and you receive a user experience that matches.

Since this is background data, there is also less of a guarantee that the service will work consistently, as it's not something explicitly tested for or verified in the many pieces of network infrastructure. As is indicated in other posts, this sounds like issues with the ALG, however, there are all kinds of things that can go wrong, NAT, Firewall, IDS.

I once worked a problem with RTCP/RTP a number of years ago, where a bug in one of the network elements, actually got confused by the STUN packet, because it was being too smart and setting up rules based on inspection of RTCP. The funny thing about it, was that everything would've worked fine, if it wasn't being too clever to try and solve the problem for us.

*I do work in the telecom industry, however, my views and comments do not reflect my employer in any way. All information here can be found publicly when knowing where to look, or is my own opinion.

VoLTE is not deployed yet (but soon).

Voice and in general IMS traffic will use a separate APN, per the GSMA VoLTE specification. This spec requires that an APN called "IMS" be used, and Verizon is complying (even today, SMS is done over IMS with LTE). All operators will do that.

Whenever this is real problem or not, one thought came to mind. SIP and HTTP are pretty similar. It would be nice to have kind of (REST?) translation SIP -> HTTP/1.1. No need for complications (like webRTC, websockets, etc), just simple and clean proxy somewhere on the internet. Example:

SIP invite: INVITE sip:bob@biloxi.example.com SIP/2.0 translated to: HTTP /path/to/invite/bob@biloxi.example.com HTTP/1.1

Client sends that HTTP request to HTTP->SIP proxy so that it is translated into specific SIP request method (INVITE). When proxy gets response from SIP it converts it back to HTTP in similar fashion and deliver to client. That way they couldn't forbid SIP signalization on client because on client side it's plain old HTTP/HTTPS :-) This should not be problem to implement as library or something like that.

In short, simple SIP over HTTP tunnel.

P.S. Unlike HTTP, SIP is federated protocol and because of that it is, in a sense, little bit more decentralized. HTTP is designed as man in the middle attack protocol. One step at a time :-)

I remember seeing similar behavior trying to do testing in a cafe using a Verizon MiFi over LTE. I called up their support staff and they tried to blame the problem on me saying they don't block anything, it must be on my end...typical.

I cannot say how I know this, but I can assure you that this is a perfect example of Hanlon's razor at work. I am 100% sure that it is not intentional.

Are they blocking Jingle as well? I hope T-Mobile won't do any such lunacy when they'll roll out their LTE.

Using standard Internet protocols (HTTP/HTTPS) isn't subject to carrier shenanigans (whether deliberate or not): http://finance.yahoo.com/news/rebtel-chooses-pubnub-pulse-sp...

It's possible it's changed, but at one point VoIP was specifically against the terms of service for Verizon wireless data plans (playing games too).

Is that legal?

Unless I missed something, there is no such thing as "net neutrality" for cell phone networks.

And anyway, this is probably a misconfigured NAT device somewhere, not an intentional plan to block third-party SIP services.

Reply on news.ycombinator.com