Introducing Pseudo IPv4 | Hacker News Reader

Introducing Pseudo IPv4

blog.cloudflare.com

142 pointsjgrahamc87 comments

Except that in the past I've had major issues with Cloudflare's IPv6 service, recently one of the issues was that Centurylink customers (at least in the Denver area) were unable to visit any Cloudflare hosted IPv6 sites.

https://twitter.com/bertjwregeer/status/470243728473325568

Was my Tweet to Stackoverflow regarding the issue.

Here is the paste of the symptoms seen: http://paste.ofcode.org/XQGqerxCNXwYsHDQMZ3aja

This meant that until I reported the issue to Cloudflare/StackExchange that people using IPv6 were UNABLE to access those resources needed to load the site (the server would hang indefinitely, so happy eyeballs did not work!). In the case of StackExchange on Centurylink that meant that CSS and other resources did not load, for my site (http://defcne.net/) that meant my site didn't load at all!

I absolutely love Cloudflare, but to me it is inexcusable that there is no monitoring to verify that these issues don't exist. It took me filing a report for them to fix the issue.

Ultimately it came down to this:

"We had recently been experiencing IPv6 routing issues with one of our upstream providers for some of our data centers which may have contributed to the issues you had been seeing. We've since disabled transit for that provider to temporarily work around the issue."

Yet this meant that my site/StackOverflow and countless other sites using Cloudflare were offline (if the customer has IPv6 enabled) for almost a week (first report from customer using CenturyLink, me trying to figure out what is going on, to CloudFlare fixing the issue).

Huh, I'm on CenturyLink ipv6 at home and was having weird issues with a number of sites from early last week onwards, and then early this week it magically cleared up. Guess that was likely it.

>IPv6 can be adopted without a performance penalty

Sadly, in my case, this is untrue. If I enable v6 on my Comcast home connection, I see routes with consistently higher latency--around 50ms more for paths within the U.S. such that even a 200 mile destination (HSV => ATL) is ~70ms away.

Wait wait wait...Comcast is using bad and/or incompetent infrastructure? Stop the fucking presses!

(I'm sorry sir/madam for your inconvenience)

Comcast has been at the forefront of offering IPv6 connectivity to their users.

Comcast is deploying IPv6. Verizon isn't even trying IPv6 with FiOS.

They have actually started working with business class customers this year. I'm still, rather impatiently at this point, waiting for it to trickle down to residential customers.

They have a landing page[0] that hints at the possibility, but I've never seen anything or heard anyone actually say that IPv6 is coming to residential customers.

[0] http://www.verizon.com/Support/Residential/Internet/HighSpee...

They haven't bothered to update that page in a few years, sadly. It says "Dual Stack IPv4/IPv6 will be launched in various areas within Verizon’s FiOS network, starting Later in 2012. Check back for more information."

You should probably go to forums.comcast.net and post some traceroutes. There's no latency penalty from the SF bay area, and Alabama seems like the kind of place that's easily overlooked.

Sorry, but this is untrue for me. I have Comcast, and if I enable IPv6 from my home in Santa Clara county, I also start seeing large increases in latency and significantly slower download speeds.

I haven't bothered posting on the Comcast forums. I simply disable IPv6 and get on with my life. Maybe I'll give it a try in another few months.

The "no latency or speed penalty over IPv6" thing is true for me in San Francisco for major sites (Google, HE.net, Facebook, Wikipedia).

Are you sure that you're not running over a Teredo tunnel? (Is the IPv6 address you're handed in the 2601::/28 range [native IPv6 for Comcast], or is it in the 2001:0::/32 range [Teredo tunnel]?)

Does some of your networking hardware handle IPv6 poorly? What happens when you plug in a IPv6-enabled computer directly into your cable modem?

How do you expect the problem to be fixed without you reporting it?

Hi there,

I work on the IPv6 team at Comcast - please email me with details... nathan (underscore) owens(at)cable(dot)comcast.com

Thanks, Nathan

I work on the IPv6 team at Comcast - please email me with details... nathan (underscore) owens(at)cable(dot)comcast.com

Thanks, Nathan

I have comcast. My router is configured to enable IPv6, and my local machine appears to have an IPv6 address (though it appears to just be an IPv4 compatibility address, as it contains my IPv4 address with an fffe in the middle). But this site claims I don't have an IPv6 address: http://test-ipv6.com/

Hi tiglionabbit,

Which router do you have, and which firmware version? Which OS/Browser are you testing on?

Thanks, Nathan

I have IPv6 turned on in my router, but Comcast seems to semi randomly give/ remove my address. Had one Oct-Dec, Feburary, and about 2 weeks in May.

So, how easy will it be to deliberately search for an IPv6 address to collide with a desired pseudo_ipv4 address? (Based on my very limited crypto knowledge, I might worry that there could be some novel denial-of-service or impersonation attacks in that direction if this is MD5 with a known format and salt.)

300MM is well inside brute force range for even a single CPU (as noted in the article, md5 is cheap!). One issue is figuring out which IPv4 address you wish to impersonate; sites don't give that information out so readily in 4chan's case, the IPv4 address only ends up in user-facing information as poster IDs, which are themselves hashes of the IPv4 address and (I assume) some thread-specific salt. For these poster IDs, I've never checked if a cookie is involved, but that could also be the case, and would make this attack a bit harder; that said, it is quite feasible to obtain the target's IPv6 address through other means.

I think what might be kind on Cloudflare's side is to add a secret domain-specific salt to this md5 hash, but I'm by no means a crypto person.

(edit) eastdakota and billpg below both pointed out that to carry out an impersonation would require connecting to Cloudflare with the correct IPv6 address. This is probably the biggest hurdle, so feel free to ignore what I wrote above.

"adding a salt to md5" really means "using a keyed-hash function", e.g.: HMAC.

Anyone with an IPv4 address can use one of several 6to4 gateways to get a whole /48. This gives them access to 2^80 addresses they can originate traffic from.

As mentioned in the article, they only hash the first 64 bits of the address. That means you only get 64-48 = 16 bits to work with.

The hash only takes the top 64 bits of the IPv6 address, so unless you have a wide choice of that half of the IPv6 address, you could only use the one you've been given by your ISP.

Even that possible vulnerability (if I can even call it that) would be stopped if they (Cloudflare) included a secret salt in the hash so the only way to know which class E a particular IPv6 address has would be to try it out and observe the connection from the other side.

You have more choice if you were allocated a /48 though.

You have 16 bits. So you can generate 2^16 unique hashes out of a space of 2^28. So you have a 1/4096 chance of getting a collision.

Exactly. Since it looks like there's no salting going on, if you know your target's IPv6 (and from that their calculated class E), you could quite easily go through your own set of available addresses and see if any result in the same class E address as your target.

It wouldn't be a good assumption that the code we posted to the blog is exactly the same as the code that is actually in production. If we included something like a salt, we obviously wouldn't reveal it.

Yep and those aren't hard to find. Alot of VPS/Dedi providers would give you this allocation for free.

Finding a collision is one thing, you'd then need to find a way to connect via the colliding IP.

Either way it's an unnecessary risk. People with access to a /smallnumber might still be able to exploit it. I find it quite ironic that they are all like "Common guys, use this new stuff, old stuff is bad!" -- then they go on presenting a solution that makes use of md5.

You're replying to their CEO.

Thank you. Sorry for directing the comment improperly.

Feel free to argue the actual argument presented.

This technique was demonstrated at the Google IPv6 Implementors Conference in 2010:

https://www.youtube.com/watch?v=QkV9ZgRZvv4#t=7m52s

Ahhh -- thanks for the link. I was just trying to dig up an old NANOG thread which discussed Gmail's use of the same technique.

I noticed class E address space in my gmail activity log years ago, and a wink from a googler implied this is what they were doing. Nice to have confirmation.

Cloudflare is really really good at writing technically interesting PR releases.

@pg, news.ycombinator.com does not have a AAAA record. Is there any chance for it to support IPv6?

I raised an issue on GitHub about this a few days ago: https://github.com/HackerNews/HN/issues/100

that is unfortunate considering they are using cloudflare... I wonder what the hacker news software does with IPv4 addresses that couldn't work behind the service described in the article.

Maybe I'm not the only one wondering why I should bother to enable IPv6?

If you have that mentality, we'll never get off IPv4. If that happens, ISPs won't have a choice but to implement Carrier Grade NAT. That's bad for a lot of reasons, but mostly because many homes share the same address. Implementing IPv6 means every device in the world has a unique, globally routable address with all ports open (no NAT).

That's pretty cool. However, we can only do this if we get over the chicken-and-egg problem, which is why it's important you enable IPv6 and encourage others to do so.

If I want to pirate content, your Carrier Grade NAT looks quite attractive as a legal shield!

I'm not entirely sure if CGN will provide some legal protection for pirates but it might make it harder to pirate content as you cannot open ports on CGN meaning BitTorrent will have less seeds which slows downloads. Afaik, IPv6 will make piracy extremely easy as UPnP won't be a requirement. Want to share a file with some friends? You can quickly spin up a web server, send them the URL and it'll just work.

You still need UPnP or something, otherwise your IPv6 default firewall policy (allow out deny in) is going to block inbound connections.

Yes, it's easier to hole punch, but a webserver won't do that.

And if you're manually configuring a firewall, I'm not sure "allow port 80 <someIPv6>" is any easier than "forward port 80 to <someipv4>.

What am I missing?

I think you are missing a lot. For instance, I have IPv6 set up at home, at work and at some homes of friends and family. I have firewall rules setup such that traffic from subnets I know is generally allowed instead of allowing access to a single port for the general internet. I also have DNS set up with names like computername.sitename.mydomain.tld

That allows me and the people I know to connect to each other's machines in a way that wouldn't be possible with IPv4 and NAT. I can be at my brothers and type \\[fqdn] in explorer and it will just work. To me, that is the way the internet was meant to function from the beginning.

If you're able to configure firewall rules, you're well outside of any normal users able to make up a significant amount of P2P traffic. And to most users, port forwarding and configuring a firewall rule are nearly identical.

Truth is that for most users, NAT today is almost always synonymous with a firewall that has deny in, allow out policy.

10+ years ago, a lot of folks often connected their machines to the Internet in the way you specified. You could go around scanning people's systems, viewing their fileshares and so on. NAT "fixed" a lot of that.

First off, good luck trying to implement any kind of decentralized network service when almost nobody has a globally routable address.

Second, if ISPs are willing to keep records of IP-address-to-customer mappings, it's not much of a stretch to add TCP/UDP ports to those records as well.

The IETF has (sadly) specced logging of cgn associations for surveillance purposes.

> Implementing IPv6 means every device in the world has a unique, globally routable address with all ports open (no NAT).

Not necessarily. I've configured my IPv6 firewall to block all the things except for specific ip:port pairs. I didn't feel like leaving my network totally open.

You setting up firewall rules at your gateway has no influence on whether an address is routable or not.

IPv6 gives you the ability to make that choice for yourself rather than having it foisted upon you.

Indeed, in this case (and I'd imagine in most cases). IPv6 also gives your ISP the ability to make that choice for you; some ISP's, for example, block ports 25 and 80.

Because you will be excluding connectivity. Maybe you think that only affects far off places and not your stuff and your networks, but be aware that many mobile networks are using IPv6. The number of mobile devices keeps increasing. One day you (or the boss) will get a new device and it will have connectivity problems. Google and Facebook will work just fine.

Sure there is NAT and various other things carriers and ISPs can do, but that will increasingly be the slow path, and possibly even charged for.

You can put your head in the sand, only to have this issue bite you badly and urgently one day, or you can start now slowly but surely making sure everything appropriate is done. For example you can make sure purchases of equipment and software claim to work correctly (how will your VPN work?), do whatever training is necessary etc and gradually add IPv6 to your infrastructure and clients.

Mobile networks use IPv6, but they also NAT to IPv4. So unless there's financial pressure, it's unlike his IPv4 will stop working. Maybe one day in what, 15? 20? years your statement will be true.

It may take until then until IPv4 stops working, but it will be before then that it becomes less optimal, and possibly more expensive. Heck I already experience slowness due to some freshly installed systems trying IPv6 first (DNS lookups, connection attempts) and only on failure falling back on IPv4.

Enabling IPv6 for content motivates ISPs to enable IPv6 for users, and vice versa; it's a positive feedback loop.

Consider an ISP with Native IPv6 and NAT'd IPv4. (T-Mobile and Verizon Wireless are specific examples in the USA.) When you dual-stack your server, that traffic no longer needs to traverse the NAT, so users may see better performance.

There's no way to avoid NAT in general, because there simply aren't enough IPv4 addresses to go around, but ISPs should be able to deploy NAT equipment with the assumption that the operational costs will decline over time as IPv6 becomes more popular.

Putting stateful NAT boxes everywhere is antithetical to the concept of a free, neutral network, so we should all be striving to make the alternative viable.

Note that with around 19k people connecting from ipv6 there is a 50% chance that two will have the same pseudo IP, so you can't use these for unique IDs alone. At 50k you hit 99% chance of collision.

[edit] I originally did this with a 24-bit rather than 28-bit space, so my numbers were way to low.

Whereas with IPv4 there is a 100% chance that two users are coming from the same IP via NAT.

Given that protecting the source IP is not a goal (keyspace is far too small for that), why use something like MD5 when something like CityHash or MurmurHash3 would do?

There are no reasons to use CityHash or MumurHash in 2014. If security is not a concern, xxHash is faster.

We have a fast MD5 implementation available through ngx_lua so using it is easy. Using one of the hashes you propose would have meant creating an API to access one of them and adding them.

How are they using class E?

I looked into enabling that for virtual networks in my app ( www.zerotier.com ) and quickly discovered that Microsoft Windows has hard coded these IPs as unusable. On a Windows box the IP stack will absolutely refuse to talk to the 240 block. I spent a few weeks looking for a workaround and could not find one, but I did talk to someone who used to work in MS and he confirmed to me that it was a hard-coded prohibition.

The IPs aren't being used for actually traffic routing. They're just included in the HTTP header so the web application can have something to use in any session, abuse, anti-spam, etc functions. You are correct that routing across Class E is practically impossible due to hard rules in place in Microsoft Windows and elsewhere.

In my research Windows is the only culprit in the OS space. There might be router firmwares too but I'm not sure.

The entire Windows networking stack from top to bottom is a tangled mess of pain.

Does anyone know of an affordable pingdom like service which supports IPv6?

Not having a convenient way to monitor the IPv6 side of things is my last reason for not enabling IPv6.

Pingdom uses AWS. There's a million of services and platforms (e.g.: Heroku) that don't work on IPv6 because AWS doesn't support it in EC2 (and services built upon EC2 like RDB and whatnot).

I'm not sure how we can talk AWS into supporting IPv6 :)

464XLAT: Breaking Free of IPv4: http://youtu.be/Xl-hIyZSAmA

I haven't seen a blogpost about it yet, but seems like Cloudflare is under heavy attack recently that causes performance degradation: https://twitter.com/CloudFlareSys Any idea when it'll be all mitigated and performance back to normal?

Congestion happens all the time on Cloudflare, they re-route it and there's not much to be worried about.

If I use an IPv6 gateway, does that mean what I think it does: all my traffic is going through their network?

That's the nature of CloudFlare.

I saw the second snippet in the post and thought, is the time you lose in that string.format significant? The real question follows: what do you use to benchmark this kind of code with consideration for luajit and also for your server architecture?

geo targeting seems to be another area of fail for ipv6. Had to disable it for netflix in UK when I was using a tunnel as it thought I was in the US so would not stream. Now I have native it might be better not sure.

That's your tunnel/ing software not supporting IPv6, I think. I experienced the same issue with Netflix in Australia.

Totally unrelated, but why the heck does the cloudflare blog have some sort of facebook iframe ad popup? When it fails (blocked by AB+) it covers up part of the article.

Noticed that too.

Have not been able to duplicate what you're reporting. Please try to duplicate in Chrome's Incognito mode (without any of your extensions enabled)? This sounds likely to be related to one of your extensions.

Seems to be a bug in HTTP Switchboard extension.

Not a bug. It's just the web page which styled the Facebook frame to be way larger than it needs to be -- given it's just to receive a Facebook button.

Edit: This will happens with anything which blocks Facebook frame, but not Facebook script.

i get this: http://i.imgur.com/MoeEuZ2.png which is caused by this: http://i.imgur.com/RwZluqB.png

i fix it by right clicking and choosing inspect element and deleting the node.

i believe the issue i have is caused by barracuda blocking the iframe's content.

If you are blocking Facebook frame, I will assume you don't mind blocking the matching Facebook script, so add this rule to your ABP filters:

    ||connect.facebook.net^$script,third-party

Edit: or really, just:

    ||connect.facebook.net^$third-party

Now you can download memes from 4chan marginally faster!!

(I'm really glad they did this, because at 4chan's patented "SOON™" dev pace, it'll take us another decade to add native IPv6 support.)

Next up, Hacker News. :-)

>memes

We changed the title ("Eliminating the last reasons to not enable IPv6") to a more neutral and informative subtitle.

The new title is not more informative; "Pseudo IPv4" is a new term that is meaningless before reading the article. The original title was more informative, especially in context with the URL domain.

The previous headline was creating too much cognitive dissonance, given that HN is a CloudFlare site with IPv6 disabled.

Sorry, I didn't see this reply until now. I still think it was more informative. You left out the "Introducing", which makes it clear that the post is introducing a new product or technology, with a name related to IPv4 (and therefore, likely, to IPv6). That seems to me to say a lot more than the article title does. But I see how one might disagree.

The real problem with the original article title is that it is linkbait (grand claim about a controversial topic) and misleading (there is far from universal agreement about this), and so violates both of HN's guidelines about titles. Therefore it needed changing, and when doing so we try where possible to take language from the article itself rather than inventing something of our own. A subtitle is often a good choice, because it's often what the article would have called itself if it weren't trying to be sensational, and that seems to me to be the case here.

Reply on news.ycombinator.com