98% of sites on Cloudflare now use IPv6 | Hacker News Reader

98% of sites on Cloudflare now use IPv6

blog.cloudflare.com

116 pointsjgrahamc62 comments

Why are Cloudflare still sharing IPv6 addresses, when there should easily be enough for one per website? (And then with correct reverse DNS.)

The linked site [1] has a download of 134k hostnames. Filtering for the Cloudflare prefix, 2400:cb00:2048, there are still plenty of sites sharing an IP. For example, www.monolith.agency (a design agency) is on the same IP as www.bobshouseofporn.com (porn).

Maybe the same company hosts both websites, and it's not Cloudflare's issue, but that seems unlikely for a US porn site, Quebec design agency, Brazilian health site and Spanish programming site.

Google have 13,000 sites on the same IP, 2607:f8b0:4005:808::2013, looks like Blogger.

[1] http://www.employees.org/~dwing/aaaa-stats/

Something like:

    egrep --only-matching 'IPv6.+?  ' ips | sort | uniq -c | sort -n | grep 2400:cb00:2048

Why not? Seems like a nice easy privacy win.

The design agency website might get blocked as collateral by a poor filter, which knows of the porn site.

I can imagine a government that would enforce that ISPs block adult sites by default (a specfic government comes to mind), and as those sites would both use HTTPS (as websites do), the design agency's website would be blocked by default as collateral damage.

"Sorry, your website is blocked by default on all ISPs. It's to protect the children."

    monolith.agency has address 104.28.8.19
    monolith.agency has IPv6 address 2400:cb00:2048:1::681c:813

Notice that the IPv4 address is embedded in the last 32 bits of the IPv6 address. I would assume that this allows both addresses to be generated from the same configuration, rather than trying to keep two copies in sync.

As far as I know having very large numbers of IPs on a system can cause trouble with normal network stacks. Why do you want an individual IP address?

Cloudflare, you have my login token cookie but you are still asking me to prove I'm not a robot. Please make using a VPN not to be a punishment since all the sites that use your SSL show me the "I'm not a robot", no matter how often I verify it. I am most times under an insecure WIFI so no VPN is not an option for security. Possible steps:

1. Make me solve it only once every X minutes/hours.

2. Make the defaults to be one step down in security, probably most webmasters don't want to block legitimate people using VPN.

3. Make it dynamic, so only those under suspicion have to do it. And consider being using a VPN NOT to be enough suspicion for it.

Right now I have to choose either to:

- Compromise my security: don't like it now, cannot do it when I start working with the new company I'm going to work

- Solve hundreds of "I'm not a robot" per day

> I am most times under an insecure WIFI so no VPN is not an option for security.

Maybe apply some netfilter or proxy magic to push traffic to port 443 over the wifi and everything else through the vpn?

These settings are available but it is currently up to each given site to enable it.

That was exactly my point 2: Make the defaults to be one step down in security, probably most webmasters don't want to block legitimate people using VPN.

That seems like it might be a difficult sell for a company that considers itself a security company for the benefit of a relatively niche use case.

It is great to see IPv6 finally taking off. I remember being exited about IPv6 back in 2003. I was fortunate to be on a university network with great admins and made sure to enable IPv6 on my Linux computer. Then nothing much happened for years and years and years. Not until around 2011 did the numbers start ticking up much above 0%, and now we are in the early part of the steep slope.

I believe most personal routers, such as the ones used to move internet from a cable-box/fibreoptic to your personal computer now use IPv6.

Interesting that Google doesn't seem to use ipv6 for their crawlers. They seemed to be big supporters for ipv6, but they don't appear anywhere on the list. I'd expect them to cause much more traffic than Facebook.

Any clue why they only crawl via ipv4?

Does google use cloudflare? If not then their data probably wouldn't show up.

They're in the chart. They're hovering near the bottom around 0%.

Google is a partner of cloudflare. They have their own CDN though.

Not directly, but their crawler will use Cloudflare. same as with Facebook. They don't use Cloudflare but crawl websites that use it.

Because web sites won't be moving to ipv6 (only) any time soon, and the limiting factor here is residential connections, not Google?

Are you proposing that Google should crawl ipv6-only web sites? And then serve those results to whom, exactly? (I don't think there is any reliable way of knowing whether an HTTP client that's connecting to you via ipv4 is also capable of ipv6?)

Facebook seems to crawl ipv6 websites using an ipv6 connection. The Google crawler doesn't show up so it's probably working ipv4 only.

I'm just wondering why Google doesn't seem to use ipv6 where available.

You mean they should prioritize ipv6 over ipv4 if a DNS lookup returns records of both? Why?

Many operating systems now prioritize IPv6 over IPv4. Otherwise there wouldn't be much IPv6 traffic at all.

If only v6 is available there is no point for them to include the site in their general index.

If both v4 and v6 access is available, what is the extra value of doing the indexing using v6 rather than v4? (The indexing process itself would add very little to the global usage of v6, comparatively speaking.)

In that case, what is the extra value of having ipv6 at all?

And Facebook seems to prefer ipv6, I guess there's a reason for that.

Facebook engineers also declared Mark Zuckerberg dead the other week. Maybe there's no deeper lesson here?

http://www.theregister.co.uk/2016/11/11/facebook_zuckerberg_...

That is what RFC6724 recommends as the default:

> Another effect of the default policy table is to prefer communication using IPv6 addresses to communication using IPv4 addresses, if matching source addresses are available.

https://tools.ietf.org/html/rfc6724#section-2.1

Because otherwise we don't need ipv6. Google advertised the use of ipv6 heavily. If everything still works fine with ipv4 and NATs, why should we ever switch? It's a serious question, I know that the ipv4 space is somehow exhausted, but I can still get very cheap ip addresses (one is included in every $5 vps at major providers).

If even Google with its huge network of crawlers doesn't see the need to communicate via ipv6, why should there be any demand?

> I don't think there is any reliable way of knowing whether an HTTP client that's connecting to you via ipv4 is also capable of ipv6?

Send XHRs to a v4 subdomain and a v6 subdomain, check which requests succeed.

Sure, messy, but would generally work.

I guess they could also move this checking to the user agent (because they control Chrome) to have the check performed before the user is actually waiting for results and milliseconds matter. Introduce some ugly but pragmatic HTTP header, like "Accept-IP: v6". :)

So, this way they could serve v6-only results reliably to clients that can access either both v4 and v6. (People who can only access v6.. that must be a very special kind of people.)

Because their crawler is so monolithic that it would be expensive and annoying overhauling it for IPV6.

There is a great use-case for IPV6 for IOT where each device gets its own IPV6 address. IPV6 addresses are appearing more like MAC addresses at this rate as IPV6 is not exhausted yet.

>is not exhausted yet.

The world will be a completely unrecognizable place when this is even a slight concern.

That's what they said about IPv4!

But seriously, there's an astronomical # of addresses in IPv6. You're probably right that if we ever exhaust that space, we'll probably be communicating between planets by then.

> That's what they said about IPv4!

And they were right!

There is an astronomical number of individual IPv6 addresses, but in most cases that is not really the meaningful number to look at, at least right now. IPv6 is not really supposed to be subnetted beyond /64, so that already slashes the network space quite significantly. ISPs are supposed to hand out full /48s to customers (probably does not apply to consumers though), so there goes another 16 bits. The basic unit that RIRs give to ISPs is a /32 (afaik). Which leaves far less astronomical number of individual networks left. 2^32 - 2^48 is no doubt still a pretty big number, but not really as mindbogglingly humongous as 2^128.

> There is a great use-case for IPV6 for IOT where each device gets its own IPV6 address.

Do you really want your IOT devices to be directly addressable on the internet? It's my understanding that having devices behind a router is safer. I go a step further and disable UPnP on my routers and everything still 'just works' including network printing.

Why does 'directly addressable' mean 'not behind a router'? Unless you've got a weird ISP that's delivering you Ethernet, you're going to need a router.

That's a good point and I don't know the answer.

I have a gigabit fiber (to the home) connection which terminates at a device with 4 Ethernet jacks. They all work, I've tested connecting directly to them with a laptop, but I plug a router into it and all devices connect through that router instead. It's the 'stateful firewall' aspect of using a router that I want for improved security. https://en.wikipedia.org/wiki/Stateful_firewall

My ISP delivers me Ethernet... I doubt it's that uncommon in midrise/highrise buildings. But they will only route to a single IP address (incl for 64-bit) so then again I still need a router.

NAT is not a security feature, it wasn't meant and it doesn't by itself add anything, except complicates communication.

You supposed to control access with firewall, and controlling security is much easier when computer/device has a routable address.

Though, IoT devices should probably be restricted of any Internet access based on their security track record (but again, this is orthogonal to being directly addressable).

While NAT does not provide perfect security, it is a component of security in networks where most people have no idea how to harden their systems or devices. It somehow gives me comfort to know that no one can just scan the net to find my phone, as I'm not sure if it would be vulnerable.

I still don't see a reason for the average consumer to have a static, reachable IP for their devices. I see privacy concerns but no advantages.

The crawler supports IPv6-only servers, but IPv4 is preferred. For example:

https://www.google.com/search?q=site:ipv6.whatismyv6.com

"You are connecting with an IPv6 Address of: 2001:4860:4801:6::1f."

But why do they do that? That's what I don't understand. Google was one of the companies advertising ipv6 as the future, why don't they use it for their crawler?

Interesting to see that Greece is the third (possibly second?) largest IPv6 deployment per-capita in the world. One of the ~3 large providers here still hasn't added IPv6 (although I might just have an old router).

I'm still trying to figure out how to set up static IPv6 so I can access my computers at home without a NAT, but it's very convenient otherwise!

Which one? I get IPv6 from (Cosm)OTE.

Do you? I have Cosmote as well but I'm on IPv4, I guess the router is old. I don't think Cyta has IPv6 yet either (they didn't when I switched away to Forthnet last year, exactly for this reason).

Yeah, I have my own box. But if I remember correctly with the new ZTE routers they're shipping it works too.

Your move GitHub and Reddit.

And HN too.

As soon as AWS gets serious about IPv6...

Now that they got CloudFront fully supporting it, it seems like it's just a manner of time until VPC/EC2 is IPv6

And Google Cloud too.

All Compute Engine networks use the IPv4 protocol. Compute Engine currently does not support IPv6. However, Google is a major advocate of IPv6 and it is an important future direction.

Not anytime soon. Asked them about it just the other week. We would love to get the IPv6 support as Google is not very eager to allocate IPv4 to us in any larger quantities.

Yeah, same issue. I'm a bit surprised that they didn't bake this in from the start with GCP. Can't even terminate IPv6 on a Google Cloud load balancer and then do v4 internally. At least GCS does IPv6.

Now ISPs should also stop dragging their feet.

They want to rip off the customers first.

Hopefully Amazon will add IPv6 support to AWS VPCs sometime in the next year.

We're happy to do IPv6 for you on the consumer side and connect to you backend over IPv4 if your backend provider can't deal with IPv6.

+1 - We actually do that automatically for you when you signup for Cloudflare.

They have little incentive. Would you change provider just to get IPv6?

I was quite wrong - IPv6 was just announced by AWS :-)

> IPv6 is faster for two reasons. The first is that many major operating systems and browsers like iOS, MacOS, Chrome and Firefox impose anywhere from a 25ms to 300ms artificial delay on connections made over IPv4.

He forgot to add that this only applies to dual-stack hosts...

And, unless something is serious broken in all of those systems, only if the target is also dual stack.

So an IPv4-only website should not incur any delay.

Reply on news.ycombinator.com