IPv6 support for cloning Git repositories from GitHub | Hacker News Reader

IPv6 support for cloning Git repositories from GitHub

github.com

245 pointsstargrave78 comments

I also don't understand what stops ycombinator from supporting IPv6. It's a pretty simple website, what's the big effort?

The most touted reason is that their anti-spam systems only support IPv4. Their old Cloudflare endpoint however is still alive and you can't disable IPv6 on Cloudflare so feel free to add the following to your /etc/hosts:

    2606:4700::6810:686e news.ycombinator.com

Interestingly when I tried to post the above comment over IPv6 I got a Cloudflare "You have been blocked" page. This might be something they do not want you to know! :D

Wouldn’t it be reasonable for their backend to only accept (write) requests from whatever the anti-spam proxy is? Otherwise, there’s little point.

Currently there are no proxies in front and you connect directly to their baremetal server hosting the site. I presume the anti-spam system is custom-built and part of their own codebase. Cloudflare is officially sanctioned, but retired from widespread use.

You in fact CAN disable ipv6 on Cloudflare, but they make you do it with an API request.

This was possible only for some time, now it's only enterprise option I'm afraid. For Free/Pro plan option is grayed out and API refuses to change it.

Doesn't that still only remove the records from DNS? So far for all Cloudflare sites that IPv6 disabled I've been able to derive the IPv6 address by hand and make requests without issues.

Interesting that apparently this is a problem, I would have thought that spam filtering is completely outsourceable by now.

Doesn’t CloudFlare have good bot detection? What does HN do that relies on IP addresses that CloudFlare can’t do?

HN can do things like "This user is posting from an IP which geolocates far from where it normally posts from". It can take into account the total post history, user upvotes, etc.

Cloudflare bot detection is more request-by-request. Cloudflares product is more intended to prevent DDoS attacks with millions of bots. I don't think it's sufficiently fine tuned to prevent a handful of spam comments through.

I know of so many websites that break spectacularly when you do that...

This was an interesting Cloudflare "feature" I found out about the hard way. Even if you only use Cloudflare for DNS hosting, they will happily accept proxied requests for your hostnames and route them to your origin. I discovered this when we received a L7 DDoS from only Cloudflare IPs - the attacker had pointed their bots at Cloudflare with our hostname (bold move!).

The official solution (and might be why you see the blocked page) is to set up the WAF to block all requests.

Do they also accept proxied requests for domains that are not yet verified?

Could you elaborate on this? What is being proxied, http requests? And can you use any CF IP?

Yes, HTTP / HTTPS requests can be proxied this way. Any CF IP seems to work. HTTPS only works if the target hasn't disabled Universal SSL (i.e, they have a TLS cert provisioned on Cloudflare's IPs).

We're just slow.

I truly feel this statement in my bones the longer I'm in technology...

How does ipv4 vs ipv6 affect speed?

They're slow in migrating, not there's a difference between IPv4 and IPv6 performance.

Considering how long it took AWS to add IPv6 to services across the board, I'm not surprised that it's taking so long. On the other hand, it would be nice if they would be transparent about the challenges or the reason for the delay, rather than radio silence or, at best, "we're working on it."

It's debatable to extent that AWS has IPv6 across the board. Many seem to be using a 6 to 4 layer under the hood which can result in noticeable behavior.

> Considering how long it took AWS to add IPv6 to services across the board

Unfortunately all but a handful of their APIs have yet to support IPv6.

My ISP (Metronet) uses CGNAT and refuses to touch IPv6. In my case, when I complained that port forwarding didn't work, they gave me a static IPv4 for free, but I have to call back once a year or else they start billing me $10/month for it.

I don't need a static IP. I'd be completely fine with a dynamic IPv4 or even dynamic IPv6. But they don't offer that. Just static IPv4 or CGNAT IPv4. Oh well, some day...

With most people leaving their router switched on all the time, the difference between a static and dynamic IPv4 address from the point of view of the ISP is probably marginal.

Yeah, that's true enough.

I guess the point I was trying to make is that I think IPv6 is a better solution to their problem of not having enough IPv4 addresses.

In Germany all bigger DSL providers still disconnect you once a day and issue new IPs.

Bad for at home hosting, good for privacy.

Is that so? At least Telekom doesn't do that for IPv4 anymore. They do have a 24h dynamic prefix for IPv6 though (which feels very weird, considering they stopped doing that for IPv4)

The ephemeral IPv6 addresses are just the IPv6 privacy extensions at work. They're just following RFCs

I guess their router stack provides this for free, while the "Zwangstrennung" (disconnect every 24h) was implemented somewhere on their side. So it's actually easier now for them

Partially. Telekom keeps up the line for months at a time. I guess that is due to telephony being done via VoIP, and they don't want to interrupt your late night calls.

Others, even Congstar (which is a cheap telekom brand), do 24h disconnects with a new v4 address, and no v6 at all.

The DSL I use gives me a fixed v4 and v6 range, but still needs to do the daily disconnects.

how do you know what ALL of the them do??

The difference is typically between a static _public_ IPv4 address or a dynamic _private_ IPv4 address, and CGNAT sharing public IPv4 addresses across subscribers.

For a long time I ran a HE tunnel to get me some sweet static IPv6, but now that my cable company has turned it on I no longer need that (probably should still have it as a backup).

https://www.tunnelbroker.net

Guess they are sticking with their old equipment, because IPv6 is free in any modern industrial ISP router.

I also have Metronet, but haven’t found any compelling reason for a static IP. I use Tailscale for remote access, and though I don’t host anything from home currently, Cloudflare Tunnel should work.

For about 4 years I have considered IPv6 first and IPv4 second. If IPv6 has an issue, I consider the service down, not just half down or slightly non operational. If I call an ISP for an IPv6 issue, I say "internet is down" even if IPv4 is working.

This policy helped move things forward on the networks I worked on. Lately I did setup a business internet with SLA, I specifically told the ISP I would not accept the contract if the SLA did not mention IPv6 as required.

But it is still a lot of battle, where it should be the default.

Github not fully supporting IPv6 is a real shame and they should really move things forward to support it quickly.

Also, systems should not use IP addresses as a mean of security or authentication, it was a bad idea for IPv4, it is even a worst idea for IPv6. To give you an example of bad firewall behavior, I was checking my electric bill from the train, and suddenly my account got blocked, and it took me a lot of time and effort to fix (physical mail...). My IP changed while I was browsing a page and the firewall didn't like it.

> For about 4 years I have considered IPv6 first and IPv4 second. If IPv6 has an issue, I consider the service down, not just half down or slightly non operational. If I call an ISP for an IPv6 issue, I say "internet is down" even if IPv4 is working.

Wow, you live in a very different world than me. If I did that, I can 100% guarantee that the answer from the other end of the line would be, "The Internet is working for everyone else just fine, maybe try clearing your cookies. Have a nice day. click"

That's the difference between residential and business class broadband. My isp in ireland, virgin media, has fairly useless support for residential, but for business, they are on the ball. And for enterprise (dedicated line in the office) they are even better. Suppose it depends on what you pay for.

I was speaking for business lines/contract. They usually have actual support with SLA. Residential is a lottery but local (public owned in a small village) will usually care.

This shortcoming becomes immediately apparent when you try to use certain VMs, like from Vultr, which are IPv6-only with no CG-NAT. You can't clone anything or fetch any release binaries at all.

If your VM provider issues IPv4 addresses you can run into another issue: your v4 address might be dirty. I recently spun up a development VM and was unable to download packages from maven.org. Apparently the address had previously been used for abuse and ended up on a blocklist.

Hmm, interesting. I tried Vultr a few months ago and had a number of issues, wonder if that was related. Is it common for a provider to only give out v6? My experiences is really only with Linode - which I've never had a problem with for years, and a bit of playing with DO which seemed fine but didn't wow me enough to move infra.

Hetzner sells v6 only dedicated servers, you have to pay a little extra for a v4 address now. So yeah, I'd consider it pretty common.

I have a weather station I run on T-Mobile which is v6only with a ipv4 CGNAT. I just Cloudflare the v6 endpoint and my legacy (v4) users can visit the station.

I'd be more accurate to say it's becoming common for providers that compete on price to give IPv6 a price advantage. I don't use Vultr, but they seem to occasionally have $2.50/month instances with IPv6 only. Hetzner charges you $0.50/month for an IPv4 IP for cloud instances, and $1.70/month for one for dedicated servers.

As others have said it's getting more and more common on the low-cost providers (especially if you get outside the US/Europe and into Asia).

But even then they often have an ability to get a NAT IPv4 connection out somehow.

This is sad :( hetzner charges extra for ipv4 address, and this means I couldn't run `git clone` without paying extra.

with no NAT64 gateway or something similar to it?

pretty lame by hetzner if that's the case

IPv4 is only 0.64€ upsell

Unless you have a dedicated server, then it is quite a bit more expensive.

2 euro if you want one primary ip is not that bad. If you want extra, yes, they charge more for a setup (20 up front for 1, still only 2 quid per month extra).

Hetzner has an official NAT64/DNS64 gateway you can use with their v6-only offerings.

I can't seem to find any documentation on this. Details?

Think of it as providing a discount for going v6 only. Every single provider is charging you to have a v4 address. They aren't charities. Some providers just let you opt out of paying for that.

I think v6 only server but also have like NAT64/DNS64 (or just a HTTP proxy) for v4 internet access is practical for now. Even if dedicated IPv4 address isn't needed, most people may want v4 internet access.

> This is sad :( hetzner charges extra for ipv4 address, and this means I couldn't run `git clone` without paying extra

Well, they added the Option, so you can get your server for less then normal. The Servers are cheaper, if you Opt-Out of IPv4. I really liked that move.

Meanwhile in both India[1] and China[2] (two biggest countries by Internet users count) IPv6 is mandated by the national policy. Everyone else should do that, otherwise the transition would never be finished. ISPs and other network businesses should be forced to do upgrades by the law or policy, otherwise they will never allocate budget and resources for that.

[1] https://www.indiatimes.com/technology/news/india-sets-new-de...

[2] http://www.stdaily.com/English/ChinaNews/202208/e154b19bb5b0...

I have had a dual stack at home and work for around a decade now but "Everyone else should do that" is a bit proscriptive.

If it ain't broke (and it really isn't quite yet) then I suggest we crack on. IPv4/6 are simply transports, one has a larger address space and quite a lot of attitude! There are translation mechanisms so it is unlikely that anyone will be left behind. As systems move to IPv6, parts of IPv4 space are released and 6to4 n that tunnels can patch up the holes.

You need to learn patience. It took me about two years to persuade a firm with around 6000 employees to deploy DHCP back in the day. I made sure it was everyone else's idea and took my time. That was a tiny thing. This is the entire internet and it requires a massive mindset shift, engineering, purchasing and what not.

I'm going to tentatively put IPv4 -> IPv6 in the "paradigm shift" category. It isn't really technically: the wires (ethernet etc) are the same but the bits are somewhat different!

If you really want to get steamed up then why not debate the semantics of how multi-WAN connections should work with IPv6? Suppose you have two ISP connections for WAN and hence two lots of addresses. How do you deal with an ISP outage? How do your PCs know which set of source addresses to use? Do you use NAT64 or NPT or something else.

Another thing to consider is how do you "bootstrap" your network with IPv6 and how do you deal with a change of ISP? Do you set DNS servers with ULA addresses so they stay static or what? Bear in mind that SLAAC doesn't give out DNS servers. OK, lets do DHCPv6 ... not on Android ...

IPv6 needs some care. It has been messed and muddled around with so many times and it still has some gaping holes. For me the biggest problem is the righteous indignation you find at nearly every turn where stuff gets broken for its own good.

It all starts to go wrong with "everyone should"!

The starry eyed approach that you think that India and China espouse is simply twaddle. No one really thinks that in the real world, despite what is said on TV. Nation policy of that sort is normally a case of "Do as I say and not as I do".

In my opinion we should damn well continue to muddle along as best we can with what we've got. We will patch the flaws and paper over the cracks because that is what engineers do.

Another elephant in the room: /64, /56, or /48. The first one is completely unacceptable in the modern world, the second is acceptable and the third is desirable ... per ISP connection.

If you only get a /64 ie one IPv6 subnet prefix then you are only a tick in a box.

Ideally you also get a separate uplink subnet too along with your shiney prefix for WAN. There is a RFC that will enable a sub-prefix from a prefix allocation to be taken out for WAN and make it all work. Sorry if that sounds like gibberish - I won't explain that lot here!

There are so many things to get sorted with IPv6 - it is not a finished thing. It's only about 40 or so years old.

I think what a lot of people like to miss is that a lot of detection and antispam stuff is not working well on ipv6. A server without any ipv4 is still limited in many more ways than not being able to reach github which probably means there is not a lot of pressure for github yet.

Any quick info on why anti-spam/bot detection is harder on IPv6?

My guess is that each user's IP suffix changes a lot more often

Probably because with IPv6 privacy is built-in somewhat into the protocol, eg you can have a different IP really easy. For example, I can see my desktop right now has 7 different addresses.

Now, you could truncate this to eg a /64 or /56 range to identify users, but each ISP has different rules. Mine gives a /56, but I also hear many give only a /64 or less.

As such, it basically means that you can’t really rely easily on IP addresses anymore for spam detection, rate limiting, etc.

Note that I’m not an expert on spam filtering, but I do have quite some networking experience and QoS, and ran into these issues a lot.

It's also the fact that having a datastructure that stores few bits per /24 range in RAM is very doable in IPv4. Banning a /24 doesn't have too much collateral damage.

Whereas the same in IPv6 isn't feasible. There is no reasonable way to divide the IP space non-sparsely and keep in RAM and still ban without ending up banning a whole ISP.

A simple HashMap works fine for blocking IPv6 /56.

> Banning a /24 doesn't have too much collateral damage.

Even at 128:1 CGNAT ratios a /24 has 32,768 potential users.

By comparison, a /56 has 256 potential users on a /64 each.

Filtering by /64 is good enough. With a /56 you have 2^8 (256) prefixes, if you spam enough for a /64 to be blocked, you have 255 more tries before all of those are blocked too.

With some heuristics of "hey, we saw two /64's from the same /60" you can catch most ISP's that are offering prefix delegation to their customers, and that's only 16 /60's in that /56 before you are fully blocked...

It's not that much harder or difficult.

But this same issue occurs with CGNAT IPv4, whose private address delegation is even more opaque than IPv6's prefix delegation. And CGNAT will become more prevalent going forward as address exhaustion becomes a bigger issue. There's no circumventing the fundamental problem that there is no 1-1 correspondence between IP addresses and "real" users.

Collateral damage is accepted for CGNAT, which is widespread already, and will be equally well accepted for v6 /56s (or /60s).

Because IPv6 addresses are free and IPv4 is expensive. Same reason why Google won't let you sign up without SMS verification. If you're caught spamming or breaking TOS you've effectively burned that v4 address or phone number.

v6 is more difficult, by design. The lower half of the address is deliberately not subnettable and it is the explicit design intent that machines on a v6 network can just make up new addresses within a /64 as they please. So you have to burn subnets. Except there isn't really a standard for how subnets are issued: most ISPs hand out /48s, Comcast insists on /64s for residential use, etc. In the IPv4 world you could ban one IP at a time, and only move on to banning entire AS allocations if you needed to. On IPv6, banning a /64 is a lot less impactful, so you have to start with the most drastic and customer-hostile option.

Comcast hands out a /60 for prefix delegation if you ask for it (i.e. software asks for it, no customer service interaction required). In fact Comcast allows you to ask for as many /60's as you want (caveat, there may be a limit, but at one point I made a config mistake that led to asking for 32 /60's and I got all of them, so I am not aware of a limit).

Lots of organizations do not support IPv6. For another example, Heroku does not (and many systems are based on Heroku): https://help.heroku.com/I8L6RW01/does-heroku-support-ipv6

It's unfortunately harder to support IPv6 than I think it should be, so many organizations do not. I'd love to see GitHub support IPv6, but they are by no means the only one.

from the NAT64 gateway in the thread I learned from ipv4 mapped ipv6 addresses

    2001:67c:27e4:1064::140.82.121.3 github.com www.github.com

if your curious too, see https://www.rfc-editor.org/rfc/rfc6052#section-2.4

Does this use a gateway?

edit: yes this is a gateway or "translator" (NAT), you can find a list of other public ones at https://go6lab.si/current-ipv6-tests/nat64dns64-public-test/

I have a dual stack DSL connection and do not particularly favor IPv4 or IPv6. But I just looked at my PiHole DNS statistics and I see that AAAA requests have taken over A requests by now, 47.2% vs 46.2%. Not much but it's something.

I wasn't aware that over half of my internet traffic goes over IPv6 already

That's just DNS requests, not traffic. You'd expect the number of A and AAAA requests to be about equal, because software simultaneously looks up both record types for each hostname in order to determine which protocols the hostname is reachable over.

A number of high traffic sites (e.g. Youtube and Netflix) have v6, so you might find the percentage of your Internet traffic that goes over v6 is actually much higher than that.

That doesn't mean much. browsers etc usually request both AAAA and A in parallel, then try to connect to both in parallel (as soon as either reply comes in), and use whichever established connection first.

Look for "happy eyeballs"

That being said, looking at real stats (number of packets/bytes through ipv4 and ipv6 firewall on my home router) I have a x1.56 ratio in favor of ipv6. It really depends on what content you consume though.

Well, if Github nowadays runs on Azure under the hood (which they probably don't) I understand. IPv6 support in Azure is patchy with many of their services.

I think they run their own servers; a little bit of DNS poking, followed by a traceroute, led me to what I think is a colo company called Twelve99. This is just for my region; I imagine they run servers in different datacenters all over the place, and maybe have some sort of presence in AWS or one of the other public clouds.

Because of this post I decided to check my network interface statistics. I'm in the Netherlands and my IPv6 usage is ~10x that of IPv4.

in the US almost nobody uses v6. consumers or businesses

Reply on news.ycombinator.com