IPv6-only subnets and EC2 instances
aws.amazon.com
aws.amazon.com
On v4 it is 169.254.169.254 hence the final 254 (and other services like DNS at 253), but using the ULA range of ec2 is a simple, effective touch.
I prefer the latter as it's far simpler.
iptables -A OUTPUT \
-d 169.254.0.0/16 \
-m owner -j REJECT \
'!' --uid-owner 0Put yourself in AWS's position. Your customer is running a VM which you have only hypervisor level control over. Could be Linux, could be AIX, could be an appliance. How could you possibly implement user/process level security from the outside? How could you know what processes inside the opaque black box are the privileged ones?
> From AWS’s perspective the whole VM is in the same security context.
Which could be a problem if intention was to allow it, as your sibling comment points out.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configur...
When you use the AWS SDK, it abstracts it away anyway
Maybe a serial device or something like that would be more appropriate.
(At least, I was trying to point that out, but re-reading my post it's obvious that I didn't do a very good job.)
The cynic in me believes Google only did this due to the US Government's IPv6 only memorandum issued last year[1] requiring networks operate as IPv6-only.
[0] https://cloud.google.com/vpc/docs/vpc#ipv6-regions
[1] https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-0...
Cannibalising reserved address space like 127.0.0.0/8 will only get the IPv4 internet so far. Eventually, the scarcity of IP addresses will make IPv6 the more business-conscious choice for many applications (like API endpoints and other services that you call from your own infrastructure where you can ensure IPv6 availability). Maybe the free tier will eventually stop coming with an IPv4 address, that might also push them over the edge.
The decision to consume loopback address space has made me even more cynical about IPv6 ever becoming mainstream within my lifetime. Using IPv4 is like using fossil fuels — we know that they are going to run out/cause us harm and yet we keep using them. I don't know of a single ISP in my country which offers functional IPv6. All they do is offer CGNAT IPv4 addresses, which offers a crippled experience of the Internet.
Rationale: It's in the custoners' best interest to have unique publically routable addresses and without IPv6 we'll be stuck with CGNAT. The providers don't care, because rollout is expensive. So, for the good of the customer, legislation with fines is needed to better align the financial incentives.
There's a reason we have consumer protection regulation and much of it came as a result of situations where evryone thought, as you do, that the market/competition would solve the problem, but of course it didn't. That only happens in ideal markets where switching providers is trivial, the consumer is equally informed about all of them and the providers (and perhaps customers too) have equal economic power.
Your starting point was that any ISP who doesn't implement your preferred technical standard should be fined. With a disregard for whether IPv6 makes sense for them or their customers. You might detect a little pushback here on the idea that your ideas are real world. Laissez faire is a very practical approach.
> By the time it becomes, it'll be too late and nobody will have it yet, so there won't be anywhere to switch anyways.
A crisis so terrible I can't even imagine it. Nobody outside some fairly obscure tech circles would even notice.
Imagine if they'd taken this approach rolling out IPv4 and weren't fast enough to change the legislation. The rollout of IPv6 would look really hard if IPv4 was a legislated requirement. The EU just isn't a competent technical standards body, and it has no business trying to be one because a couple of tech types like IPv6.
> That only happens in ideal markets where switching providers is trivial...
I switched providers to get IPv6 support. It was pretty trivial.
"my preferred standard" == the only alternative to the current collapsing standard that has been agreed upon by industry leaders for some 20 years now.
> With a disregard for whether IPv6 makes sense for them or their customers.
IPv6 can't "not make sense", it can only "not be necessary yet".
> A crisis so terrible I can't even imagine it. Nobody outside some fairly obscure tech circles would even notice.
What, service costs going through the roof because the ISP now has to buy IPv4 addresses? Getting IP blocked from sites because the address you share with half the town got reported for spam? Not being able to access IPv6-only sites from smaller providers who can't afford to lease IPv4 addresses? And only then the ISPs start investing in it, which will take a few years to finish, all the while your service is shit? Pretty obscure, yeah.
> I switched providers to get IPv6 support. It was pretty trivial.
I've been trying to for several years and have not been able to for a multitude of reasons.
If it was profitable, the ISP would do it and pocket the now unnecessary service costs.
>Getting IP blocked from sites because the address you share with half the town got reported for spam?
99.9% people don't want to run their own SMTP server. It's an ultra-niche use case ISP don't care about.
>Not being able to access IPv6-only sites from smaller providers who can't afford to lease IPv4 addresses?
There aren't any.
Look, it's just an internet addressing protocol, not something that justifies legislation.
This is another "if you ask an economist" situation. In the real world, people will happily spend more effort justifying not doing v6 and working around problems caused by being v4-only than would be needed to just do v6.
> 99.9% people don't want to run their own SMTP server. It's an ultra-niche use case ISP don't care about.
This misses the point. It only takes one person behind CGNAT to get the CGNAT source IPs on a spam list, at which point it affects everyone. Plus spam exists on the web too, it's not an SMTP-only thing.
> There aren't any.
Which is exactly why it's so important to be deploying v6: so there can be v6-only sites. Or rather, more than there are at the moment, because they do exist.
> Look, it's just an internet addressing protocol, not something that justifies legislation.
I've seen an argument that it's already covered by existing legislation: the refusal of large/old providers to support v6, which forces the use of v4 which small/new providers can't get enough of, is beginning to look rather a lot like an abuse of market position. That would make it an anti-trust issue, which definitely falls within the remit of government.
Yea, I don't like silly economism too, and I know half the jokes too. However, right now IPv4 addresses aren't so expensive. Sure, way more expensive than the 0 bucks they should cost, but not that expensive in the grand scheme of things.
Here's a very rough calculation: The cost of IPv4 will start to bite when getting the needed addresses costs more than a single engineer salary where one lives. If a one time address buy costs less, it's not something business would care much for, and anti-trust won't touch it.
I'm not smart enough to be able to forecast pricing, right now it's not there yet. Eventually the price will force every public network to IPv6, while many private networks will remain IPv4 forever.
> Look, it's just an internet addressing protocol, not something that justifies legislation. "Just an internet addressing protocol" kinda puts the entire internet to shame doesn't it? It's important that people can communicate on the internet, with IPv4 we decrease reliability and increase cost (in the long run) for everyone except those who sucked up as many addresses as possible early on.
But if we are to follow your reasoning:
A common charging port is just a port, doesn't require legislation
A standard protocol for troubleshooting your car is just a protocol, doesn't require legislation
* which brings benefits to billions of people but you don't agree with, doesn't require legislation
That one thing you agree with that probably isn't important for a lot of people, requires legislation.
Let's raise the standard and discuss why IPv6 is necessary or not and what implications it has for consumers.
IPv6 decreases privacy because you're not sharing an IP with half of town, which is bad. Even if you're randomizing v6 on the local subnet you're still in the same /64 so it's still relatively easy to track you.
P2P protocols actually work properly with IPv6 since machines can commmunicate directly with each other.
IPv6 is more complex to understand the addressing scheme in.
IPv6 isn't IPv4, so there's a learning curve.
IPv6 encourages decentralization.
IPv4 enourages centralization, because you can't get addresses anymore you can't start up new ISP's and we could call that anti-competetive. It also encourages proprietary solutions like cloudflared, or just hosting your stuff on AWS rather than in a colocated datacenter, because you can't get access to IP addresses.
We can't stack unlimited people on top of IPv4 either, there are only so many port numbers available. And NAT is just such a major PITA to work with.
Obtaining an IPv4 address is not an issue I ever had with hosting. People are using Cloudflare, Akamai or ELB because they are sick of script-kiddies buying a DDoS for 10$ an hour and extorting businesses for 1000$ and/or because they have customers from other continents complaining about load times.
What if I have a great software stack in the pipe ready to be colocation deployed somewhere and I want to run my own network with multiple transit providers to be sure I don't go down with either of them.
When that possibility is dead, as mentioned in other parts of the thread we're getting awfully close to an anticompetetive market.
Why should existing on the internet be a privilege for the richest of the rich? (Those who can afford to buy IP's/ISP's to fulfill their needs).
But yes, Cloudflare is a great way to protect your site from skiddie "booting". with ELB I'm not sure about pricing, might just prefer it to die for the "booter" duration if it's a personal site, if business.. Well there's hopefully $$$ to foot the AWS bill.
It's not SMTP. It's StopForumSpam and CloudFlare that you need to worry about.
Your use case is not everyone's.
I was forced to change ISPs recently and didn't have a choice as to which one (which is another, unrelated, issue).
I did some research and the ISP in question does provide IPv6 support for some customers. However, the sales drones I spoke with claimed to not even know what IPv6 was, let alone whether or not it was available in my area.
I'd say that has more to do with the exorbitant fees they charge for static IPv4 addresses than anything else.
As such, it's not always "pretty trivial" to get IPv6 on your ISP link. I sure wish it was.
Edit: Clarified my prose.
That assumes it's possible to switch ISPs. Where I live, within the city bounds of the capital of my country, there's only two ISPs that can deliver broadband service and neither do IPv6 properly.
This is absolutely not a given. IoT has drastically changed the equation on how good it is to have devices exposed to unsolicited ingress connections from the internet by default.
This is speaking from experience of enabling ipv6 at an ISP. We were rewarded with: - happy geeks
- anemic ipv6 usage
- a bunch of abuse reports and customer support hours wasted on people with poorly configured IoT (or just old computers with no recent patches).
I guess you should implement this too?
If my understanding is correct, it's equivalent to using -m state --state ESTABLISHED,RELATED -j ACCEPT in linux/iptables where -P DROP is the default input policy.
Basically you allow ipv6 tcp traffic only for tcp connections initiated from within.
Or, we can move to IPv6. Yes, it might be painful for some people when they get their IoT devices hacked but at least it should spur change and make people realize that they can't just buy a toaster and connect it to the Internet.
That's why firewalls exist. And they work with IPv6.
My ISP gives out an IPv6 address to my Asus, which also picks up some prefixes for allocation via DHCP-PD. This causes my printer pick up an IPv6 address, but it is not accessible to the outside world.
Statefull firewalls still exist with IPv6, so by default connections from the general Internal cannot connect to your 'internal' systems. Hole punching still needs to be done with UPNP/PCP:
* http://upnp.org/specs/arch/UPnP-arch-AnnexAIPv6-v1.pdf
* https://en.wikipedia.org/wiki/Port_Control_Protocol
The advantage of IPv6 is that you no longer have to have things like STUN, TURN, etc. (Remember Skype super-nodes?) Your client knows its own IP(v6) address, gets the IP address of the other end, and then tells your firewall to allow connections between just those two addresses. Once your session is done the ACL is deleted and you're completely default-blocked from the outside again.
Copy-pasting from a previous discussion a little while ago:
---
An IP connection is started from the 'inside' to the 'outside', and the source-destination tuple is recorded. When an 'outside' packet arrives the firewall checks its parameters to see if it corresponds with an existing connection, and if it does it passes it through. If the parameters do not correspond with anything in the firewall's table(s) it assumes that someone is trying to create a new connection, which is generally not allowed by default, and therefore drops it.
The main difference is that with IPv4 and NAT the original (RFC 1918?) source address and port are changed to something corresponding to the 'outside' interface of the firewall. With IPv6 the address/port rewriting is not done.† Only state tables are updated and checked.
New connections are not allowed past the firewall towards the inside with either protocol, and only replies to connections opened from the inside are passed through.‡
There's no magical security behind NAT: tuples and packet flags are read, looked up in a state table, allowed or not depending on either firewall rule or state presence. The security comes from the state checking.
† It is possible to have private IPv6 addresses using ULA, and then the router/firewall uses NPTv6 to rewrite the prefix (leaving the /64 interface component alone).
‡ Just like with IPv4 (NAT), to allow unsolicited 'new' connections in you have to do do firewall hole punching with (e.g.) UPNP. But by default things are blocked.
---
> The idea that the internet should not involve the state is, honestly, pretty weird when you try comparing it to other stuff.
Although I agree in principal that enforcing standards has been (and arguably should be) in the domain of government functions, I feel the principal of keeping the "inter" in "internet" is more important. I'd love to see an incentivized conversion to IPv6, but there are many other incentivized conversions that I would wholly oppose. Thus I would prefer not to see government intervention in internet protocols.
Brazil is western
Wait what? Are you saying that they’re going to allocate the loopback range as v4 addresses?
Yes, there's a proposal to do so, which was discussed here last week (https://news.ycombinator.com/item?id=29246420 and https://news.ycombinator.com/item?id=29245466).
1. IPv4 private with NAT gateway and ELB dual stack providing IPv4/IPv6 for clients
2. IPv4 private with 1:1 ephemeral IP (avoids the NAT gateway tax, but now every VM gets a public IPv4 address) and ELB dual stack providing IPv4/IPv6 for clients
3. IPv6 globally unique with no IPv4 and ELB dual stack providing IPv4/IPv6 for clients
With option 1 you end up paying money for the NAT gateway for any outbound traffic instantiated from the instance (not return traffic through the ELB), this costs you 2 IPv4 addresses (1 for NAT gateway, one for the ELB, ignore AZ's and HA for now)
With option 2 you end up not paying for the NAT gateway, but get a free IPv4 address per VM, so N VM's + 1 ELB IPv4 address is N + 1 IPv4 addresses. (NAT Gateways are expensive, even if you have one sitting mostly idle, and don't forget cross AZ traffic if you have only one NAT gateway but instances in multiple AZ's, or you need to spin up multiple NAT gateways).
With option 3 you end up not paying for the NAT gateway, nor do you use a public IPv4 address. This might not work if your instance needs to reach IPv4 parts of the internet, but if its just API servers that communicate internally, they likely don't need to and can use IPv6 for the remaining traffic. Now you have just 1 IPv4 address on the ELB.
If AWS were able to incentivize its customers to deploy their systems in an IPv6 only configuration this would put pressure on the publishers of software to make sure their systems are IPv6 enabled so that their customers could download it, and it would reduce the amount of public IPv4 addresses that are required across AWS as a whole.
Running an IPv6 only core is entirely possible, and makes sense. No more worrying about overlapping IP space (having worked at two companies where they ran out of RFC1918 space and you had to deal with multiple hops to get into various networks, which is no fun), thus simplifying network management.
I get running out of 192.168.0.0/16, that is easy enough if you have a lot of locations and one /24 per location. But how in earth does one run out of 10.0.0.0/8 IPs, outside of a company merger?!
Twenty company acquisitions.
I've had to address that with partners/service providers. It just requires a double NAT to make it work. Which is a pain, but necessity is the mother of invention after all.
It's particularly silly because none of that is necessary at all. v6 means that each side has unique prefixes and you don't need to do any of the above. Even if you need to connect to someone without v6, all you need to do is NAT each partner's RFC1918 space into 64:ff9b:1:N::/96, with a different N for each partner. Then they're all accessible in a single unified address space.
You won't get any argument from me about the utility of IPv6 over IPv4.
I just pointed out that the scenario presented is workable with IPv4 -- in fact, I've done it more than once.
Sometimes you don't get the choice as to what IP version or addressing scheme you use. Not every environment is a "green field."
Back in the early 'teens, when we contracted/partnered with others that had overlapping IPv4 ranges, if I'd told my boss that "it would be so much easier to do this if we were native IPv6. Given me three-six months to migrate the routers at our dozen locations to dual-stack, replacing any equipment (don't worry, that'll cost less than $250,000) that can't support IPv6, then renumber our dozen class B's, re-implement our 802.1x infrastructure, reorganize our firewall policies and the myriad other tasks we need to do to migrate away from IPv4. By the way, I'll pretty much be engaged in this full-time, so you may need to hire a couple guys to do the work I'd otherwise be doing. And don't forget to tell the C-suite that they can't have the application access they wanted until this is all done."
I suppose I could have said that to my boss. But rather than doing that, then updating my resume as I'd probably have had a lot of extra time on my hands, I said "I'll make it happen. Please give me the contact info for my counterpart at the other organization."
When every set top box, cable modem, customer wifi gateway, and all of your routers/management interfaces need RFC1918 space it goes real quick, real fast.
It gets even worse when you are have to have a lab where you can setup infrastructure that looks/feels exactly like production and thus needs similar setups, now do that for testing/staging... and there's doubling of your required IP space already.
however now since they explictly need to enable our services we need a static ip and it is way easier to do that with nat gateway. basically we have a service that connects to http to their external ip which is firewall'd and thus if our client ip's will change it's a headache
It is just a matter of getting an inet6 prefix from your RIR/LIR, and configuring it in your network. IPv6 is not hard nor is it difficult to implement, and its nitty gritty details are actually far more logically sound then IPv4. (ARP is a major paint point for instance in large networks, so is IP broadcast traffic).
Most large WAN networks have a core which is either label switched (MPLS) and this doesn't care about anything in regards to the protocol which it is transporting, or is doing dual stack already.
My house is IPv6 first only because my ISP mandates such.
Which is easier said than done. Since my ISP won't provide me with IPv6, requesting a block from my RIR is my only option.
However, I don't meet the requirements[0] for getting a block directly from an RIR (in my case, ARIN).
In fact, unless you have a fairly large internet-facing presence, RIR's won't even give you the time of day:
Have an IPv4 assignment from ARIN or one of its
predecessors
Intend to immediately be IPv6 multi-homed
Have 13 end sites (offices, data centers, etc.)
within one year
Use 2,000 IPv6 addresses within one year
Use 200 /64 subnets within one year
Edit: Note that the requirements listed above are for end users not service providers.
you can get provider independant IP space from a LIR.
>you can get provider independant IP space from a LIR.
My understanding was that it was primarily ISPs that filled the role of LIRs. Is that incorrect?
If so, how can I find an LIR in my area?
Edit: I should have added that I've looked around, but AFAICT, the only "provider independent" group handing out IP addresses in my area is ARIN. I may be missing something obvious, but I don't think my google-fu is so bad that I can't find it, if it does indeed exist.
I wasn't blaming anyone. If that's the impression I gave, then I communicated poorly.
Just expressing my frustration WRT the inability to get an IPv6 block for myself.
It shouldn't be this hard.
And what's more, they will need to continue to buy more IPv4 addresses as they grow and their customers needs for address space grow. Amazon actually has an incentive for IPv4 to become less expensive so that further purchases of addresses are less costly to their business. And the best two ways to do that are: 1) encourage adoption to IPv6 so that IPv4 becomes less expensive and 2) encourage adoption of IPv6 so that Amazon needs to buy less IPv4 addresses.
ISPs don't want to implement it because their customers aren't pushing them to, because websites aren't typically IPv6-only, because that's a terrible business model.
So unless we force or incentivize ISPs to implement IPv6 access through law, it's going to be a long time before we hit the breaking point where something HAS to be done by ISPs
Here I'm assuming that infra teams already have a lot to do and would experience some harm having the government give them more user stories. Maybe not tons of harm, but what is the other side of the equation that justifies the harm?
It's a principal-agent problem. There are lots of people who want IPv6, but can't get it because their local ISP doesn't care.
Is it really such a great idea to force every single device on the internet to be globally uniquely identifiable?
We have run out of ipv4 addresses, CGNATs cause problems with self-hosting, ipv4 allocations are not fair and are driving up the cost of infrastructure/access. Particularly problematic in developing Countries.
Security is not an argument, ipv6 firewalls have existed for a long period now.
If we want to make the internet to scale then we need to encourage the use of ipv6.
From another angle, most people are walking around with a globally-unique identifier (IMEI) in their pocket with all kinds of sensors. Does using IPv6 materially change the situation for any attacker?
ipv6 much cheaper
if they don't then they should be ashamed
Azure's IPv6 support is terrible, and does not support IPv6 only networks: https://docs.microsoft.com/en-us/azure/virtual-network/ip-se... including limitations on the prefix size you can get for your VM's: https://docs.microsoft.com/en-us/azure/virtual-network/ip-se... no /56 like you can get in AWS that you can then slice /64's out of for subnets and no prefix delegation for running container workloads on their own GUA.
Where Azure was ahead was that its load balancers supported targeting IPv6 addresses and thus didn't require IPv4 for forwarding traffic.
AWS fixed that today.
AWS should be lauded, and while it is long overdue (IPv6 core, with IPv4 only on the edge makes so much sense) AWS is the first hyperscaler to do so.
There is zero benefit to enabling IPv6 in Azure, because they carefully prevent any use of the beneficial features of IPv6 that make it worthwhile over IPv4.
A key purpose of IPv6 is no longer needing NAT to the point that IPv6 implementation typically do not support NAT at all -- so Azure engineers developed their own custom IPv6 NAT to force NAT on all of their IPv6 users.
Another key purpose of IPv6 is the enormous address space, which means you no longer need to carefully "carve up" and manually allocate addresses in a stateful way -- Azure engineers restricted IPv6 public address prefixes to blocks of just 16 addresses. Not /16 or anything like that. No. Sixteen.
To assist migration, IPv6 is designed to be dual stack and coexist side-by-side with IPv4 with no (or minimal) impact. Azure engineers made sure that if you turn it on, wildly unrelated IPv4 features will break. In other subnets. Even other virtual networks! Critical features are masked out, making this a non-starter for practically everybody.
To enable layered systems to be migrated, IPv6 is sufficiently "compatible" with IPv4 to allow the use of relatively simple protocol-translating middle boxes. So you can have IPv6 on the outside and IPv4 on the inside, or vice-versa. Not on Azure! It's only IPv4-to-IPv4 or IPv6-to-IPv6. You can't have both, and you can't translate. PICK ONE, NOW.
It's shameful.
Seriously, if any Microsoft network engineers stumble upon this post, you should feel bad. It's 2021 when I'm writing this, mere days from 2022, and you've utterly failed to even begin to support the transition to IPv6 in the public cloud.
Shame, shame.
"Just tell me what the minimum set of tasks is to enable IPv6!"
"You don't understand, it's not like turning on a tap! We have to redo all of the infrastructure around networking!"
"Do we need all that? Just tunnel the packets or something!"
"That's not the same thing, that's only supporting IPv6 in a technical sense, it doesn't really mean the same thing as doing it properly, and it won't work long term. It'll have to be ripped out and replaced, but while maintaining backwards compatibility with a jury-rigged IPv6 implementation. It'll be a nightmare!"
"Don't worry about that now, I just need to say that we've enabled IPv6 so I can satisfy this requirement."
"It'll just tick the check box! Nobody will be able to actually use it! Our customers will be confused and support will have to deal with the fallout!"
"Let me deal with that."
Etc, etc...
Translation: I just want my KPIs so that I can get my bonus. I'm quitting before the fallout hits.
https://www.wiz.io/blog/chaosdb-explained-azures-cosmos-db-v...
That's a /124, right?
I tried a dual-stack vnet in Azure earlier this year, and eventually had to rip IPv6 out of it: flexible Postgres servers couldn't be instantiated into an IPv4-only subnet (yes, IPv4) if the vnet had IPv6 at all. (Worse, attempting to do so caused the API call to create the flexible PG instance to just time out, and it for some odd reason has a two hour timeout that suggests it's a transient internal error & you should retry. So, four hours.) I didn't really expect flexible PG to support IPv6, but that it seemed to poison the support for IPv6 for the rest of the vnet was really, really disappointing.
The reason things like Postgres break is because internally they use what is essentially a VPN Gateway to communicate with your vNet.
The end to end IPv6 support for ELBs is welcome, and I try to add IPv6 to every externally facing service, however Global Accelerators and API gateways still do not offer support.
However, nitro instances use their own network hardware which includes the ENA NICS and possibly the switches, where it seems like they may have pushed their virtual network stack into hardware, in which case it makes sense that they'd not bother implementing features separately for non-nitro.
Another random network feature nitro instances get is transparent end-to-end encryption for any traffic flowing between two nitro instances in a VPC. There used to also be Network Load Balancer features that only worked for nitro backends/clients.