Closing the IPv6 first-packet gap with GRAND
labs.ripe.net
labs.ripe.net
If I understand correctly, traditional IPv6 flow is:
* A host configures its own IP address via SLAAC
* The host sends a packet to its gateway with some destination address
* The gateway forwards the packet to the Internet
* Eventually, a response packet arrives to the gateway
* At this point, the gateway does neighbour discovery to try to figure out how to send the packet to the host
* The gateway might drop the packet or delay forwarding it until neighbour discovery completes
* A host configures its own IP address via SLAAC
* The host sends a packet to its gateway with some destination address
* The gateway forward the packet, and at the same time starts neighbour discovery because almost all computers which send outgoing packets will eventually receive some incoming packet
* When the response packet arrives, neighbour discovery is likely already done, or if not it got a good head start
In the IPv6 world, neighbor discovery is IPv6, but only sort of, because the participants don’t necessarily have real addresses. So it’s a mess.
Then there's this whole pretend "it's not broadcast but multicast" song-and-dance with ND in IPv6. In IPv4/ARP the separation is clean, and no lower protocol details leak into the IP layer.
Link-local addresses were also meant to be used for LAN-only apps. Except that it quickly turned out that you can't actually use them reliably because some interfaces (like PPP tunnels) do not _have_ MACs.
It's a mess.
IPv4 is an example of _correct_ layering. The hardware address is a detail that does not leak into upper layers. It's confined purely to the network layer.
In contrast, with IPv6 the whole 64/64 separation is a result of leaking the MAC address into upper protocols. Indeed, MAC was supposed to be a part of the publicly visible IPv6 addresses for hosts!
I know you're talking about SLAAC, but SLAAC is just a convenient way of picking a unique address. Changing the address wouldn't result in e.g. the packet being sent to a different MAC. Even sending packets to link-local addresses still does NDP, rather than parse the MAC out of the address.
Yes, it was: https://www.rfc-editor.org/info/rfc3513/#section-2.5.4
The 64/64-bit split was in fact a result of (then planned) Bluetooth having 64 bit MACs. Moreover, the initial IPv6 RFCs did not have privacy extensions for SLAAC: https://www.rfc-editor.org/info/rfc2464/#section-4
> Even sending packets to link-local addresses still does NDP, rather than parse the MAC out of the address.
It doesn't.
This kind of thing doesn't normally count as violation of layering though. In protocol design its common to leverage identifiers from lower layers for addressing. For example many workings of the internet would be hard to imaging with the rule that you could not use IP addresses and ports in upper level protocols (like DNS, P2P protocols, etc)
> In protocol design its common to leverage identifiers from lower layers for addressing.
Yes, that's why my email has the IP address of the mail server. And why my WhatsUp contains the IMEI of my phone.
...it does. You can spend five seconds in tcpdump to see that it does:
$ ping6 fe80::506c:e9ff:fe08:9ba3%eth0
11:58:48.741688 d6:57:10:b8:52:77 > 33:33:ff:08:9b:a3, ethertype IPv6 (0x86dd), length 86: fe80::d457:10ff:feb8:5277 > ff02::1:ff08:9ba3: ICMP6, neighbor solicitation, who has fe80::506c:e9ff:fe08:9ba3, length 32
11:58:48.742459 00:23:6e:5b:b8:2b > d6:57:10:b8:52:77, ethertype IPv6 (0x86dd), length 86: fe80::506c:e9ff:fe08:9ba3 > fe80::d457:10ff:feb8:5277: ICMP6, neighbor advertisement, tgt is fe80::506c:e9ff:fe08:9ba3, length 32
Yes, those bytes can be picked by looking at a MAC address, but that's only one way to pick them and the protocol doesn't treat those bytes as having any particular significance, and in particular it never assumes they contain a MAC or tries to use them as an actual MAC, so it doesn't qualify as a layering violation.
That's exactly what's happening in IPv6. The host address leaks information about the underlying hardware into higher-level protocols.
For example, you can't use a /80 site prefix because there aren't enough bits left for the hardware address mapping.
In contrast, IPv4 works just fine over Ethernet that has more MACs than the entire IPv4 address space.
> Nor was what you're discussing a requirement of IPv6, it was an optional addressing scheme.
It was a requirement initially.
> Nor does it do anything to explain why IPv4 leaking address resolution down instead of self containing it is supposed to be a correct example.
WTF is "leaking down"? The higher protocol levels are supposed to use lower protocol levels.
"Leaking down" would mean, for example, that you need to set MACs of your network cards to be equal to IPv4 addresses.
I think there is still confusion what "A network layering violation is when a protocol at one layer relies on its information being carried in protocols on other layers" means. As a practical examples:
"Reading a book has a main character 'John' in it and deciding to use that as your name in your speech" is not a layering violation for speech. At no point does anyone need to read to understand your name is John while speaking with you nor does anything break when you change your mind and decide to be called Andsynstd even though it has never been written in a book written in a book.
"You can find my name if you read that book over there" is a layering violation. They have to stop using speech with you, switch to reading the book at a completely different layer of communication, and then suddenly start calling you John in speech even though it was never communicated in speech. If they just say "what's your name" and you say "John" they don't need to get any information from outside the network layer, regardless if the bits in your response also contained your L2 address or not.
In your example, that you read your hardware as one option to come up with your address does not force anyone on the network to use a protocol other than IPv6 to learn your address and talk with you. The litmus test for this is "if you replace Ethernet with a different L2 which can't transport any protocol but L3 protocols on top of it, can you still resolve addresses?" If the answer is no then it's handled externally, if the external handling happens on L2 then it's a layering violation.
> It was a requirement initially.
Not at all. From section 2.4.1 of RFC 1884 in 1995, which introduced the concept of IPv6 addressing architecture you can continue reading past the paragraph mentioning the example of a link-local derived address to see it was never the only example:
Another unicast address format example is where a site or organization requires additional layers of internal hierarchy. In this example the subnet ID is divided into an area ID and a subnet ID. Its format is:
| s bits | n bits | m bits | 128-s-n-m bits |
+----------------------+---------+--------------+-----------------+
| subscriber prefix | area ID | subnet ID | interface ID |
+----------------------+---------+--------------+-----------------+
This technique can be continued to allow a site or organization to add additional layers of internal hierarchy. It may be desirable to use an interface ID smaller than a 48-bit IEEE 802 MAC address to allow more space for the additional layers of internal hierarchy. These could be interface IDs which are administratively created by the site or organization.
Hopefully this is already explained in the part about what a layering violation actually is, but the problem is indeed not related to IPv4 riding on top of an L2. Oblivious transport of higher layers is the point of abstracted layers. The problem is ARP, an L2 protocol, is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction. IPv6 corrected this, the neighbor exchange information is always encapsulated in an L3 packet.
"Layering violation" has a pretty clear meaning in CS. It means that a layer needs information from an upper layer for the system to work, or if a lower-level layer internal details are not abstracted properly.
For example, NATs are a layering violation because a router, which is supposed to work on the level of individual packets, needs to understand the details of sessions established in higher protocols (TCP, SIP, FTP, ...) and mangle the packets accordingly.7
The other way around is IPv6. The details of SLAAC that are driven by 64-bit MACs of the Ethernet layer. They make it impossible to use masks larger than 64 bits. The largest installed base of devices (Android) does NOT support DHCP, which is the only non-manual way to configure such addresses.
> The problem is ARP, an L2 protocol
And? What is your point? ARP is not a layering violation, it operates at the correct layer and properly abstracts it. MAC addresses are an internal detail of its functionality, they don't leak into upper layers.
Beep boop :). No, at least not last I checked. I'm just a guy who's day job was developing a NOS which targets both ASICs and a custom software-based forwarding pipelines at one of the main enterprise network vendors. Nowadays I'm PLM for it but kinda miss getting to spend years working with every single bit of these kinds of protocols.
> "Layering violation" has a pretty clear meaning in CS. It means that a layer needs information from an upper layer for the system to work
Maybe you're used to layering in areas of CS outside of networking? E.g. page 476 of TCP IP Illustrated by Fall and Stevens gives an example in the opposite direction than what you just said:
The careful reader will note that this causes a so-called layering violation. That is, the UDP protocol (transport layer) is directly processing bits “owned” by IP (network layer).
> The details of SLAAC that are driven by 64-bit MACs of the Ethernet layer.
MACs of the Ethernet layer are 48 bits.
> They make it impossible to use masks larger than 64 bits. The largest installed base of devices (Android) does NOT support DHCP, which is the only non-manual way to configure such addresses.
Android does not use the MAC address derivation mode of SLAAC, it uses randomized addresses mode of SLAAC for privacy. There are several such standardized modes for SLAAC which are not based on the link layer identifier. This should follow because Android's most common IPv6 interface is the cellular radio which does not even have an Ethernet MAC to derive from.
> And? What is your point?
The part you cut off: is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction.
This point will only make or not make sense once we agree what a layering violation in networking is, until then there's not really sense trying to debate it.
Yes. The interaction of UDP and IP _is_ a layering violation, just like NATs or even VPNs. IP and ARP are not.
> MACs of the Ethernet layer are 48 bits.
Bluetooth MACs are 64-bit.
> Android does not use the MAC address derivation mode of SLAAC, it uses randomized addresses mode of SLAAC for privacy.
Yep. So it wastes 64 bits of the address space essentially for no reason. There is literally no advantage of IPv6 ND over stateless IPv4 autoconfiguration, except that stateless IPv4 autoconfiguration doesn't waste half of the address bits.
> The part you cut off: is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction.
You probably misunderstand what "abstraction" _is_, then.
Are you saying L4<->L3 mixing is a layering violation but L3<->L2 mixing is not or is there a more detailed reason you're trying to give?
VPNs are not usually considered a layering violation in networking. They do encapsulate lower layers but they place no expectation protocols in other layers rely on the encapsulated data or vice versa. Again, the key being whether or not there is cross reliance on data between layers in the protocols, not whether or not the bits exist in the packet.
> Bluetooth MACs are 64-bit.
Sure, but Bluetooth is not Ethernet and work on any form of Ethernet or IP over Bluetooth (or even Bluetooth as an IEEE standard) was not started until several years after the IPv6 standards we're discussing were already finalized.
> Yep. So it wastes 64 bits of the address space essentially for no reason. There is literally no advantage of IPv6 ND over stateless IPv4 autoconfiguration, except that stateless IPv4 autoconfiguration doesn't waste half of the address bits.
I'd be glad to explain some of the actual reasons why we keep chasing a 64/64 split if you'd care to know. It has nothing to do with an alternate history where embedding MACs was a requirement, it has to do with other reasons which are still relevant today.
Keep in mind it's very much supported by IPv6 (and even many networks out there) to use something other than /64s. We just keep choosing to do so and use assignment methods which require so because it makes sense for other reasons more important than how densely populated the host bit portion is in a given subnet.
> You probably misunderstand what "abstraction" _is_, then.
Always a possibility :), I hope you keep the same possibility open as well.
(I am not, however, defending DHCPv4 - that has some of the same problem.)
This is the same broadcast-then-unicast process ND uses, except ND can also start as a multicast forward if MLD is supported (naturally falling back to broadcast on the switch if not).
ARP is a protocol that makes perfect sense even when spoken by hosts that only know their own MAC addresses and do not yet know their IPv4 addresses.
IPv6 ND is IPv6 except it has the weird edge case in that it is spoken between hosts that may not know their own IPv6 addresses. So you end up with delights like the “unspecified address) built into IPv6.
If I'm 2600::10 and I want to resolve who 2600::17 is, the IPv6 destination for the ND packet is set to FF02::1:FF00:17. This is a union of the multicast range with part of the destination address (so the request can almost always only go straight to the 2600::17 node rather than using a placeholder to blast to everyone). If Ethernet is in use, the L2 destination is derived and set to 33:33:FF:00:00:17 by and for the same reasoning. Different addresses will be derived e.g. for 2600::18
If I don't know my address yet (say, for DAD in this example), ARP actually uses a second made up address "0.0.0.0" for the source IP which just means unspecified. In ND, I do the same to be able to DAD my link local address by saying I'm :: (also all 0s) but at least the destination is still not ff:ff:ff:ff:ff:ff. As a bonus, since ND only uses the link local address as the source for ND, DAD for the link local address is the only time the source address can be unknown. DAD for any number of unicast addresses will always have the link local to put as the source, even if they are not in the same subnet in the L2.
There is no real difference. All Ethernet packets that have bit 7 set in the first octet are broadcast. A packet to 33:33:FF:00:00:17 will be broadcasted across the LAN.
In practice, ND will flood the network just like ARP unless switches are configured to snoop on higher-level protocols (proxy ARP/ND).
That bit is the I/G (individual/group) bit, not the broadcast bit. In switches/routers participating in multicast (IGMP/MLD or snooping of), it is used as the hardware key for the L2 multicast replication lookup. In switches/routers not participating in multicast, unique multicast groups still allow a dedicated MAC entry hardware trap to send the packet to the CPU for processing (VRRP, LLDP, STP, NDP, LACP, and more). Because ARP uses ff:ff:ff:ff:ff:ff you either need to use an ACL on the protocol type in the ingress pipeline or trap all broadcasts to the CPU (both are inefficient in their own ways). The same is true of the host NICs, regardless what the network gear is doing, who can filter all ND requests not to their address(es) by have a match on the ND multicast MACs relevant to the device be processed and then a larger deny matcher for 33:33:FF:xx:xx:xx just drop all others. Also, an ff:ff:ff:ff:ff:ff destination can never be eligible for multicast lookup (even if the switch/router is participating in multicast), so even it's still needed even if a given node might treat it similar to ff:ff:ff:ff:ff:ff.
But yes, if you ignore all of those other things and are in a network without MLD support it'll all fall back to ARP forwarding behavior with just a less generic placeholder address filling the bits. One of the great failings of IPv6 - its approach can be as inefficient as IPv4 in pathological scenarios.
Mostly because of a historic accident.
> Because ARP uses ff:ff:ff:ff:ff:ff you either need to use an ACL on the protocol type in the ingress pipeline or trap all broadcasts to the CPU (both are inefficient in their own ways).
Since you're talking about switches, they can just snoop on ARP and avoid broadcasts entirely. Some switches do that. And the last time I checked, multicast on most (all?) modern switches is also implemented by punting packets to the CPU.
This is where assumption fails, in the original formulation it was even called the multicast bit (instead of the I/G bit) and broadcast was considered a special subset of the multicast use case. Quite the opposite of how you have framed things as an accident of having so many broadcast addresses. (pdf warning) https://archive.computerhistory.org/resources/text/DEC/ether... ironically, this is
> Since you're talking about switches, they can just snoop on ARP and avoid broadcasts entirely.
ARP broadcast suppression is definitely a thing but it requires more than just snoop, you still need some form of replication of the information to the other switches in the network and you need the actual suppression+generation functionality (ARP snoop alone just lets an L2 switch build an ARP table, it doesn't define what to do with it). In the best case this is itself done via multicast, in a middle case it's thrown into BGP or similar and distributed that way (if all of your nodes are routers), and in the worst case it falls back to broadcast across the network for anything not known on a local port.
ARP broadcast suppression is also harder than with the multicast address for the reason above. Snooping also does nothing for the NICs connected to "basic" L2 switches not doing ARP broadcast suppression while the multicast MAC still does (even when not actually forwarded via multicast).
I haven't ever done any programming at this layer of the stack, so I'm purely spitballing.
Technically, there's no broadcast in IPv6, so the host is supposed to join the local multicast group and do the neighbor discovery flow to find the "on link" address. And it's not guaranteed that the network is "symmetric".
Technically, this is also true for IPv4. You can have a proxy-ARP host impersonating the sender, but since it had never been fully specced, nobody cares about this scenario.
So, to implement what you said, we need more than a simple router upgrade: we'd need to change the hardware, so that when a packet is forwarded from a source that's not in the mac table, the software can (asynchronously) perform an arp/nd lookup
There is probably a world of issue behind that behavior, but I do not know
You can either have the hardware do additional lookups for every packet it processes or you can just follow the normal process for the very first packet from that IP. Or, exactly what this article is about for the best of both worlds.
Advanced ASICs usually go down a different path of offering the ability to validate the ND process (to prevent spoofing) rather than doing even more to trust whatever is sent. This is compatible with the way GRAND works, making it a win-win-win approach.
Maybe there are some weird situations where a host that just got its own IP address starts proxying for a third node that wants return packets to asymmetrically bypass the host?
I'm guessing it's one of those completely over-engineered bits about IPv6 that is that way just because they wanted to engineer in so much complexity almost for the sake of it
"STALE allows the router to use the information it has already learned without requiring a new multicast address-resolution operation. The router can subsequently verify reachability using the normal Neighbour Discovery mechanisms."
edit: actually the RFC goes over this option as well and the reasoning there is slightly different than above (and maybe even the blog post) - see 8.9 at in https://datatracker.ietf.org/doc/rfc9131/ . Also notable that the RFC is from 2021.
1) SLAAC/static ip 2) multiple default gateways, and then you import routes(including multiple default gateway's) over bgp and start BFD to figure out which one is active 3) and then you start broadcasting your own address over bgp
i wish there were a daemon in systemd, that i could configure to do this. It seems absurd to people that aren't aware of how it works, but it's quite the interesting flow in a datacenter.
wish there were an equivalent in the kubernetes/normal linux world, but i can understand the scalability concerns :)
I know BIRD exists, but i wish this was kind of built in ... lemme see if systemd supports extensions... it would be cool to add :)
i'm sure more people might want it.
Started reading, the text is dreadfully lengthy, hard to stay concentrate on due to all the filler and mindlessly arranged words that obscure the actual content, notice AI tell signs.
Stop reading, paste into Pangram, mostly AI,
scroll down on the text, find this:
> Note: The author used AI tools to assist with drafting and language. The technical work, analysis and conclusions are the author’s own. For more information about our policy on AI-generated content, see our Contributing to RIPE Labs page.
Do better RIPE. I assumed RIPE had some standard. If you decide to not have any standards, at the very least put the AI disclaimer at the top so I don't waste my time on slop. It's disrespectful.
Does anyone have an article about NDP and GRAND that is not slop and worth sharing and reading?