Hacker News Reader

Comment by mort96 | Hacker News Reader

Wait I don't get this.

If I understand correctly, traditional IPv6 flow is:

    * A host configures its own IP address via SLAAC
    * The host sends a packet to its gateway with some destination address
    * The gateway forwards the packet to the Internet
    * Eventually, a response packet arrives to the gateway
    * At this point, the gateway does neighbour discovery to try to figure out how to send the packet to the host
    * The gateway might drop the packet or delay forwarding it until neighbour discovery completes

Why couldn't we change the flow to:

    * A host configures its own IP address via SLAAC
    * The host sends a packet to its gateway with some destination address
    * The gateway forward the packet, and at the same time starts neighbour discovery because almost all computers which send outgoing packets will eventually receive some incoming packet
    * When the response packet arrives, neighbour discovery is likely already done, or if not it got a good head start

Isn't this the obvious solution which wouldn't require changes to hosts or new protocols, just a small tweak to the router? Usually, when there's a seemingly obvious simple solution to a real problem and that solution hasn't been implemented by any of the clever people working in networking standards, there's a good reason and the solution isn't as simple as it seems. So what am I missing?

View on HN

Im mildly confused as well, there is an even more immediate shortcut that I've certainly implemented before. in arp its not unusual to to just create a ip->mac binding from the source information in the ethernet header. where this potentially breaks down if we start looking at issues of trust. but its already the case in ND that we trust the endpoint to have executed the state machine to search for duplicates. so what's preventing us from doing the same thing here? maybe just layering concerns?

I’ve always thought that IPv6 has dramatically worse layering than IPv4. In IPv4 over Ethernet, there’s ARP, which layers over plain Ethernet, and IPv4 sits on top of the combination of ARP+Ethernet.

In the IPv6 world, neighbor discovery is IPv6, but only sort of, because the participants don’t necessarily have real addresses. So it’s a mess.

IPv6 link local layers over Ethernet the same way Arp does. Both contain a source/dest MAC which is used for forwarding, both contain the relevant neighbor info. If anything, keeping the protocol's self-discovery messages wrapped in the protocol itself is actually cleaner layering at the cost of complexity (the extra link local signalling addresses).

It really is not. There's a whole morass with possibly overlapping "on link" networks that nobody can implement correctly on the first try.

Then there's this whole pretend "it's not broadcast but multicast" song-and-dance with ND in IPv6. In IPv4/ARP the separation is clean, and no lower protocol details leak into the IP layer.

Link-local addresses were also meant to be used for LAN-only apps. Except that it quickly turned out that you can't actually use them reliably because some interfaces (like PPP tunnels) do not _have_ MACs.

It's a mess.

Morass, mess, broadcast/multicast, etc aside (seems more like complaints of complexity than layering), IPv4+ARP is the textbook example of a layering violation. When you do want to violate, having the L2 info in the L3 packet is still cleaner than L3 info in L2. One is a protocol carrying its own glue in itself, the other is a protocol using different protocols (per L2) to discover the glue the same way it could have itself anyways. It's certainly convenient of course, but that doesn't make it cleaner layering. It also gives a consistent answer for different L2s e.g. cellular links because of this.

Whut?

IPv4 is an example of _correct_ layering. The hardware address is a detail that does not leak into upper layers. It's confined purely to the network layer.

In contrast, with IPv6 the whole 64/64 separation is a result of leaking the MAC address into upper protocols. Indeed, MAC was supposed to be a part of the publicly visible IPv6 addresses for hosts!

Huh? No, the MAC was never a part of the publicly-visible v6 address.

I know you're talking about SLAAC, but SLAAC is just a convenient way of picking a unique address. Changing the address wouldn't result in e.g. the packet being sent to a different MAC. Even sending packets to link-local addresses still does NDP, rather than parse the MAC out of the address.

> Huh? No, the MAC was never a part of the publicly-visible v6 address.

Yes, it was: https://www.rfc-editor.org/info/rfc3513/#section-2.5.4

The 64/64-bit split was in fact a result of (then planned) Bluetooth having 64 bit MACs. Moreover, the initial IPv6 RFCs did not have privacy extensions for SLAAC: https://www.rfc-editor.org/info/rfc2464/#section-4

> Even sending packets to link-local addresses still does NDP, rather than parse the MAC out of the address.

It doesn't.

Broadly it's true that historically there was this idea for ethernet networks at least. It was always optional though. Even in that long obsolete rfc2464 it's described as the way to do SLAAC which was optional even in 1998.

This kind of thing doesn't normally count as violation of layering though. In protocol design its common to leverage identifiers from lower layers for addressing. For example many workings of the internet would be hard to imaging with the rule that you could not use IP addresses and ports in upper level protocols (like DNS, P2P protocols, etc)

The early RFCs were written more informally, so it's hard to say what was optional. However, the consensus was that SLAAC was supposed to be the main way to configure IPv6, along with fully manual configuration.

> In protocol design its common to leverage identifiers from lower layers for addressing.

Yes, that's why my email has the IP address of the mail server. And why my WhatsUp contains the IMEI of my phone.

> It doesn't.

...it does. You can spend five seconds in tcpdump to see that it does:

    $ ping6 fe80::506c:e9ff:fe08:9ba3%eth0

    11:58:48.741688 d6:57:10:b8:52:77 > 33:33:ff:08:9b:a3, ethertype IPv6 (0x86dd), length 86: fe80::d457:10ff:feb8:5277 > ff02::1:ff08:9ba3: ICMP6, neighbor solicitation, who has fe80::506c:e9ff:fe08:9ba3, length 32
    11:58:48.742459 00:23:6e:5b:b8:2b > d6:57:10:b8:52:77, ethertype IPv6 (0x86dd), length 86: fe80::506c:e9ff:fe08:9ba3 > fe80::d457:10ff:feb8:5277: ICMP6, neighbor advertisement, tgt is fe80::506c:e9ff:fe08:9ba3, length 32

Notice how a) it's doing NDP, and b) the link-local is fe80::506c:e9ff:fe08:9ba3 while the MAC is 00:23:6e:5b:b8:2b? The "506c:e9ff:fe08:9ba3" part of the address isn't being treated as a MAC address by the protocol -- it's just some opaque bytes.

Yes, those bytes can be picked by looking at a MAC address, but that's only one way to pick them and the protocol doesn't treat those bytes as having any particular significance, and in particular it never assumes they contain a MAC or tries to use them as an actual MAC, so it doesn't qualify as a layering violation.

A network layering violation is when a protocol at one layer relies on its information being carried in protocols on other layers. It's not just when the addressing bits happen to match between layers, which would be done by the host locally without a separate L2 protocol anyways. Nor was what you're discussing a requirement of IPv6, it was an optional addressing scheme. Nor did it take on as a popular option. Nor does it do anything to explain why IPv4 leaking address resolution down instead of self containing it is supposed to be a correct example.

> A network layering violation is when a protocol at one layer relies on its information being carried in protocols on other layers.

That's exactly what's happening in IPv6. The host address leaks information about the underlying hardware into higher-level protocols.

For example, you can't use a /80 site prefix because there aren't enough bits left for the hardware address mapping.

In contrast, IPv4 works just fine over Ethernet that has more MACs than the entire IPv4 address space.

> Nor was what you're discussing a requirement of IPv6, it was an optional addressing scheme.

It was a requirement initially.

> Nor does it do anything to explain why IPv4 leaking address resolution down instead of self containing it is supposed to be a correct example.

WTF is "leaking down"? The higher protocol levels are supposed to use lower protocol levels.

"Leaking down" would mean, for example, that you need to set MACs of your network cards to be equal to IPv4 addresses.

> That's exactly what's happening in IPv6. The host address leaks information about the underlying hardware into higher-level protocols

I think there is still confusion what "A network layering violation is when a protocol at one layer relies on its information being carried in protocols on other layers" means. As a practical examples:

"Reading a book has a main character 'John' in it and deciding to use that as your name in your speech" is not a layering violation for speech. At no point does anyone need to read to understand your name is John while speaking with you nor does anything break when you change your mind and decide to be called Andsynstd even though it has never been written in a book written in a book.

"You can find my name if you read that book over there" is a layering violation. They have to stop using speech with you, switch to reading the book at a completely different layer of communication, and then suddenly start calling you John in speech even though it was never communicated in speech. If they just say "what's your name" and you say "John" they don't need to get any information from outside the network layer, regardless if the bits in your response also contained your L2 address or not.

In your example, that you read your hardware as one option to come up with your address does not force anyone on the network to use a protocol other than IPv6 to learn your address and talk with you. The litmus test for this is "if you replace Ethernet with a different L2 which can't transport any protocol but L3 protocols on top of it, can you still resolve addresses?" If the answer is no then it's handled externally, if the external handling happens on L2 then it's a layering violation.

> It was a requirement initially.

Not at all. From section 2.4.1 of RFC 1884 in 1995, which introduced the concept of IPv6 addressing architecture you can continue reading past the paragraph mentioning the example of a link-local derived address to see it was never the only example:

  Another unicast address format example is where a site or organization requires additional layers of internal hierarchy. In this   example the subnet ID is divided into an area ID and a subnet ID. Its format is:
  
  |         s bits       | n bits  |   m bits     | 128-s-n-m bits  |
  +----------------------+---------+--------------+-----------------+
  |   subscriber prefix  | area ID |  subnet ID   |  interface ID   |
  +----------------------+---------+--------------+-----------------+
  
  This technique can be continued to allow a site or organization to add additional layers of internal hierarchy. It may be   desirable to use an interface ID smaller than a 48-bit IEEE 802 MAC address to allow more space for the additional layers of   internal hierarchy. These could be interface IDs which are administratively created by the site or organization.

> WTF is "leaking down"? The higher protocol levels are supposed to use lower protocol levels.

Hopefully this is already explained in the part about what a layering violation actually is, but the problem is indeed not related to IPv4 riding on top of an L2. Oblivious transport of higher layers is the point of abstracted layers. The problem is ARP, an L2 protocol, is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction. IPv6 corrected this, the neighbor exchange information is always encapsulated in an L3 packet.

Are you a bot?

"Layering violation" has a pretty clear meaning in CS. It means that a layer needs information from an upper layer for the system to work, or if a lower-level layer internal details are not abstracted properly.

For example, NATs are a layering violation because a router, which is supposed to work on the level of individual packets, needs to understand the details of sessions established in higher protocols (TCP, SIP, FTP, ...) and mangle the packets accordingly.7

The other way around is IPv6. The details of SLAAC that are driven by 64-bit MACs of the Ethernet layer. They make it impossible to use masks larger than 64 bits. The largest installed base of devices (Android) does NOT support DHCP, which is the only non-manual way to configure such addresses.

> The problem is ARP, an L2 protocol

And? What is your point? ARP is not a layering violation, it operates at the correct layer and properly abstracts it. MAC addresses are an internal detail of its functionality, they don't leak into upper layers.

> Are you a bot?

Beep boop :). No, at least not last I checked. I'm just a guy who's day job was developing a NOS which targets both ASICs and a custom software-based forwarding pipelines at one of the main enterprise network vendors. Nowadays I'm PLM for it but kinda miss getting to spend years working with every single bit of these kinds of protocols.

> "Layering violation" has a pretty clear meaning in CS. It means that a layer needs information from an upper layer for the system to work

Maybe you're used to layering in areas of CS outside of networking? E.g. page 476 of TCP IP Illustrated by Fall and Stevens gives an example in the opposite direction than what you just said:

  The careful reader will note that this causes a so-called layering violation. That is, the UDP protocol (transport layer) is directly processing bits “owned” by IP (network layer).

That said, you're correct NAT is still also another network layering violation driven by IPv4's limitations.

> The details of SLAAC that are driven by 64-bit MACs of the Ethernet layer.

MACs of the Ethernet layer are 48 bits.

> They make it impossible to use masks larger than 64 bits. The largest installed base of devices (Android) does NOT support DHCP, which is the only non-manual way to configure such addresses.

Android does not use the MAC address derivation mode of SLAAC, it uses randomized addresses mode of SLAAC for privacy. There are several such standardized modes for SLAAC which are not based on the link layer identifier. This should follow because Android's most common IPv6 interface is the cellular radio which does not even have an Ethernet MAC to derive from.

> And? What is your point?

The part you cut off: is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction.

This point will only make or not make sense once we agree what a layering violation in networking is, until then there's not really sense trying to debate it.

> Maybe you're used to layering in areas of CS outside of networking? E.g. page 476 of TCP IP Illustrated by Fall and Stevens gives an example in the opposite direction than what you just said

Yes. The interaction of UDP and IP _is_ a layering violation, just like NATs or even VPNs. IP and ARP are not.

> MACs of the Ethernet layer are 48 bits.

Bluetooth MACs are 64-bit.

> Android does not use the MAC address derivation mode of SLAAC, it uses randomized addresses mode of SLAAC for privacy.

Yep. So it wastes 64 bits of the address space essentially for no reason. There is literally no advantage of IPv6 ND over stateless IPv4 autoconfiguration, except that stateless IPv4 autoconfiguration doesn't waste half of the address bits.

> The part you cut off: is not oblivious to the information of the layers above it, such as L3 IP information, breaking the abstraction.

You probably misunderstand what "abstraction" _is_, then.

> Yes. The interaction of UDP and IP _is_ a layering violation, just like NATs or even VPNs. IP and ARP are not.

Are you saying L4<->L3 mixing is a layering violation but L3<->L2 mixing is not or is there a more detailed reason you're trying to give?

VPNs are not usually considered a layering violation in networking. They do encapsulate lower layers but they place no expectation protocols in other layers rely on the encapsulated data or vice versa. Again, the key being whether or not there is cross reliance on data between layers in the protocols, not whether or not the bits exist in the packet.

> Bluetooth MACs are 64-bit.

Sure, but Bluetooth is not Ethernet and work on any form of Ethernet or IP over Bluetooth (or even Bluetooth as an IEEE standard) was not started until several years after the IPv6 standards we're discussing were already finalized.

> Yep. So it wastes 64 bits of the address space essentially for no reason. There is literally no advantage of IPv6 ND over stateless IPv4 autoconfiguration, except that stateless IPv4 autoconfiguration doesn't waste half of the address bits.

I'd be glad to explain some of the actual reasons why we keep chasing a 64/64 split if you'd care to know. It has nothing to do with an alternate history where embedding MACs was a requirement, it has to do with other reasons which are still relevant today.

Keep in mind it's very much supported by IPv6 (and even many networks out there) to use something other than /64s. We just keep choosing to do so and use assignment methods which require so because it makes sense for other reasons more important than how densely populated the host bit portion is in a given subnet.

> You probably misunderstand what "abstraction" _is_, then.

Always a possibility :), I hope you keep the same possibility open as well.

Sticking L2 into L3 means that L3 needs the ability to communicate with nodes with as-yet-unknown L2 addresses and that L3 nodes that don’t have an L3 address yet need to be able to transmit L3 packets. Both of these are quite messy, and APR completely avoids these problems.

(I am not, however, defending DHCPv4 - that has some of the same problem.)

ARP does not avoid this problem at all, it broadcasts until enough L2 information is exchanged to unicast (which usually happens to also be the point the L3 information is resolved).

This is the same broadcast-then-unicast process ND uses, except ND can also start as a multicast forward if MLD is supported (naturally falling back to broadcast on the switch if not).

You’re misunderstanding my point.

ARP is a protocol that makes perfect sense even when spoken by hosts that only know their own MAC addresses and do not yet know their IPv4 addresses.

IPv6 ND is IPv6 except it has the weird edge case in that it is spoken between hosts that may not know their own IPv6 addresses. So you end up with delights like the “unspecified address) built into IPv6.

If I'm 192.168.129.10 and I want to resolve who 192.168.129.17 is, I make an ARP with the destination as ff:ff:ff:ff:ff:ff. This is a placeholder L2 destination which just means "everyone". I likely need to do something completely different when not on Ethernet (which is surprisingly common when you get beyond PCs on a LAN) and that may or may not involve ARP but we'll stick with ARP on Ethernet for now.

If I'm 2600::10 and I want to resolve who 2600::17 is, the IPv6 destination for the ND packet is set to FF02::1:FF00:17. This is a union of the multicast range with part of the destination address (so the request can almost always only go straight to the 2600::17 node rather than using a placeholder to blast to everyone). If Ethernet is in use, the L2 destination is derived and set to 33:33:FF:00:00:17 by and for the same reasoning. Different addresses will be derived e.g. for 2600::18

If I don't know my address yet (say, for DAD in this example), ARP actually uses a second made up address "0.0.0.0" for the source IP which just means unspecified. In ND, I do the same to be able to DAD my link local address by saying I'm :: (also all 0s) but at least the destination is still not ff:ff:ff:ff:ff:ff. As a bonus, since ND only uses the link local address as the source for ND, DAD for the link local address is the only time the source address can be unknown. DAD for any number of unicast addresses will always have the link local to put as the source, even if they are not in the same subnet in the L2.

> In ND, I do the same to be able to DAD my link local address by saying I'm :: (also all 0s) but at least the destination is still not ff:ff:ff:ff:ff:ff

There is no real difference. All Ethernet packets that have bit 7 set in the first octet are broadcast. A packet to 33:33:FF:00:00:17 will be broadcasted across the LAN.

In practice, ND will flood the network just like ARP unless switches are configured to snoop on higher-level protocols (proxy ARP/ND).

Have you ever asked yourself why there would be 140,737,488,355,328 (half of all) MAC addresses reserved for broadcast if it had no utilities over setting ff:ff:ff:ff:ff:ff? You're correct about fallback replication behavior matching that of broadcast (though that has to do with participating in or snooping IGMP/MLD rather than proxy ARP/ND), I'm just not sure you are considering any implications beyond a single aspect of that one scenario in the above.

That bit is the I/G (individual/group) bit, not the broadcast bit. In switches/routers participating in multicast (IGMP/MLD or snooping of), it is used as the hardware key for the L2 multicast replication lookup. In switches/routers not participating in multicast, unique multicast groups still allow a dedicated MAC entry hardware trap to send the packet to the CPU for processing (VRRP, LLDP, STP, NDP, LACP, and more). Because ARP uses ff:ff:ff:ff:ff:ff you either need to use an ACL on the protocol type in the ingress pipeline or trap all broadcasts to the CPU (both are inefficient in their own ways). The same is true of the host NICs, regardless what the network gear is doing, who can filter all ND requests not to their address(es) by have a match on the ND multicast MACs relevant to the device be processed and then a larger deny matcher for 33:33:FF:xx:xx:xx just drop all others. Also, an ff:ff:ff:ff:ff:ff destination can never be eligible for multicast lookup (even if the switch/router is participating in multicast), so even it's still needed even if a given node might treat it similar to ff:ff:ff:ff:ff:ff.

But yes, if you ignore all of those other things and are in a network without MLD support it'll all fall back to ARP forwarding behavior with just a less generic placeholder address filling the bits. One of the great failings of IPv6 - its approach can be as inefficient as IPv4 in pathological scenarios.

> Have you ever asked yourself why there would be 140,737,488,355,328 (half of all) MAC addresses reserved for broadcast if it had no utilities over setting ff:ff:ff:ff:ff:ff?

Mostly because of a historic accident.

> Because ARP uses ff:ff:ff:ff:ff:ff you either need to use an ACL on the protocol type in the ingress pipeline or trap all broadcasts to the CPU (both are inefficient in their own ways).

Since you're talking about switches, they can just snoop on ARP and avoid broadcasts entirely. Some switches do that. And the last time I checked, multicast on most (all?) modern switches is also implemented by punting packets to the CPU.

> Mostly because of a historic accident.

This is where assumption fails, in the original formulation it was even called the multicast bit (instead of the I/G bit) and broadcast was considered a special subset of the multicast use case. Quite the opposite of how you have framed things as an accident of having so many broadcast addresses. (pdf warning) https://archive.computerhistory.org/resources/text/DEC/ether... ironically, this is

> Since you're talking about switches, they can just snoop on ARP and avoid broadcasts entirely.

ARP broadcast suppression is definitely a thing but it requires more than just snoop, you still need some form of replication of the information to the other switches in the network and you need the actual suppression+generation functionality (ARP snoop alone just lets an L2 switch build an ARP table, it doesn't define what to do with it). In the best case this is itself done via multicast, in a middle case it's thrown into BGP or similar and distributed that way (if all of your nodes are routers), and in the worst case it falls back to broadcast across the network for anything not known on a local port.

ARP broadcast suppression is also harder than with the multicast address for the reason above. Snooping also does nothing for the NICs connected to "basic" L2 switches not doing ARP broadcast suppression while the multicast MAC still does (even when not actually forwarded via multicast).

This was my immediate thought as well. Though I have not thought through any of the details, it did occur to me that the information needed would be in the "source" section of the header. I wonder if it is too much work to validate it somehow before using it?

I haven't ever done any programming at this layer of the stack, so I'm purely spitballing.

Ah, one of the recursive WTFs in the IPv6 design.

Technically, there's no broadcast in IPv6, so the host is supposed to join the local multicast group and do the neighbor discovery flow to find the "on link" address. And it's not guaranteed that the network is "symmetric".

Technically, this is also true for IPv4. You can have a proxy-ARP host impersonating the sender, but since it had never been fully specced, nobody cares about this scenario.

Even weirder: when the router forwards the packet from the host to the Internet, it already sees both the source IPv6 and MAC address, so it could store them.

Maybe there are some weird situations where a host that just got its own IP address starts proxying for a third node that wants return packets to asymmetrically bypass the host?

This is what I don't understand. I'll be the first to stand up and say there's a lot about IPv6 I don't know, but why can't/doesn't the router learn how to talk to the host when the host sends that outbound packet?

I'm guessing it's one of those completely over-engineered bits about IPv6 that is that way just because they wanted to engineer in so much complexity almost for the sake of it

Generally IP does not make that assumption. If I remember right (it's too early), having to go through neighbor discovery combined with a switch doing some special processing on ARP/ND packets protects against identity hijacking in the LAN.

Wild guess: packet forward is implemented in hardware while arp/nd is software, with probably some things (think "hardware interrupt" or something alike) that allows hardware to "call" the software stack (for instance, when the link-layer addr is unknown)

So, to implement what you said, we need more than a simple router upgrade: we'd need to change the hardware, so that when a packet is forwarded from a source that's not in the mac table, the software can (asynchronously) perform an arp/nd lookup

There is probably a world of issue behind that behavior, but I do not know

Great guess, not sure why it's at the bottom of the responses so far :).

You can either have the hardware do additional lookups for every packet it processes or you can just follow the normal process for the very first packet from that IP. Or, exactly what this article is about for the best of both worlds.

Advanced ASICs usually go down a different path of offering the ability to validate the ND process (to prevent spoofing) rather than doing even more to trust whatever is sent. This is compatible with the way GRAND works, making it a win-win-win approach.

Apparently it's not good to require the router to start a new multicast address resolution on seeing unknown sources. It could be spelled out more but the text "This is particularly relevant for anycast and proxy addresses, where more than one node may be capable of responding" says that in some circumstances a lot of link local addresses might correspond to a single global address, and there could be a lot of traffic generated. So it's better to have the host opt-in to the prepopulating of the STALE entries.

"STALE allows the router to use the information it has already learned without requiring a new multicast address-resolution operation. The router can subsequently verify reachability using the normal Neighbour Discovery mechanisms."

edit: actually the RFC goes over this option as well and the reasoning there is slightly different than above (and maybe even the blog post) - see 8.9 at in https://datatracker.ietf.org/doc/rfc9131/ . Also notable that the RFC is from 2021.

My assumption is this isn't the way it works because it would be doing work up front, when it's not clear that the return will be necessary at all (think UDP). The response could be some time in the distant future or never, keeping the mapping in memory could be a problem (IPv6 design is 30 years old... and fast memory was even more expensive back then).

Reply on news.ycombinator.com