Hacker News Reader

Comment by em3rgent0rdr | Hacker News Reader

Cerf: "I'm serious, the decision to put a 32-bit address space on there was the result of a year's battle among a bunch of engineers who couldn't make up their minds about 32, 128 or variable length. And after a year of fighting I said -- I'm now at ARPA, I'm running the program, I'm paying for this stuff and using American tax dollars -- and I wanted some progress because we didn't know if this is going to work. So I said 32 bits, it is enough for an experiment, it is 4.3 billion terminations -- even the defense department doesn't need 4.3 billion of anything and it couldn't afford to buy 4.3 billion edge devices to do a test anyway. So at the time I thought we were doing a experiment to prove the technology and that if it worked we'd have an opportunity to do a production version of it. Well -- [laughter] -- it just escaped! -- it got out and people started to use it and then it became a commercial thing. So, this [IPv6] is the production attempt at making the network scalable. Only 30 years later."

[1] https://youtu.be/mZo69JQoLb8?t=815 "Google IPv6 Conference 2008: What will the IPv6 Internet look like?"

View on HN

This is a good lesson. I've done a lot of pilot studies, prototypes, proof-of-concept experiments. It is scary^1 how many times "let's just get this tested and see if it even works" turns into "welp, that worked and we don't have time or funding to do another one, so that's the final version." The constraint is often time. Gotta get a product out, or run an follow-on experiment that depended on the first, or submit a paper or funding application, or whatever.

Test the thing you want to test!

1. Not scary like fear, just like, I don't know, astonishing.

An old Russian proverb goes: There is nothing more permanent than a temporary solution.

Nice. Added to https://github.com/globalcitizen/taoup

oooh, it's on nix! done and done

Yeah, I've heard the same thing in several contexts before too. Without any mention of nationality. :)

In Italy we have "temporarily permanent" laws. I wish I were kidding.

In hindsight. ;)

To me this sounds like the winner's bias: in hindsight.

It is hard to balance a prototype against a "works under all conditions no matter what they are" production-ready solution without growing into a hard to handle monster solution. On the other side I know off quite some overhyped apps, that rest on a overpowered machine, because the specs did not account for one thing: no one cares about the product.

Extensibility is the key to balance certain weaknesses in a spec.

:-P ChatGPT?

On the flip side, how many times do we hear about projects that failed because the dev team was stuck building things that were never needed or never used?

It's hard to see the future.

That's it. The causality analysis in the grandparent is exactly backwards. We don't live in a world of "productized prototypes" because people are too lazy to do things right. We live in the world of prototypes because prototypes are the only products that reach market. Even where beautiful works of engineering exist, then tend to be beaten to the punch by the competing prototype anyway.

You don't treat this by whining that the prototypes are winning. You treat it by finding ways to cleanly evolve prototypes[1], and by collecting intuition (c.f. the linked article) about common mistakes and how to avoid them.

[1] Once upon a time in an age lost to history, this was the idea behind something called "agile". The thing we call "agile" today is pretty much the opposite.

Right. In that vein, it wasn't the 32 bit limitation that was the real problem. It was the inescapable 32 bit limitation built into the design that was the problem. They could have spent a few bits to version the packet format, for example, which would have at least provided an escape hatch. Or heck, a single bit: Value of 0, known original format. Value of 1, unknown format (or known future format, to be parsed by some future IP packet decoder, but which errors out old decoders).

And they repeated the same mistake with the IPv6 design by not making it an evolution of or backwards-compatible with IPv4.

Literally neither of these was built with scalability in mind.

The first 4 bits of both IPv4 and IPv6 packets are a version field.

> The first 4 bits of both IPv4 and IPv6 packets are a version field.

And in case it wasn't clear enough: it's called IPv4 because the value of that version field is 4, and it's called IPv6 because the value of that version field is 6.

TLS has version fields everywhere, so TLS 1.3 identifies as 1.2 and hides truth deep in extension fields, because anything else broke traffic analysis tools.

You can also design prototypes so that they're impossible to operationalize beyond experimental scale.

I forget the name of it (there are probably several by now), but there's a UX prototyping toolkit where all the components look hand-drawn, so that nobody expects to be able to interact with them and then gets mad when nothing happens. It also prevents you-the-designer from being tempted to add interactivity logic at design time.

There's nothing stopping someone from creating a programming language that works the same way — something intentionally constrained to e.g. run on an abstract machine that uses a 24-bit address space (and builds this into the design, using the other 40 bits of each pointer on 64-bit VM implementations for other important info) so that only prototype use-cases can be implemented and tested, while the system would inherently be unable to reach any kind of scale serving multiple concurrent customers.

While I don't think anyone's ever tried creating a runtime like this for general-purpose software prototyping, I know of at least one domain-specific example:

The PICO-8 (https://www.lexaloffle.com/pico-8.php) is a game runtime that's kind of like this, designed to put constraints on game development that resemble, but are orthogonal to, the kind of constraints imposed by old consoles like the Gameboy. (For example, it is programmed in high-level Lua, but has an 8192-lexeme source-code size limit, to achieve similar complexity constraints to old games that needed to fit their object code on 16k ROMs, while not forcing you to compile to/write in an assembly language, nor impacting source-code readability.)

The prototyping toolkit you're thinking of is probably Balsamiq Mockups, which explicitly emulates making wireframes on paper. Other wireframing tools adopted the technique as well.

I've done something like this when bootstrapping a system that would eventually have an API with proper admin tools, but we just need something quick-and-dirty to get off the ground.

So I wrote some scripts in bash to do things the "worst" way possible. Directly accessing the production DB, minimal safety checks, etc. Nobody else on the team liked writing bash, so there was no fight to replace these awful tools with proper APIs.

But that's missing the point again! If you do that, you will never reach the release with your perfect app because you wasted that time and effort on making an unusable mockup and there might be no money left to do the actual app. Those who make a mockup that can actually be used release it and that becomes the business.

Note this part of my statement: "It also prevents you-the-designer from being tempted to add interactivity logic at design time."

In practice, in many industries, prototyping tools are used as the first step in the design process. The constraints built into these prototyping tools force those using them to spend less — often orders of magnitude less — time, and effort, and money(!) prototyping, than they would if they allowed themselves to prototype with production-oriented tooling.

Consider painting. A painter — unless they're working off of a photographic reference — will almost always draw a pencil sketch of the scene they want to capture in paint, before they begin the actual painting. A pencil sketch has no need to consider color (or how to mix to achieve particular color effects), or light and shadow, or dimensionality (how real light reflects off of built-up paint on the canvas), or any of that. They just need to concentrate on proportion, perspective, anatomy, etc. The sketch focuses only on (a subset of) the design of the painting, while inhibiting any of the implementation details specific to the medium of paint, from being worked on. Which means the sketch only takes a few minutes, rather than days.

Once they have this sketch, they can show the sketch to the client who commissioned the painting (or to the master of the studio, if you're painting for gallery sale), and the client/"product owner" can point out places where the sketch does not align to their vision for the painting, which can be used to iterate on the design.

There are many other things to get right once the "real" painting starts, but if you don't get the "bones" of the thing right, the client won't want the painting. The sketch lets you evaluate just the "bones" of the painting before even considering the meat on those bones.

A prototyping tool for programming should be the same: something to let you consider the "bones" of business logic, preconditions/postconditions, etc., without the "meat" of the particular library APIs and data-structure juggling required to glue things together and achieve scalability in a particular language.

The best prototyping tools are ones that pare down your focus to the smallest useful kernel of design, and thereby allow you to iterate in near-realtime. An experienced painter will become skilled enough at making sketches, that they can sit down with a client and sketch in response to the client's words, changing the sketch "interactively" in response to the client.

In fact, some prototyping tools are streamlined enough to allow the client themselves to iterate and ideate on the sketch by themselves; and then only submit it to the productization process once they're happy with it!

Game-development example again: RPG Maker. As far as I can tell, RPG Maker as a software product was never really expected by its vendor to be used in the production of commercial games (although it has been repeatedly marketed that way.) Until very recent releases in the series, it was far too constrained for that — unless you entirely eschewed most of its engine [as most of the RPG Maker "walking simulators" like Yume Nikki do], it gives you a very static set of engines: battles that work exactly one way, menus that work exactly one way, etc. A tool that's actually for building role-playing games as commercial products, would have an almost-monomaniacal focus on letting you customize these systems to make your game distinctive; but RPG Maker is entirely the opposite. Rather, I believe that in its idiomatic use, RPG Maker has always been intended as a tool to allow a client to "sketch out" the narrative(!) "bones" of an RPG. That's why it gives you so many built-in assets to work with, but also why these assets are so generic, and also why ASCII/Enterbrain never created an "asset store", nor documented the asset formats: the assets are meant to act essentially as wireframe components. You're not meant to re-skin an RPG Maker game; you're meant to just use the generic assets to build a generic-looking game, because the look of the game isn't the point, any more than the look of a pencil sketch is the point.

If by doing things the way I need them to work now rather than the future lets me achieve the success, longevity, and ubiquity of TCP, I think I'm, fine with waiting until later to figure out things that don't matter right away.

That's always the bind.

- If it fails: welp, we were right to not spend months on this discussion.

- If it wildly succeeds: welp, it's now too costly to change for so many users.

Having a mild and steady usage growing curve is a blessing that's too often overlooked.

It's always frustrating and amusing to read an anecdote about a design flaw in something where the creator goes "Yeah, we discovered that X was a bad idea, but there were already almost 10 people using it so it was too late to change."

Tabs in makefiles is another classic.

Except tabs in makefiles only require ones to acknowledge 'makefiles use tabs and here are the rules'. It's otherwise only something for pedants to obsess over.

IPV4 address contention has a bit more real-world impact.

No one and I mean NO ONE had any clue how widely deployed IPv4 would end up being. It went literally beyond everyone's wildest expectations who was alive at the time of its inception.

I don't think anything I wrote disagrees. As someone who started implementing IP networks just a couple of years after RFC791 was published, I'd say that was completely correct. But more than 40 years later, the fact that it has gone beyond everyone's wildest expectations means some of those early design decisions are starting to bite. Unlike tabs in makefiles, which is just something certain types of people like to complain about.

> Unlike tabs in makefiles, which is just something certain types of people like to complain about.

http://catb.org/~esr/writings/taoup/html/ch15s04.html

Which "certain type" are Stuart Feldman and Steve Johnson?

Knowing this and being the person (or part of the team) to first implement a thing is the true thrill of programming. You, sometimes exclusively you, understand the weight and permanence of the minor decisions you make. This is why non-programmers often find programmers hard to understand, overly picky about minor details, strangely emotional about things they have a hard time verbalizing and so on.

Maybe there should be a margin factor, just like in architecture. Build it for 5x the load, then 10x that.

Well ARPA already had a margin factror...they had about 256 computers that needed addresses, and left space for 2^24 times 256, or 4,294,967,296 computers.

The growth ended up to be exponential over time. So maybe a better margin would be expressed exponentially along the lines of "build it for 2^(doubling_constant * number_of_years) the load."

fair enough but cerf seemed to hint at the fact that 32bit was their lower estimate, I meant to multiply that number.

For experiments it is often a good idea to purposefully limit them so that if they do escape they can't become too entrenched.

In this case probably the right number of address bits would have been 16. That would be big enough to to build test networks (real or simulated) with more hosts than were currently on ARPANET, but small enough that it would not take too long for ARPANET growth to hit the limit.

With 32 bits the running out of addresses problem is far enough in the future that someone choosing to go ahead and put the experiment in production is not making a problem for themselves. They are making a problem for whoever has their job long after they are retired.

With 16 bits the running out of addresses problem is soon enough that they can see that it might be something they might have to deal with.

Klingon software is not released. Klingon software escapes, leaving a bloody trail of design engineers and quality assurance testers in its wake.

Never have gotten a good answer to this, “why is 64 bits not enough?” Here, why not an option?

It’s astronomically larger than 32, not double.

Heck, just adding another byte to it to make it 2^40 would have given us 1,099,511,627,776 or 1 trillion+ possible addresses.

My answer to that is that there's no way to know in advance if 64 (or any other #) of bit is enough or not. Tech history is full of examples of arbitrary limits being defined thinking that there's no possible way anyone would need to exceed them, only to find that everyone needs to exceed them later on.

Is 64 bits enough? Maybe. Maybe not. But if you're wanting to future-proof, then setting an arbitrary limit is not the way to go.

I'd agree, but the historical computing restrictions were usually hitting ceilings over 8, 12, or 16 bits when those things were very expensive.

32bits is just about large enough (within an order of magnitude sense) if the space was used more efficiently.

128bits I've heard described is like "every atom in the universe" big. If so, then 64 is probably enough for every atom on Earth.

Now I've just thought of another angle, similar to UUIDs. They are used because they can be assigned randomly without worry of collision. But I don't think IP6 addresses are being assigned randomly, hmm.

> 32bits is just about large enough (within an order of magnitude sense) if the space was used more efficiently.

Well, not really. Between just the populations of the US, Europe and China (places with high levels of internet connectivity), you have over 2 billion people (this site claims over 5 billion internet users: https://www.statista.com/topics/1145/internet-usage-worldwid...).

> But I don't think IP6 addresses are being assigned randomly, hmm

That is, in fact, exactly how IPv6 addresses are assigned using SLACC with Prefix Delegation. Your ISP assigns you a prefix, and your computer randomly picks an address within it. You can also self-assign a (non-routable, like 10...*) prefix from the fc00::/7 ULA block by randomly filling in the remaining 57 bits to form a /64 subnet.

> Between just the populations of the US, Europe and China (places with high levels of internet connectivity), you have over 2 billion people

You have to consider the context: back then, multi-user computers were common. Each user didn't have their own computer; instead, they had a terminal to connect to a central computer. So a single computer would serve tens or hundreds of people, and as computers became more powerful, you could expect each computer to be able to serve even more people.

Not really. Under ten billion would be plenty if used efficiently. A significant fraction of addresses we want to be private, and not directly routable.

Sure, if you want your refrigerator, oven, and dishwasher publicly addressable on the internet it isn't enough, but you don't actually want that.

Further, 64 bits is many orders of magnitude overkill already. So what does 128 bring to the party, besides making addresses harder to type?

Also, random with a prefix is not really random.

Those estimates typically confuse the 10^100 upper bound on the number of atoms in the universe with 2^100. The 2^128 number of addresses in IPv6 is clearly more than the latter, but dwarfed by the former. There are roughly 10^40 or so atoms in the universe per IPv6 address; by mass that's approximately one address for each combined biomass of earth.

Earth mass divided by 2^64 is roughly 357 tons. There are roughly 2^33 humans on earth, so 2^64 is "only" a billion or two addresses per person; tha's far fewer addresses than the number of human cells out there.

In practice, ipv6 often uses only 48-56 bits for global routing with 8-16 bits usable for subnet ids or routing within a customer site. The last 64 bits of the address are used to identify a particular device in a subnet. So, an ipv6 address is a lot bigger than an ipv4 address - but that is because those bits are doing more things than they are in ipv4.

IP4 already has the concept of subnet routing in 32 bits. Let's say we add 1 more byte for country: 0 = Legacy, 1 = USA, … 127 = Vanuatu.

IP6 didn't add one byte, it added 12! I think 64bits is plenty (as mentioned astronomically larger than 32), and still looking for reasons to change my mind.

IPv4 subnetting and IPv6 subnetting don't really work the same way, though. With IPv4, there are (almost?) no ISPs that are handing out class A blocks to consumers. The vast majority of consumers will get a single IP address - and then they have to use a NAT setup to create subnets on their end.

IPv6 is _dramatically_ different. Its actually possible to get a /48 or a /56 - sometimes just for free as part of general operations. That leaves 8 or 16 bits for the customer to create hundreds or thousands of subnets. Unlike with IPv4 where most customers don't get more than one IP address, you don't have to use a NAT setup if you have a /48 or a /56. Even if you only get a /64, that still leaves 64 bits to give every device its own globally routable address without having to setup NAT.

Do we need 64 bits to identify individual devices in a subnet? IDK, maybe not. but if you are doing an apples-to-apples comparison, with IPv4 you have 32 bits for global routing. With IPv6 you have somewhere between 48 and 64 bits. The remaining 64-80 bits of the IPv6 address don't have a good analog in IPv4. So, in a lot of ways, IPv6 is a lot like taking IPv4 and expanding it to something between 48 and 64 bits and then the remaining bits you can think of basically almost like extra fields to encode information that IPv4 doesn't support.

> “why is 64 bits not enough?” Here, why not an option?

Because then you don't really have enough bits to do some nice things:

1. It would be nice for your upstream network provider (and you) if they can delegate some network prefix and thus don't need to concern themselves with the address plan inside your subnet. That means, if there is an address collision on your prefix it's not their problem.

2. Assuming you have been delegated a network prefix, having at least 48 bits for within-subnet addressing greatly simplifies self-assigned addresses:

2.1 You can use the data link layer's address (the 48 bit MAC address for ethernet) which often has weak guarantees of uniqueness (though this isn't really advised today)

2.2 You can also randomly generate addresses or generate them by hashing things, and be reasonably confident of not colliding with another station.

2.2.1 Of course, you want to check that no other station is using the same address, but there's always the hidden node problem: How does station C cope with stations A and B both claiming address XYZ?

2.3 You can (if you want) still use DHCP to assign addresses if you want. But you don't have to.

3. It would be nice if each station was not limited to a single address.

3.1 Among other things, having multiple addresses vastly simplifies building a multi-homed network. For example, you can (today) sign up for two (or more!) ISPs that support IPv6 Prefix Delegation and have your router(s) issue Router Advertisements (RAs) for each prefix.

If a link goes down, your router(s) don't need to do anything or share any state... it just works (yes, really, I've tried it!). Each of your routers, of course, only routes packets for the prefix it was delegated by its upstream.

(BTW, IPv6 also has some neat RFCs for letting foreign hosts know about a prefix change, so you can even transfer existing open connections initiated from an address on prefix A through a different router with connectivity on prefix B)

3.2 For privacy, wouldn't it be nice if you could generate a new address for every external server you connect to? You can do that with the 64 bit subnet address space--remember, your ISP has no say about the address plan within your network.

4. With 128 bit addresses, IPv6 allows you to delegate a /64 to yourself from the ULA block (the IPv6 equivalent of 192.168.. or 10...). You can just randomly pick a 57 bit suffix to append to fc00::/7, and you can be pretty darn confident that nobody else will pick the same one. And you get all the same advantages of self-assigned addresses (mentioned above) within that prefix.

4.1 Having a unique* local prefix maybe doesn't sound like such a big deal until you've tried to merge two enterprise networks that both have hosts on 10..., and clients hard-coded to connect to those hosts.

Finally, think about how a 64 bit global address space would be split up:

We're running out of IPv4 addresses, even with* NAT (and "CGNAT", which is just regular NAT with a fancy name). Most of those endpoints are users, not servers--you need at least 30 bits to give one prefix to each household with a current internet connection. Between the expected growth of internet users and all current enterprise networks, you need well in excess of 40 bits for the prefix.

So you'd realistically get no more than /48 prefix at home (16 subnet bits). That's not enough to do the cool autoconfig and privacy things I mentioned. More likely, you'd get a /56 because 256 hosts "ought to be enough for anybody" and you can't do the cool stuff anyway.

> For example, you can (today) sign up for two (or more!) ISPs that support IPv6 Prefix Delegation and have your router(s) issue Router Advertisements (RAs) for each prefix.

As a user of OpenWrt (which does exactly this) I strongly disagree that it is the right solution. If you announce on the LAN the prefixes obtained from a fast fiber and a slow LTE link, then devices will, well, get an address for each prefix. The problem is that they have absolutely no information to choose the source address correctly - it's all just numbers! So, just by bad luck, they choose the source address from the LTE prefix and waste the LTE data and my money if I use the default setup. Which is why I don't. My network uses IPv6 NPT, but announcing one prefix at a time would have been even better (because I only want fail-over, not real multihoming), although impossible with OpenWrt.

Forum discussion, which also serves as a proof that it is really hard to explain the "don't overcomplicate fail-over by generalizing it to multi-homing" notion: https://forum.openwrt.org/t/ipv6-wan-fail-over-without-ipv6-...

Interesting, but still looks like most of this could be done in 64 bits, or ~4+ billion Internets, one for each public household.

IPX had a lot of convenience features in a similar number of bits. (80, but 24 were wasted on manufacturer, so 56 useful bits)

A rotating address number will not provide privacy if the prefix attached to you is the same. Would be like saying a rotating port number would provide it, but not the case.

No organization will ever need a /64 (not even close, not even wastefully-that's 18 quintillion).

The slow uptake in IPv6 seems to imply that it's over-engineered and people don't care about these potential additional features. I'm a lifelong geek and can barely get interested. Network engineers are maybe .01% of the population.

I guess bits are only getting cheaper in the future so why not spill them incredibly wastefully at anything we can think of, is ultimately the answer. Although it doesn't answer why 64bits was not even an option.

> even the defense department doesn't need 4.3 billion of anything

uhm. counterexample: they need more dollars than that

need?

they are able to create the need.

it's hard to make a bill communicate over copper wire though

Not that hard really, smart cards are a thing and the chips themselves can be tiny.

Every bullet gets its own address.

CNLP had variable-length addresses. I wonder what the world would look like with variable-length addresses.

It had variable length addressing "up to 20 bytes", which IIRC was done so that it could easily accommodate different addressing systems into one. Unfortunately, variable length fields make for really annoying hardware implementations.

TUBA (replacement of IP with CLNP, running TCP and UDP on top) mandated use of max size addresses always, to make it simpler to decode, as part of "Internet Profile CLNP".

A prominent example of: nothing more permanent than a temporary solution.

Reply on news.ycombinator.com