You won’t live to see a 128bit CPU (2019)
blog.cloudware.bg
blog.cloudware.bg
I think a better argument is to look at history. Looking at https://en.wikipedia.org/wiki/X86, 16-bit x86 appeared in 1978, 32-bit in 1985, and 64 bit in 2003.
So, we have 7 years from “8 bits isn’t enough” to “16 bits isn’t enough, and 18 from there to “32 bits isn’t enough.”
I see something similar to (and probably a consequence of it) Moore’s law there: “we ‘need’ one more bit of address space every year or so”.
If that holds and Moore’s law continues, I expect we’ll run out of 64-bit address spaces about 32 years after we ran out of 32-bits, so around 2035. Top-end machines also should have use for 50-bit address spaces by now.
There were 64 TB systems six years ago (http://highscalability.com/blog/2015/8/5/how-do-you-program-...). That’s 46 bits, so I think my prediction is in the right ballpark.
I suspect it’ll happen because a 128 bit wide bus will be useful for calculations.
A 64-but cpu has registers that are larger than 64-bits (eg 512bits for AVX512) and the memory bus itself is already either 128 or 256bit already.
128bit pointers seems questionable for scalar processing and vector processing already has wider registers/GPU/TPU for acceleration.
From a security perspective 64bits of random space (actually 48ish) is pretty darn good and increasing that space for security is not something I’ve seen compelling papers on.
We do not need 128 bit data types in hardware. We barely even need data types with 64 bits of precision. It crops up every now and then- for instance, 32 bit floats cannot store a latitude/longitude value precisely enough, however a 32 bit integer can store a longitude value to about 1 foot of precision if -2^32 is 180 degrees west and 2^32 is 180 degrees east. (this has the nifty benefit of 2s complement integer overflow doing the right thing, although only with longitude, not latitude. but I digress.)
There are a handful of situations where you do need more precision than that- RSA and DH come to mind. But in those situations, you'll need arbitrary precision arithmetic anyway. A 128 bit register won't help a whole lot- certainly not enough to offset the cost of doubling the size of all your ALUs, even if the machine is a special purpose one that does nothing but sign SSL certs all day.
And then there was the time, a handful of years later, when "No one will ever need more than 640KB".
And then 4GB was stupidly massive, and no-one could want, let alone afford, that much RAM.
The same with CPU clockspeed (my first computer ran at 2MHz), storage capacity (I paid several thousand dollars for my first 5MB hard drive).
Which is not to say that the curve won't flatten out as we approach the number of atoms in the universe, but ... there's never quite enough of anything.
And those 2D circuits fit, eg. 4Gbits of DRAM into a single layer chip. The actual chip inside the plastic packaging is quite small -- let's say 4x4mm.
Now, you can can pack multiple layers of chips into a single package. Samsung is doing at least 12 layers for HBM, for instance, and they're only 60um thick each.
So you've got a chip that is roughly 65536 cells wide, and 65536 cells long, but you can only stack it 12 cells high.
Let's get that to 65536 layers: roughly a 4mm cube. That's now 256Tbits per package. Put 36 of those on a DIMM, and you've got an 8PB DIMM, and 512 PB in a 64 DIMM slot server.
Perhaps we can double the density? That gives us 4EB in a server.
This totally glosses over myriad difficult issues, but it's a very, very long way from being inconceivable when you consider the difference between a 512 byte 2114 RAM chip that was popular when I got my first computer, and what we have today.
Nintendo 64 and Playstation 2 were using 64-bit CPUs before PCs.
x86 was really late in the game.
It would not be that surprising if x86 ceases to be dominant in the future, at which point when the x86 became 64-bit is about as relevant as, say, the release of the PowerPC 620 (first 64-bit PPC chip, released in 1997).
I think the analogy with Moore’s law still holds if you pick machines used in comparable roles. Reason is that you can’t make much use of more memory unless your computer gets faster (an exception would be extremely sparse hash tables, but those would need amounts of memory we don’t have yet)
Assuming an extremely generous 2 cycles/byte, a 1MHz 6502 would take 2 seconds to clear 1MB of RAM, or 2,000 to clear 1GB. That’s over half an hour.
That first Alpha also only had a 34-bit address bus, limiting it to 16GB of physical RAM (https://en.wikipedia.org/wiki/Alpha_21064#Address_unit)
I picked x86 because it was used in more or less the same role for decades and because it made the 16-32-64 steps.
Notably, the bandwidth of RAM does not and has not been following Moore's law, improving linearly, and the latency is more or less constant. So as workloads scale up in size, performance will get worse and worse and worse, until "just buy more RAM" no longer solves the problem. Similar to how swap partitions used to be a thing, but performance is now so bad that swap can't be used as cheap RAM anymore.
https://www.semanticscholar.org/paper/Understanding-and-Impr...
If the top N bits of the global addresses are only ever read by routing software they don't need to be represented in the processor hardware.
I just don't think people would want to address random parts of memory even if a hypothetical exabyte memory hardware allowed it.
What we have are 64-bit instruction sets, which could have been used in the 16-bit era if we had wanted to for some reason. The real difference between 64 and 32 bit architectures is that when they went to 64, they said "who cares if we waste a few bits in pointers, we'd rather never have another change in pointer sizes." It represents no longer trying to shave every bit off of memory representations of code and data.
I can see that growing larger over time, in practice it already has with ARM SVE.
Because of that, memory bus width can also grow in a useful way.
But for adressable memory, I don't see it.
64 bits is likely to be the end-game until we use a radically different CPU architecture. (much bigger than the difference between a Z80 and a Xeon)
Of course, predicting the future is notoriously hard, but if you want to make an educated guess, it is better to not expect infinite and monotonous growth. There is always an end-game.
https://www.arm.com/company/news/2020/10/morello-program-one...
We have so many uses for pointer tagging we really ought to have more.
- The free software used to implement RISC-V architecture is defined for 32, 64 and 128 bits [...]
- Universally unique identifiers (UUID) consist of a 128-bit value.
- IPv6 routes computer network traffic amongst a 128-bit range of addresses.
- ZFS is a 128-bit file system.
- 128 bits is a common key size for symmetric ciphers [...]
- The IBM i virtual instruction set defines all pointers as 128-bit.
- Increasing the word size can speed up multiple precision mathematical libraries [...]
- MD5 is a hash function producing a 128-bit hash value.
- Apache Avro uses a 128-bit random number as synchronization marker [...]In particular, MD5 use in new software is strongly discouraged.
You'll note that some of them are essentially identical with https://en.wikipedia.org/wiki/256-bit_computing so they seem more guided by abstract arguments than concrete engineering reasoning.
However, it has an unusual architecture in which all programs run in a bytecode-based virtual machine, comparable to Java or dotnet. The 128 bit pointers are only used in the virtual machine. The underlying physical hardware had 48 bit addressing initially (starting in the late 1970s); in the 90s they switched physical hardware to support 64 bit addressing
Also, even though the pointers are 128 bit, many of the bits in the pointer are reserved for future use. Partly it was an exercise in extreme future-proofing. Partly because they use tag bits to mark valid pointers in memory (to stop programs from forging pointers), but they only have one tag bit reserved for every 16 bytes of RAM, so pointers must have 16 byte alignment.
Or running two copies of Chrome at the same time.
-possibly Bill Gates.
> "Gates has denied the quotation and the evidence is not compelling"
and
> "the comments by Nancy Andrews and Jerry Pournelle showed that a common mindset of the period was compatible with informal hyperbolic remarks of this type. The 1989 speech by Gates presented a more sophisticated analysis of the growth of computing power and memory."
so I interpret the current retelling of the joke as putting words into Gates' mouth to try and knock him down a notch, and the original 1985 editorial by James E. Fawcette more along the lines of truthiness ("the belief or assertion that a particular statement is true based on the intuition or perceptions of some individual or individuals, without regard to evidence, logic"). Note how the editorial starts "Legend has it".
There's no evidence that Gates said it, and he denies saying it.
I gave citations for how several people at the time said it unironically.
Nancy Andrews: "When the IBM PC-1 was introduced in 1981, it came equipped with 64K of memory. Those of us who upgraded our systems to 128K thought we had all the memory we’d ever need."
Jerry Pournelle: "My first microcomputer had 12K of memory. When I expanded to a full 64K, I thought I had all the memory I’d ever need. Hah. I know better now."
And the earliest reference to that supposed quote was an editorial with many examples of how software wasn't prepared for larger memory use - hardly an example of "a joke nobody got at the time".
As I don't have a Dr. or Phd but I do have a MS, you shouldn't call me "Doc". The appropriate academic title is "Master", though that seems excessive for HN.
[0] I started with microcomputers in '83 and did some mainframe programming in FORTRAN so I'm aware of the context.
Nancy and Jerry were not programmers. We have no indication who said it, or when, but clearly somebody did. If that somebody was a programmer, they could only ever have said it ironically. It would not be surprising for somebody who was not a programmer to miss the joke.
If you get a job as an assistant at the airport and somebody asks you to fetch a bucket of prop wash and a hundred yards of flight line, don't. If you are told to ask somebody for a penny long stand, be prepared for a long wait (or just take the afternoon off).
> We have no indication who said it
Kirk in ToS never said "Beam me up, Scotty", but that's the quote everyone knows.
Truthiness trumps truth.
It appears Gates' and Microsoft's views at the time were something like what he said in 1989: "That is, a move from 64k to 640k felt like something that would last a great deal of time."
Note that "a great deal of time" ruins what you think is a sarcastic joke. That qualifier is also exactly the sort of nuance that's worn off in retelling, as dropping it makes the retelling more interesting.
Truthiness trumps truth.
We know that the earliest form of that attributed quote is in an Infoworld article by editorial director James E. Fawcette, dated April 29, 1985 as "When we set the upper limit of PC-DOS at 640K, we thought nobody would ever need that much memory. — William Gates, chairman of Microsoft"
It has no citation, no additional context, and several other parts of the piece ("Legend has it" and "was reported") strongly suggest it was not written with historical accuracy foremost in mind. Instead, I interpret it as using these examples as parables to make the editorial argument more vivid.
Truthiness trumps truth.
Thing is, I agree it's a joke. I think the joke was "Look those derpy engineers. They're so close-minded! Don't they know that we always want more RAM, more disk space, and more of everything?!"
But that raw joke questions engineering tradeoffs and decisions, and as you rightly point out, journalists rarely have that expertise. So the joke is made of Jobs and Gates, because it's always fun to take CEO's down a notch. Even if it wasn't true.
Truthiness trumps truth.