Moore on Moore – The past, present and uncertain future of Moore's Law
thechipletter.substack.com
thechipletter.substack.com
I've largely concluded that it's a combination of three things.
1. As Fenyman said, there's plenty of room at the bottom. This is why so many orders of magnitude improvement have been available.
2. As Turing proved, computers are general purpose. A consequence of this is we have only a small number of problems to solve that apply to all software, and so as the industry has grown exponentially, we've been able to put concentrated resources into making hardware better.
3. Exponential growth is simply the best we can do as human innovators.
Exponential growth isn't a property of the problem; it's a limit of our innovation. We are able to hit the limit due to the economic consequences of computers being general purpose, and able to sustain it due to there being plenty of room at the bottom.
I would imagine that faster computers could help design better computers in the future, thus there's some level of exponential growth there. However, i think computer aided designs are reaching the limits of what help it gives, and the bottleneck is human intelligence and material engineering difficulties (that cannot use the help of computers).
that's why moores law was more exponential early on in the miniaturisation of chips, but it's slowed down more and more - till one day it's gonna be sublinear.
Of course, this doesn't preclude a new breakthrough in a different medium (optical computers, rather than electronic?), or a different form like quantum.
where 2) implies that the total (economic) effort expended towards the process is small compared to the total (economic) benefit derived from the process. As the sibling comment already states: We could not produce modern chips without (at least modernish) chips to execute the design tools. But I'm not sure how critical that actually is for the exponential growth (extrapolating from the inefficiency of all other software written today). The interesting question is, whether we are close to a limit yet. Which, I think, translates to the question, whether we are already close enough to the edge of what is physically possible to have some significant economic cost (with respect to the derived economic benefit)
What makes me think right now is your 3) .. What would a (sustained) better-than-exponential growth require and how would we identify such a thing/opportunity
Innovations in transistor design help to shrink it on one dimension (e.g. the semiconductor size gap). This translates to a silicon wafer in two dimensions.
Therefore, for every factor (n) decrease we get in transistor length, we can fit (n^2) more transistors on a chip. Therefore we don't need (as) exponential amounts of innovation to get exponential outcomes.
Maybe I'm being crazy, Babbage you can correct me if this is way off :)
So I wonder if the emphasis on exponentials is misleading. A geometric decrease in feature size is sufficient, there were lots of atoms to work with while making wires ever smaller, and up till 2010 or so the regime of Dennard scaling gave shrinking transistors the very nice property of not affecting power consumption.
I know the article goes into more complex techno-economic reasons (and I guess that is what you are really discussing as well), but most IEEE papers cite Dennard scaling as a critical enabling driver. The end of Dennard is what made the IDRS roadmap identify an "end" to Moore's law.
Quick aside, I get what you are saying (that is: multiply the original size by 0.7), but as a non-native speaker of English: isn't "shrink by" the opposite of "shrink to", and wouldn't this wording technically imply shrinking to 30% of the original size?
This is why parallel compute (multi core/GPUs) has taken off since 2006 - heat puts a cap on clock speed, so the only option is to do more operations per cycle. Even some microcontrollers are dual-core now.
I don't see this trend stopping. Massive parallelism is the future of computing.
Price increases 50% every 2 years
Also, even if Moore's law remains in place for now, it will end at some point and the world needs to think about what happens afterward.
From my dealings with parallel algorithms, there is much less sequential dependency than I think most people expect.
The brain is massively parallel - 80 billion neurons operating asynchronously at only 100hz.
>is having a neural network the end problem that needs solving?
Absolutely! Neural networks are a general-purpose way to solve a huge number of problems, especially problems that involve raw data or the messy real world.
You contradicted yourself in that last paragraph. I don't know what else to say to you about that.
Parallelism allows you to do computations that integrate huge amounts of information, like checking every pixel of an image against thousands of weak rules all at once. This is very inefficient to do serially because you can only check one at a time. It's also a huge usecase right now because machine learning lets you learn weak rules from data.
Of course there may be some practical limit to Gustafson's Law, but I don't think we've found it yet, at least in many scientific domains.
Rowhammer has to do with RAM density, but again, hard to exploit and there are mitigations.
https://news.ycombinator.com/item?id=32077583
I run with `mitigations=off` in offline cluster.
So outside of popular culture, it is dead...
It’s not my field of expertise, but couldn’t we adopt a measure of density of transistors? Billions per mm or something like it.
A single number is also good for marketing as the MHz of the 90s have shown us
If it's for engineers: In 2024 a single metric is going to be far too simplistic to realistically drive any decision.
If it's for the general public: Any metric will do, might as well stick with nm.
That's actually what the current "marketing nm" are meant to do. They're the notional extrapolation of simple planar transistor density to complex 3d transistor architectures.
People like to stand on a soap box and stand impressed with themselves for pointing out these are "just marketing numbers not the real half pitch anymore" but they're missing the point. The industry continues to use the nm nomenclature because it's clear to them what everyone is referring to.
Modern fab processes are so complex you're not going to characterize their actual technical details by any single number.
The best way to understand the dimension is to look at the contacted gate pitchm2 pitchcell height,and take the square root. The pitches obviously make an area, the cell height sets the 'design' component. So you get a reasonably scaled number.
Putting it that way also clarifies technology. Why euv, to tighten contacted gate pitch. Why cobalt: to tighten up metal 2 pitch (without killing resistance). Why backside metal: lowers cell height. There is of course more to it, but it is a good way to coarsely understand
A good way to know if it is marketing is if someone talks about feature sizes instead of pitches. Patterning is done by pitch. For example, you don't do euv to make smaller features. You do it to make more complex layouts at tight pitch.
I am having trouble parsing `pitchm2 pitchcell height`.
Assuming that you meant to use `*` and then HN's markdown assumed it was italics, and placing `×` where italics start and begin:
sqrt(pitch × m2 pitch × cell height)
Then I am left with the questions:
What are the units of pitch? Best guess: length units? (Based on: cell height having length units. `m2 pitch` being dimensionaless?
Why do the pitches "obviously make an area" then?
What are the units of metal to pitch? Best guess: dimensionless. (Based on the final units of the sqrt being repoted in length, and the cell height presumably having units of length).
What the units of cell height? Best guess: length units, based on what the variable is called.
And there are a lot stuff in there besides transistors: contacts, metal traces, power routing, ...
And there's also the quality of the software that draws the transistors. (No one draws billions of transistors by hand) And to further complicate things, there are many good reasons to make your transistors larger than the minimize size, and in fact many are.
So, it vaguely means density went up and unfortunately there isn't a single resource that says how to measure it or what the limit is.
Come to think of it, It's kind of like Moore's Law (the topic of TFA) in that it used to have a definite meaning. But then the meaning sort of evolved over time into something else without society actually agreeing on what that "else" was.
EDIT: did some snooping around, Moore himself said it in 2005:
"We have another 10 to 20 years before we reach a fundamental limit."
New semiconductor materials or entirely different approaches (like quantum computing, optical ICs, 3D stacking etc) may give us a few more orders of magnitude.
But as the article states: most likely it's economic barriers. Market conditions must support the technical effort required. Otherwise that tech is a dead end no matter how far it could go in theory.
FWIW: in a way I'd welcome an end to Moore's law. It's about time software bloat gets addressed more aggressively.
1) Enable case insensitivity, particularly for things like "transistor" which are rarely capitalized or "integrated circuit" but even for "should be capitalized things" like "Moore's Law" vs "Moore's law".
2) (Especially on things spanning less than a century) check the impact of smoothing. It can "smooth out" the introduction of term to appear to take off years early or hide the true peaks/variabilities because they only lasted a couple of years instead of decades like the default smoothing might be better suited for.
3) (Not as applicable here) make sure the terms are popular enough you're not just comparing noise.
4) Adjust your range so the data is properly scaled. If your term was coined around the 1970s (and comparatively uncommon prior to that)then the 1870s shouldn't be midway through your graph.
5) If various case insensitive spellings create a messy set of graph lines adding in a less popular term like "exaflops" will cause them to be merged and you can extract that line.
6) Be careful comparing things that can be pluralized or otherwise modified with things that aren't. E.g. "Moore's Law" vs "Integrated Circuit[s]?"
Combining some of these, notes like "Here is a further ngram of the ‘End of Moore’s Law’ which has clearly been a live topic of discussion since the late 1990s." actually look to be a bit off when you switch the graph to clue in on the last 30 years, set smoothing to 0, and enable case sensitivity to catch "[Ee]nd of [Mm]oore's [Ll]aw". Case insensitivity changes the shape of the humps and 2019 ends up significantly higher than the peak in the early 2000s. Shrinking the x-axis and not smoothing out the peak to be wider than it is shows there is very little in the way of the end of Moore's Law in the late 90's, only really starting to take off in 99 and not peaking until 2003.
A similar story repeats for the opening graph, the gap between variants of "Moore's Law" and variants of "Integrated Circuit" is more than double the frequency, adding in that variants of "Integrate Circuits" are even more than either and counted separately.
Overall the points turn out right enough (a few years difference in when the phrase found its first wave doesn't impact the story, nor does how frequent the use of Moore's law is vs ICs as much as the general rise of one and decline of the other) but I always want to highlight how the friendly interface of Google's Ngram viewer can easily lead you to the wrong conclusion on the data.
I'd also like to point out the above tips are neither exhaustive or universally applicable to any type of search. Sometimes you want more smoothing, sometimes you do want an exact version of a phrase, sometimes you do want to show the term was used long before the modern meaning, and sometimes there are additional things that can bite you. E.g. one I was worried about here was "Moores Law" typos excluding the ' but it turned out to not be common at all. You really don't know until you check each possibility though.
I.e. what initially looked like something that had been a common phrase before and into popularity in recent times turned out to be something that had been previously mentioned but not really in use then lines up exactly with the release dates of the book and movies for A Game of Thrones instead of something which slightly preceded it. This shows how easy it is to end up with two opposite conclusions by using Ngram Viewer slightly differently.
Alright, enough rambling about the hazards of Ngram Viewer in a post focused on Moore's Law :p.
Hence the commonly-encountered phrase: You believed a naive ngram reference? Oh my sweet summer child.
Either that or it's a Ship of Theseus moment and we can just redefine moore's law to be whatever the current rate becomes.