Moore’s Law is not Dead
tsmc.com
tsmc.com
Note the historical context the observation was originally made in (1965) when doubling was achieved through process size reduction, the effect this had on speed was twofold: operating frequency would increase roughly in proportion (increasing about 40%); secondly and more uniquely to the time, since predecessors transistor count were so limited there was significant room for improvements in instruction implementation and specialization as available transistors increased.
Although it's implied, Moore also explicitly stated this in his 1965 paper:
> [...] In fact, shrinking dimensions on an integrated structure makes it possible to operate the structure at higher speed for the same power per unit area. [1]
Later this effect was more explicitly defined as Dennard Scaling in 1974 [2]
Transistor count increases in recent years has very little to do with dennard scaling or improving individual instruction performance and everything to do with improving some form of parallel compute by figuring out how to fit, route and schedule more transistors at the same process size, which does not have the same effect Moore was originally alluding to.
[1] https://drive.google.com/file/d/0By83v5TWkGjvQkpBcXJKT1I1TTA...
It makes the point that Moore's law is dead. You can see the slope change in in 2006 with the Dual Core Itanium2. So it died after Dennard scaling toppled (and some of the fastest growth ever), and although transistor count continues to increase it is noticeably slower... I fear that it may go slower still since economic costs continue to increase, but profits don't (as fast). When the CFO figures out it's not worth investing in scaling, you just won't build new fabs (unless the gov just says so).
To put some numbers to this (topline growth):
1971 TMS1000 (8k) to 1979 M68000 (70k) 2.6yr/double
1979 M68000 (70k) to 2003 Itanium M6 (400M) 1.9yr/double
2003 Itanium2 6M (400M) to 2006 Dual Itanium (1.6B) 1.5yr/double
2006 Dual Itanium2 (1.6B) to 2018 GC2 IPU (24B) 3.1yr/double
Personally, I think that an era of backend improvements (wafer stacking etc) to combine processes (Memory/Logic/Flash) into locally interconnected 2.5D is possible, if there remains sufficient investment, and that in turn could drive some improvements to performance and pricing. However, that relies on ever more complex system design and integration... and heat dissipation issues. I don't see full 3D scaling or Quantum compute on the 3-5 year horizon where big new investments will be required.
(edit formatting commentary and qualification)
(edits corrected grammar and double negative added cooling comment)
https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrh...
Is that what Moores law says? Or does die size count? If you plot total #transistors your fitted slope will plummet in the later years and you'll see the change post 2006.
Yes, I would be surprised if the rate manages to be even linear - it can never be exponential as it was with process shrinkage because now the opposite is happening - we are increasing the total die size, but without changing the frequency. The first problem of this method is dealing with increased propagation delay between distant blocks, requiring different architecture and scheduling (assuming we just accept parallel computing as the only way forward). Then there is the much harder problem to sustainably solve: power consumption and heat dissipation, requirements that grow in proportion to the transistor count.
... all of these issues were naturally avoided with process shrinkage, propagation delay reduced in proportion, and the power density per unit area remained roughly the same, and so up to the physical limits of the material it was a truly sustainable window of exponential growth.
You're ignoring the physics. You can do all you want with new hardware designs, larger dies, more CPUs etc but ultimately it cannot make the same exponential gains that came with process shrinkage because power consumption and heat dissipation problems increase with every extra transistor if the process size remains the same.
I'm not saying gains cannot be made without shrinking, but they will be incremental, at the most linear with a short window, but they will not be exponential and they will come with another cost (more power consumption). For Moore's Law to be revived, look to material science for answers.
He's not ignoring the physics, he's saying Moore's law (the "throw more hardware at the problem" version) is dead because of physics...
There isn't anything wrong with that, but for TSMC's Global Marketing guy to try to do that, it makes me think that TSMC doesn't get it.
The argument "That isn't what Gordon meant, if you look at what he really said..." Attempts to dismiss four decades of what every engineer expected it to mean, and nobody was was correcting them because, well why should they because it was working.
It wasn't until it started obviously failing to be true, that semiconductor companies started arguing in favor of an interpretation they could meet, rather than admit that Moore's law was dead, as pretty much any engineer actually building systems would tell you. Somewhere there must be a good play on the Monty Python Parrot sketch where Moore's law stands in for the parrot and a semiconductor marketing manager stands in for the hapless pet shop owner.
It is really hard to make smaller and smaller transistors. And the laws of physics interferes. Further its really hard to get the heat out of a chip when you boost the frequency. Dennard, others, have characterized those limits more precisely and as we hit those limits, progress along that path slows to a crawl or stops. Amdahl pretty famously characterized the limits of parallelism, we are getting closer to that one too, even for things that are trivially parallelized like graphics or neural nets.
The fear semiconductor companies have is clear, if the only way to improve performance is better software, then you don't need new chips, and an idle fab, ready to do a wafer start, costs nearly as much as an active one.
Tl;Dr - old Moore's law is dead , but our new systems(ai , gpu...) still need a huge number of transistors , beyond what can be made on a single chip today, so scaling is still valuable.
So while Moore's law is still ticking along (albeit a little slower), and transistors are still shrinking (7nm vs 14nm). It doesn't really affect the price, so instead of being able to buy 4x the performance (transistors) for a similar price, it hasn't changed. It seems pricing is no longer based on cost, it appears to be based on what the market will pay.
Strangely, it doesn't seem to be the same with CPUs. During the same period, AMD has doubled in core/ transistor count between Zen 1 and Zen 3 while has stayed the same price[3].
[0]https://www.guru3d.com/articles_pages/msi_radeon_rx_5700_xt_... [1]https://en.wikipedia.org/wiki/List_of_AMD_graphics_processin... [3]https://en.wikipedia.org/wiki/Ryzen
In many ways, Nvidia is acting like Intel did. AMD simply couldn't compete on performance so this allowed them to get cosy (Intel's high-end for consumers stayed at 4 cores for a decade, Nvidia's prices have been steadily raising for a while).
Also, a lot has happened in the last few years: RAM prices skyrocketed, Bitcoin mining drove up demand for AMD GPUs. However both of those have been over for a while now.
The rx480 was priced very aggressively, the $199 price grabbed some headlines (8gb card cost a little more). Given the die shrink and three years (plus the Ryzen cores doubling), you'd be forgiven for expecting the rx5700 to come in closer to $199, than the $349 (XT is $399) keep in mind that these new cards aren't much over double the performance.
While, it's no secret that shrinking fabrication is getting harder and more expensive (it's hurting Intel right now). Prices per transistor have dropped for CPUs and SSDs has dropped noticeably, even DRAM to an extent, but not GPUs...
Even demand for machine learning doesn't explain the price hike (GPUs are now being artificially handicapped, just as professional graphics have been, shifting demand to the even more expensive Tesla cards).
If something changed and it’s not dead, I’d love to hear more about the new processes that are making more transistors possible. Working at a chip maker now, but I’m a software guy. My understanding of the problem is that we’ve reached the optical limits of resolving power for the lithography processes. Trace widths are a tiny fraction of the wavelength of light, and chip area is so large we don’t have lenses than can keep the projection in focus at the edges. While there is theoretical potential to get smaller due to gates being many atoms across, actually building smaller gates has real physical barriers, or so I’m told.
I’d love to hear more about the manufacturing processes in generally, and more specifically whether something really has changed this year. Does TSMC have a new process allowing larger dies or smaller traces, or is this article mostly hype?
Extremely hard to make economical though.
There's no _real_ discussion of Moore's law. No new revelations about chip design. You say workloads need to exploit parallelism these days to see increased performance gains? No shit. Putting memory closer to the logic cores is a good idea? duh. Hell, the author makes the common mistake of conflating AI with ML, because it's clearly illegal for a businessman in any industry to not buzz about "AI".
> by Godfrey Cheng, Head of Global Marketing, TSMC
Yeah, this is fluff.
It's like documentation these days that is all autogenerated boilerplate.
> [..] divided by two every 18 months [..]
Wikipedia wrote:
> [..] whose 1965 paper described a doubling every year in the number of components per integrated circuit [..]
Godfrey Cheng, Head of Global Marketing, TSMC wrote:
> [..] The number of transistors in an integrated device or chip doubles about every 2 years. [..]
(Even if we ignore the difference on the cost versus number of transistors or components.)
12 months, 18 months, 24 months. How long is it? I have no clue with regards to the answer. It seems to me we first need to agree on the definition of a law before we can discuss it.
From https://www.cnet.com/news/moores-law-to-roll-on-for-another-... (a source for the the Wikipedia page).
https://en.wikipedia.org/wiki/Transistor_count
2019 - Epyc ROME: 32B transistors 2009 - Six-core Opteron 2400 - 0.9B transistors
(32/0.9)^(1/5) = 2.04, so it seems that Moore's law is actually working well.
If it was once 100 years it was also still exponential.
Take a look at this: https://www.hotchips.org/program/
For the first time in a long while they gave the entire first day to non-semi companies: Amazon, Google, Microsoft
Nobody could've imagine the industry turning this way a decade ago.
They've given the entire first day to the semiconductor divisions of these hulking behemoths.
The problem is somewhere along the next 5 years we may see cost per transistor stop decreasing. i.e Your 100mm2 Die would be double the price of previous 100mm2 Die assuming the node scaling was double.
At which point the cost of processors, whether that is CPU, GPU or others becomes expensive and the market contracts, which will slow down the foundry pushing for leading node. We could see Hyperscaler each designing their own Processor to save cost, and we are back to the mainframe era, where these Hyperscaler has their own DC, CPU, and Software serving your via Internet.
Like, I'm a gamer in 2029 and I'm looking for the equivalent of todays Intel Core i7 or AMD Ryzen. How much faster will it be? How different will it be from today? Etc.
The point I'm making is that the processor doesn't matter much for gaming. An older computer can easily outperform a new computer if it has better components where it matters for gaming and casual use (SSD and GPU)
Cloud is limited by Amdahl scaling like everyone else, but distances and latencies between nodes are higher than in a single box. So instead you should think of time sharing like in the ancient days of supercomputers. Most tasks do not require neither massive parallelism of a cloud or cluster nor computational resources of a supercomputer.
If it's the question of ownership or price... Why not make it free? (Ignore this junk, make libreoffice etc. better.)
Just think of it as a PC that gets smaller. You cannot quite go super small unless you replace input and output devices. That will require more understanding of our own sensorium.
I last upgraded my CPU when it was nearing 6 years old, and I could barely tell the difference. My current CPU is nearing 5 years old, and if it wasn't for some of the enticing stuff AMD's recently put out, a CPU update wouldn't even be on my mind at all yet - in fact I was surprised to recently notice it had been more than 4 years since I last bought a workstation CPU.
I expect I'll buy 2 workstation CPU updates between now and 2030, 3 at most. and even that would take something special to drop in the early-to-middle of my normal timeline.
on the other hand, game developers are only just starting to spread the workload across more than one or two cores. far cry 5 is a pretty unfun game to actually play, but if you look at benchmarks you'll see that it actually gets some decent speedup when you go beyond four cores. ultimately it's still bottlenecked by that one main thread though. if the trend continues, gaming might actually benefit from the current race to cram in as many cores as possible.
a final random thought: I wonder if we will start to see big.LITTLE style architectures on x86 desktop parts. maybe 2-4 big cores optimized for very high clocks paired with 16+ smaller, more efficient cores for parallel tasks.
Depending on how much companies try to milk servers, we should see commoditization of large and fast non-volatile memories. This has many impacts; we might see laptops with hundreds of GB of effective RAM, and filesystems, with appropriate software to handle it, when run from these devices should approach the performance of in-memory data structures.
Cores themselves will probably be a fair bit wider, but struggle even more than they currently do to utilize their full throughput. Typical programs will run faster mostly because of the increased cache size and (potentially) vastly faster IO; optimized programs will benefit more from the actual throughput increase.
Hopefully chips will have continued increasing their core counts—it's certainly possible, if the will is there. Programmers will follow through after a delay; programming for parallelism is only done when cores are numerous. I could imagine 8 core 24 thread being fairly standard, for instance.
GPUs will just continue to get better, as density and memory and packaging allows them to do so. Neural network accelerators will be a lot better than today, and will very much appreciate the larger, faster memories that the future offers.
The x86 tax continues to grow as competition from Apple's CPUs (though not necessarily direct competition) results in cores wider than x86 can handle without more and more caching and other expensive optimizations.
Mill Computing finally start work on their FPGA.