Will Chiplets Save Moore's Law? | Hacker News Reader

Will Chiplets Save Moore's Law?

semiengineering.com

42 pointschipletman51 comments

Moore's law is about transistor density, which has already slowed way past what Moore's Law (more like Moore's Observation) noted, a doubling of density around every ~18 months (give or take 6)... Stacking may have allowed the trend to continue, but it doesn't seem to work well.

What chiplets offer is effectively more compact compute units designed to work together. What was effectively a component design for multi-cpu systems a couple decades ago happens in a single socket package. With some better design considerations.

This allows for an optimization for loss... if you have a huge single chip, there's more chance you lose a whole chip in a wafer, meaning higher manufacturing loss. With smaller chiplets, you may lose one smaller chiplets, but the other 4-6 around that one lost still works. This means better yeilds out of each wafer.

The other advantage is that you can mix different manufacturing nodes... such as compute or graphics chiplets being latest and greatest with memory and bus interfaces on a prior generation. Again, this allows for greater production and reducing loss.

> Moore's law is about transistor density, which has already slowed way past what Moore's Law (more like Moore's Observation) noted, a doubling of density around every ~18 months (give or take 6)...

What makes you say that? It looks right on track:

https://en.wikipedia.org/wiki/Moore's_law#/media/File:Moore'...

Dennard scaling is the one that has broken down and forced us into multi-core versus high-frequency:

https://en.wikipedia.org/wiki/Dennard_scaling

I don't think that chart actually reflects Moore's Law well, because it doesn't normalize for total area. Many of the chips listed, especially at the high end, are very large but not especially dense.

The two are pretty related though. If you could build a chip with more transistors economically but had to run it at 500MHz for cooling reasons, you probably wouldn't do that. (The exact nature of the challenges in the two domains differ though.)

>What makes you say that? It looks right on track:

Especially given that the vertical axis is a log scale, it looks to me as if it's flattening out a bit. I'd have to run an actual regression but that's what it looks like in terms of eyeballing. (And it's consistent with what we know of process nodes slipping out.

The processors which are creating this flattening of the curve are for low-power, mobile or embedded use-cases where transistor density comes at a bigger cost. The more cores you add, the more overhead power consumption there is for switching.

Density affects high end processors too. We already are past the point where it's necessary to throttle down to protect a processor from self-destruction. Nobody wants to employ the sort of heat exchanger necessary to keep running full bore all the time. Power consumption takes a back seat to power density.

It's been past that point more or less for 20 years. Thermal throttling is nothing new.

People do exactly that. More cores, even at a lower frequency still benefit from the ratio of cycles to memory latency being reduced. The computing power and energy usage is much better, the problem is only software taking advantage of more cores. Intel's most expensive 28 core CPUs have lower clock speeds and more cores.

This means better yeilds out of each wafer.

This can potentially help keep scaling alive. Aside from the question of "can we do it", there's always the other question, "at what cost". Instead of an "impossible" tech node, it's conceivable that we could hit a "too expensive to be worth it" tech node. Thus any scheme to keep yields up helps delay that point.

Chips are manufactured

There are many things in this world where one can, for a premium, through your social network, or dumb luck at a second hand store, acquire something that is relatively unique.

But if you want something cheap, you are left with what can be made reliably. And if you want to keep buying it in the future, that thing has to be made and sold at a profit.

So the only chips you can get are the ones that can be made all day every day. Chiplets are 'just' more of them. Where 'just' means 'assuming you have a cost-effective way to manage the logistics of processing, testing, and assembling an order of magnitude more pieces of inventory, and that the error rate in those processes is less than the difference in yields between chiplets and giant monolithic chips'.

Writing that down gave me some new appreciation for a side comment that was made in a chip design interview posted here this year. The guest mentioned how important manufacturing processes are today in chip design, and I don't think any of us outside of that industry understand the smallest fraction of that.

> Moore's law is about transistor density

It's about the number (not density) of transistors on the most economic die size.

It's broken for a long while, since modern processes are much more expensive than older ones.

Scaled for inflation?

Where exactly do you expect inflation to apply?

> modern processes are much more expensive than older ones.

Huh?

The original Pentium was $878 in 1993 dollars when it was released. Adjusted for inflation that's $1,567 2020 dollars. An i9-9900 is $549.

> The other advantage is that you can mix different manufacturing nodes... such as compute or graphics chiplets being latest and greatest with memory and bus interfaces on a prior generation. Again, this allows for greater production and reducing loss.

The "previous generation" doesn't necessarily mean worse these day, nor say much about transistor density.

It's a non-secret in the industry that after gate size scaling froze at around 30-40nm, all further advances in density has actually reduced the performance of an individual transistor.

You were still able to cram more of them in the same space, and get a net performance benefit from transistor count.

It really depends on how you interpret Moore's law. I personally think that counting all your transistors across multiple discrete dies is kinda cheating.

But, I absolutely love the concept of chiplets and can see a trivial 10x in logical transistor density, even if we don't shrink the process nodes any further or even make major changes to the current generation of chiplet-based CPUs.

We are currently only arranging these things in 2D. If AMD/TSMC develop a way to stack chiplets in 3 dimensions, we could wind up with another huge bump in logical transistor density per socket/node.

Chiplets are ultimately about solving the manufacturing and business problem of defect density. There isn't much additional magic going on here in terms of beating Moore's law.

> We are currently only arranging these things in 2D. If AMD/TSMC develop a way to stack chiplets in 3 dimensions, we could wind up with another huge bump in logical transistor density per socket/node.

We already hit up against Dennard scaling more than strict Moore's law issues (although there's hard problems on that side too). Cooling those transistors is already such an issue that we have concepts like dark silicon, even when you assume that the transistors themselves are in a 2D plane immediately backed by cooling.

> If AMD/TSMC develop a way to stack chiplets in 3 dimensions, we could wind up with another huge bump in logical transistor density per socket/node.

Stacking is pretty close to being a solved problem. Cooling a stack of high-power chips is the challenge.

Right. Memory can be stacked. Flash memory, especially. Only a small fraction of memory cells do anything on each cycle. But parts where most of the gates do something on every clock, like CPUs, heat up too much.

I think an appropriate way to measure Moore's law is computational power per dollar. Whether that's mips or floating point math, there's some discussion there.

> If AMD/TSMC develop a way to stack chiplets in 3 dimensions, we could wind up with another huge bump in logical transistor density per socket/node.

Isn't that already done for HBM (high bandwidth memory)

Yeah, but it'd be way more complex if you changed those chips to being regular logic, or your memory access pattern to HBM changed.

GPU memory is overwhelmingly accessed as long streaming transfers, which means that the hotspots (like, literally tiny points of high temperatures) migrate around the chip which is ideal for cooling. That's why you haven't even seen HBM for general purpose CPUs yet (I guarantee you that a threadripper could consume that bandwidth under the right circumstances).

There is nothing preventing usage of the same microvias and wafer thinning for non-memory ICs.

Sure, but the hotspots are going to be intrinsically more localized in logic chips, or even memory chips with non streaming access patterns.

And? Semiconductor thermal engineers are paid salaries to solve exactly that.

I mean, we've thrown tens of billions at that problem and don't have a good solution beyond "chips with consistent hot spots array their transistors on a 2D plane, and even then you have to play games with turning parts of the chip on and off to keep it from melting itself".

If you've got a solution that's manufacturable, I'm all ears, and so is every fab out there.

I am a layman here, but couldn't they just use more aggressive cooling on the outside? If the exterior were cooled to -20°C, wouldn't that keep the interior from getting too hot as well?

That's even worse, frequent thermal shocks are not good for IC longevity.

ICs frequently passing zero degrees, which aren't packaged in completely water impermeable packaging like ceramics or metal are also having issues

Sure, they'll last shorter, but at least they will work.

Besides, couldn't they keep it at a stable -20 °C?

> ICs frequently passing zero degrees, which aren't packaged in completely water impermeable packaging like ceramics or metal are also having issues

Issues with condensation?

Well, that makes sense. So why can't they be packaged in metal or ceramics?

> Well, that makes sense. So why can't they be packaged in metal or ceramics?

Money

> Sure, they'll last shorter, but at least they will work. Besides, couldn't they keep it at a stable -20 °C?

I can't say that. I'm not a thermal engineer

A CPU costs at least $50 already. Is it that much more expensive to package it in metal?

Also, aren't they already packaged in metal? On graphics cards, the dies are directly exposed.

Liquid nitrogen cooling can already get good results in overclocking, so it seems like it should work here too.

It's not a new thing. The Vax 9000 had multichip modules with a very very special interposer for the same reasons. The transistor density they wanted combined with the yields they were getting meant that they couldn't fit the whole design on a single chip, so they broke up the chip and tried to heavily control the cross chip interconnects more than you could if you went out to a PCB and back to sort of 80/20 your way to what a single chip would get you.

The difference now though is that it's not certain that yields will get markedly better at these nodes and smaller, whereas yeah, we got a lot better yields on much smaller nodes than 1750nm ECL logic eventually.

> The Vax 9000 had multichip modules with a very very special interposer for the same reasons.

Pictures, for anyone interested: https://vaxbarn.com/index.php/feat/8-collection/394%20-dec-v...

Moores law has both scaling and economic aspects. Folks seem to focus on the scaling, which to a degree is still happening, but the economic aspect has not been true for the last few nodes

I've recently seen two talks on how Moore's Law is alive and kicking, by none other than Jim Keller and (IIRC) a chief scientist at TSMC. Wonder why this notion of Moore's Law dying stays so pervasive.

Do you have links to those talks?

Here's what I found: https://www.youtube.com/watch?v=Nb2tebYAaOA&t=1805s (94min, jump to 30m for Mr. Keller discussing Moore's law and his interpretation of it.)

Jim Keller: Moore's Law, Microprocessors, Abstractions, and First Principles | AI Podcast Lex Fridman, host.

I think that is the one people should watch. The 1000x1000x1000 atoms analogy. But then again he hasn't touch on the economic issues of Moore's Law

Here's another one: https://eecs.berkeley.edu/research/colloquium/190918

Moore’s Law is Not Dead Wednesday, September 18, 2019

Every 2 years the number of people saying Moore's law is dead doubles.

-- A microsoft exec I cannot recall!

The notion of Moore's Law dying is because John Hennessy and David Patterson said so in their Turing Lecture: https://www.acm.org/hennessy-patterson-turing-lecture

To a typical consumer "Moore's Law" used to mean they could buy a computer that's the same price but twice as fast as the one they bought a couple of years ago. That trend has been dead for a while now.

The technical definition of Moore's Law is still standing, but it's relevance has retreated into areas where relatively few people notice. E.g. massively multi-core server chips and flagship phone SoCs.

This is mostly Intel's fault. The day Ryzen came out, I almost quadrupled the computing power of my similarly priced Intel CPU. If I were to do so again I could get an eight times more powerful CPU for that money.

Intel and TSMC's business models are built on top of Moore's law continuing, and them maintaining an edge over the lower-end manufacturers. The end of Moore's law is an existential threat to their viability.

At the same time, businesses that design specialized architectures, or work on alternative devices have been publishing things about the End of Moore's Law for a long time [1], since it means that their 2x improvement of the day won't be obliterated by next year's Moore's law jump.

[1]: E.g, this article from 2000, saying that the "End of Moore's Law" is why their research is relevant https://link.springer.com/chapter/10.1007%2F978-0-306-47013-...

Those are called "multi-chip modules". The Pentium Pro was built that way. AMD uses them today.[1] They're useful, but more to help with cooling and yield than density. The transistors are not any smaller.

[1] http://developer.amd.com/wordpress/media/2017/11/LE-62006-SB...

I'm both irked and refreshed to read such a poorly written article. There are a few grammatical errors and generally poor prose that bother me as a reader. At the same time it's clear the author is an engineer and well versed in the subject matter and attempts to gift some of their expertise as a reader.

The title about "Save Moore's Law" appears to have been inserted as click bait. The chiplet idea as an approach to building configurable logic has some merit but is really unrelated to Moore's Law.

No, chiplets will not save Moore's "law". They may provide another path to configurable logic beyond custom ASICs and FPGAs. It's kinda like boards and TTL logic but at the chip scale.

Did they change the Title or Someone submitted it as this? The article is now "Chiplet Momentum Rising". And no where does it suggest Chiplet saving or extending Moore's Law.

Betteridge's law of headlines vs. Moore's Law: who wins?

Will chiplet model work? Or is it all hype?

The chiplet model already happened and is already working. AMD's much-lauded new line of processors are all using it.

Reply on news.ycombinator.com