Hacker News Reader

Comment by trashtester | Hacker News Reader

We may be getting close to the limit for 2D density, but have just barely started moving in the 3rd dimension.

Chips are also quite small, limited partly by the ability to cool them once they enter a computer and partly because a single defect often means the whole chip must be discarded (which means large chips generatel lower yields).

I suspect we will see much more development in all of these directions, with individual chips extending deeper into 3d and getting improved tolerance to defects allowing them to get larger, as well as with chiplet, die-2-die, stacking and similar methods of combining chips in a package continuing to move forward at a rapid pace.

I don't think we should expect foundry development to stagnate in the near future. If anything, as AI starts to be used in developing new chips, it may well accelerate.

View on HN

>> We may be getting close to the limit for 2D density, but have just barely started moving in the 3rd dimension.

Nah, 3D is already well under way. Micron is stacking 232 layers of memory cells to get high density: https://www.electronicdesign.com/technologies/embedded/artic...

Flash memory is also increasing the number of bits per cell by essentially going analog. So those two thing are why you can get 2 TB on your keychain.

Then AMD is doing their 3D V-cache, putting a bunch L3 or L4 cache on a die stacked on top of the CPU. The issue there is that both the CPU and cache dissipate too much heat to keep stacking higher much higher.

GPUs are using HBM memory which is also stacked, but again power dissipation is going to limit how far that can go.

Even in low end devices - Raspberry Pi - we've had DRAM stacked on top of the SoC die for many years now.

20 Years ago 3D was seen as a way to higher levels of integration once scaling came to an end, but it's already been happening in more and more places as the end of scaling is near. Innovations will continue to squeak out slowly for some time.

> layers

Thinking in terms of layers is still essentially 2D.

The manufacturing process uses layers, so I'm not sure how that follows.

Tell that to my 3d printer

I'd say the 3D printer is an example of my point. You can design and think in 3D despite the manufacturing process being layered. In contrast, chips are designed from a 2D standpoint. As an example, does the ALU circuitry span layers or is it just on one?

This is all just memory and not the processor itself

Because logic dissipates a lot more power. You can't stack CPU die because the power density goes up linearly with the number of chips.

Edit: power per area goes up.

3D makes every part of your heat story much worse.

You don't want to just stack layers... modern chips already have tons of layers and each one is an opportunity to screw something up. Stacking of dies has its own problems but people are shipping stacked dies and have been for some time and as noted the heat is a major problem.

Things will improve but not like they used to. It doesn't take that many more scaling nodes before gates are a handful of atoms and quantum effects dominate. But really gate sizes stopped shrinking a while ago - scaling has been terribly uneven for some time.

Imagine if you can produce 3D compute substrate as 1 liter modules that can be built as Lego's. With holes to run water-cooling pipes through.

If you have, say, 100l of such substrate for less than $10,000, with a density similar today's tech, just in 3 dimensions, you can probably afford to run most of it at very low frequencies to conserve energy, with only non-parallelizable operations running at GHz-level frequencies.

Such a machine could still probably provide as much compute as a small datacenter can today.

Now I don't know if it will take 20 years or 200 to be able to do something like that, but I'm certain that we have a looong way to go before we reach the physical limit to how much compute we can stuff into a rack.

I am not in the chip industry at all, but I understand that chips offload so much heat in part due to cramming as much as performance as possible into a small 2D space. If you increase the number of layers by 10-20x you now have much more "surface area" to work with. Could we see chip designs that operate at a much lower voltage (thus minimizing heat) and are "slower" as measured by chip frequency, but have greater overall bandwidth? Maybe a chip for servers that has 256 cores on it, each with their own caches?

From a Moore's law perspective this would continue the transistor count doubling trend too.

No not really.

Dynamic switching power consumption is proportional to frequency which is why things slowed down dramatically once we hit 1Ghz.

Of course you must reduce voltages to reduce current and losses... but at a certain point your transistors no longer switch between clear on/off states because the voltage swing is too small. This is a quantum effect of all semiconductors (related to their valence band energy IIRC). The tiny leakage this represents only makes your heat dissipation story much worse. So you end up trapped between two walls: higher voltage is required to drive the transistor between full on-off states but lower voltage is required to keep the gate from burning itself up. No matter what material you pick (silicon, germanium, etc) there is some minimum voltage required.

Also your insulating layers become a problem as they shrink: quantum tunneling allows electrons to jump the insulating layer. There's a tradeoff here too... alternative oxide layers reduce leakage at the gate but have worse barrier constants and so are prone to more quantum tunneling (related to conduction band energy).

Adding more layers just makes more problems, as each layer you stack risks mangling the layer below it. Plus the risk of introducing a defect. The more defects you have the worse your yields and the more expensive the chip gets.

The short version is: we are at the point where no matter which direction we go some quantum effect kicks in to cheat us out of further optimizations in that direction.

It’s a larger surface area only if you increase the area without increasing the heat generation. That is a heat sink. If you want all those layers computing and therefore producing heat, then you might as well keep it flat and have more fluid moving heat away. In a 3d space that fluid is going to be started between different layers and reach its carrying capacity of heat quicker.

I don't think we'll see much of a change in how we're approaching an asymptote for the foreseeable future. In order to do something like that, you would need a significant innovation that disrupts the whole trillions of dollars industry and it's unusual for a maturing industry to see that kind of change. If any one of these innovations were significant enough, I think it would have been prioritized above the other innovations that have been done instead which only get us smaller incremental gains.

I imagine we'll still see gains of 10-15% a generation with whatever improvements come over the next few decades but I don't see us going back to the gains of the 80s and 90s where performance for the end user was doubling regularly, if that's what you're hoping for.

Edit: Also I think the price for the performance will continue to rise. If we're looking at performance per dollar, I think gains will only be in the 5-10% range

The funny thing about the 3D chips is that if extrapolated far out, we end up with cubes. Definitely sci-fi territory. And maybe questions from kids like "but if they are shaped like cubes, why are they called chips?".

A major difficulty with 3D is heat dispersion. It will be interesting to see how this will evolve.

Reply on news.ycombinator.com