Hacker News Reader

Comment by eloff | Hacker News Reader

> Think Moore's Law, which is dead now.

Look again. Moore's law in terms of single threaded performance is dead. In literal terms of number of transistors on a wafer, it's still increasing exponentially, just slower. And in terms of what mattered when people cited Moore's law, the decrease in price/performance, it's still exponential, just slower and multi core.

It will end, as all exponential trends must end lest they consume the universe, but the end is not yet in sight. Recently chips have started to go multi chip and three dimensional. So even when we finally hit that scaling limit, we haven't quite reached the end of exponential progress.

View on HN

With the current silicon manufacturing processes, I'd say the end is in sight. Every node shrink is taking longer and longer and getting more and more expensive to fabricate.

We've been reliant on node shrinks to push things further. That hard limit is 0.2nm (silicon atom size) we are at ~2nm gate features.

It remains to be seen if we can get to that 0.2nm size, I honestly don't think we can push much further past the 2nm size (I'd imagine 0.5nm will be the limit).

That, to me, is very much being "in sight".

3d chips. Imagine current size with 1 million layers. Sure it will start with just a few layers. And many obstacles to be overcome like temp and delay. But one day our cpus will look like little cubes. Moore's law can continue for at least another 30 years

Couple of problems.

First, how do you cool such a chip? Sure, you could add a bunch of transistors but the thermal costs per transistor will stay the same. Assuming similar design but adding 20 layers, how would you account for your CPU now consuming 600W?

Second, The tech doesn't exist. The problem we have is that layering chips today is basically growing crystals in a controlled fashion. Defects in lower layers spoil upper layers. Today that's already a problem (CPUs today have ~7 or 8 layers). Can you imagine the problem of 20 or 30 layers?

Who knows, maybe we'll come up with solutions to both problems, but that's a big maybe. My bet is that instead we focus on architecture designs. We revisit old but deemed too expensive concepts like an async CPU.

This is real, shipping technology.

See 3d nand flash cells, see stacked cache and HBM memory. All the usual characters are working on it. Intel, AMD (TSMC), Samsung.

Cooling is an issue, but there are solutions.

3d cells for nand aren't the same as 3d CPU cores. Nand is incredibly simple which makes it possible to do things like 3d cells which would otherwise be more complicated.

There's a reason memory can generally beat computational cores to smaller node sizes. It's because highly uniform circuits are much easier to manufacture. A lot of problems present in CPU logic cores (such as cross talk) just aren't present to the same extent in NAND cells.

Maybe stuff like Upmem (many small CPUs embedded into memory) can take off then?

Possibly, that'd be one of the "major architectural changes" that I mentioned earlier. Things we've simply not tried yet because they are too radically different from the way hardware currently works.

Sorry to have to ask.Could you please show a Wikipedia-like intro to Upmem?

It may be a simple typing problem, but Wikipedia doesn't seem to have it.

I found a previous HN submission about them: https://news.ycombinator.com/item?id=20766283

> Today that's already a problem (CPUs today have ~7 or 8 layers).

No, I think x86 passed 14+ layers at least half a decade ago.

So I'm behind :D. What I remember is them trying to cut down the layers due to yield issues. Guess it's creeped back up again.

That's a continuation of moore's law, it's the same as having a bigger chip horizontally, or more chips. You can make giant chips but you need giants bank accounts.

It does not reduce price per transistor because each layer still has to be manufactured the same way

Sort of true, except you can use cheaper process scales for some of the layers.

The silicon is not actually the expensive part.

There are still many ways this may change. 3d transistors (which are currently used), different materials, and 3d chips (stacking layers of chips).

We are not yet close enough to the end to say where it is, so that's out of sight IMHO.

Like I said, cooling is going to be a major issue that doesn't currently have a solution (as far as I know) Even if it does, you are going to run into the real problem that even if you can cool a 600W CPU, would the average customer WANT to have such power hungry parts? Now imagine the problem in server space where the CPUs are already constrained by power consumption.

The entire reason cloud service providers are looking at and advertising ARM racks is the power savings from operating them.

I think there are almost endless gains to be had on the software side. Current software is so insanely inefficient you could probably triple the speed of the average computer without any hardware upgrade.

hasn't it always referred to transistor count and not performance?

Oxford Dictionary says: "the principle that the speed and capability of computers can be expected to double every two years, as a result of increases in the number of transistors a microchip can contain."

Which notably mentions performance. I have often interpreted Moore's Law to be the fact that doubling improvements in performance will continue roughly every 16-18 months, but not always directly proportionally to transistor count -- other ways to squeeze out doubling performance gains arise and often in unexpected places.

Oxford Dictionary doesn't know what Moore's law is, then.

Moore's law was an observation by Gordon Moore, co-founder of Intel, relating to the doubling of transistors per densely integrated circuit chip every 24 months. Originally, it was every 12 months, then every 24 months, etc.

Moore's second law, which isn't as well known, relates to the exponentially increasing capital cost of manufacturing ICs.

Single threaded performance and overall performance are quite different. But I'm not sure that Oxford is the most authoritative source here, as they are talking about a "computer" which is an awfully vague term. Wikipedia uses transistor count of an IC, which is much more specific:

https://en.m.wikipedia.org/wiki/Moore's_law

Would be interested to know if the process inside Oxford's is as precise to distinguish, or if they intentionally dumbed it down.

It has always referred to the minimum of the cost-complexity curve for integrated circuits. The progress of Moore's Actual Law has never stopped.

The thing people mean when they are talking about the deceleration of single-threaded CPU performance is actually Dennard scaling. Dennard said that density and power efficiency were complementary in such a way as to keep areal power constant. That was true until it suddenly stopped being true. If Dennard scaling had continued you'd be using a 20GHz CPU right now.

Did you read the whole comment? I edited it, but that part was in there from the beginning, just in different words.

I agree with you, I just think you weren't making your case strongly enough by implying that Moore's Law referred to more than just transistor count.

> ...I just think you weren't making your case strongly enough...

I get where you are coming from, but this is one of the most aggravating types of comments.

Fair enough

I am sure there is some sci fi novel where species compete for neutron stars because they are a very dense compute substrate.

In the Perfect Imperfection by Jacek Dukaj [0] has something called "Remy's curve" [1].

On the curve you can find:

- Ultimate Computer is the computer using the best hardware as allowed by the physical constants of our universe. See also physics of computation.

- Inclusions are in essence 'pocket universes'. They are created for specific entities to run hardware in a dedicated universe with physical constants different from ours, allowing for better performance than those in our universe

- Ultimate Inclusion is the inclusion with the best possible set of physical constants in the entire multiverse

Seth Lloyd wrote a paper "Ultimate physical limits to computation" [2].

[0] https://en.wikipedia.org/wiki/Perfect_Imperfection

[1] https://en.wikipedia.org/wiki/Perfect_Imperfection#/media/Fi...

[2] https://arxiv.org/abs/quant-ph/9908043

Not neutron stars specifically, but Accelerando by Charles Stross covers this.

That was a good book, it challenged many of my assumptions.

Asimov's "The Last Question" is close, though the focus is on entropy rather than any kind of space wars.

Isn't Three Body Problem kinda like this?

Is that book worth reading ? I am half-way through and I still don't see the point ;)

I read all three books. The first was a couple of years ago. But from memory, half-way through the first should be enough to tell. If you haven't seen anything worthwhile yet, you won't by continuing.

Reply on news.ycombinator.com