Comment by fizixer | Hacker News Reader

Comment by fizixer | Hacker News Reader

Moore's law is dead.

First, Intel and the semiconductor consortium started the garbage definition of a technology node about 10 years ago, when pitch levels stopped following Moore's law.

Now TSMC and all Intel competitors straight up started lying through their teeth. I'm sure what TSMC is calling '5nm' is a complete joke even in face of the horrible redefinition 10 years ago.

Don't believe me? See for yourself:

- nVidia GeForce GTX 1080 Ti was announced in 2016. base clock 1481 MHz, 11.34 TeraFLOPs FP32, on a 471 mm2 die [1].

- Four years later, RTX 3080 Ti, base clock 1905 MHz, 21.1 TeraFLOPs FP32, on a 700 mm2 die [2].

- 21.1 / 11.34 = 1.86x improvement in raw TeraFLOPs.

- 700 / 471 = 1.486x die size increase

- 1905 / 1481 = 1.286x base clock increase

- 1.486 * 1.286 = 1.91x

Conclusion: nVidia could use the bigger die size and the higher base clock to deliver 1.91x the performance using the same 16nm technology node (theoretically). But instead, using the 7nm node, they're delivering even less of a performance bump, 1.86x, when in reality it should've been 7.64x (= 4x * 1.91x, considering two node jumps, and the die-size and base-clock bump).

[1] https://www.techpowerup.com/gpu-specs/geforce-gtx-1080-ti.c2...

[2] https://www.techpowerup.com/gpu-specs/geforce-rtx-3080-ti.c3...

> Moore's law is dead.

GTX 1080 - 11b transistors (2016)

RTX 3080 - 45b transistors (2020)

From Wikipedia

"Moore's law is the observation that the number of transistors in a dense integrated circuit doubles about every two years."

> RTX 3080 - 45b transistors (2020)

That's just... insane. That's five transistors for every person on the planet. In 700 mm2.

Why does it make sense to compare it with the world population?

It doesn't, of course, except in the sense of "that's something that we know is a Really Big Number".

[Edit: And a number that people have some intuition of how big it is.]

True but the die size has also grown. So it not a 1:1 comparison.

An impressive feat for sure but it is just using brute force to increase performance nowadays.

Doing the math:

GTX 1080: 42.81 mm^2 per billion transistors. 41.53 mm^2 per TFLOP. 0.97 billion transistors per TFLOP.

RTX 3080: 15.55 mm^2 per billion transistors. 33.17 mm^2 per TFLOP. 2.13 billion transistors per TFLOP.

They scale well for transistors against die size, and TFLOP against die size, but terribly in transistors per TFLOP? That can't be right.

Checking https://www.techpowerup.com/gpu-specs/geforce-rtx-3080-ti.c3... I now see it says "Based on speculation / placeholder", which makes me think it's all just totally made up, and not reliable evidence to declare the death of Moore just yet.

Keep in mind that with the 2080, they added raytracing acceleration and tensor cores. Both add area and improve performance of specific user cases, but my guess is that they don't show up in that TFlops number.

This is it - you can't just compare FP32 TFLOPS. The newer cards have a ton of extra precisions, custom cores for certain workloads, and on chip memory, all of which use silicon area and transistors, but none of which boost the FP32 TFLOPS metric.

I could design you a chip that is nothing but FP32 multipliers and adders that has, theoretically, a ridiculous TFLOPS per mm^2, but it would be next to useless in any real workload.

I think in Moore's paper it was actually about economics of transistor production. But I think this definition is the more commonly cited.

The 1080 and the rumored 3080 are not the same architecture: they didn't just get die shrunk.

For the same number of SMs, you're also paying substantial area for the tensor cores, ray tracing acceleration, independent thread scheduling, different I/O to support memory technologies, more NV link, larger caches, etc. That stuff didn't happen for free.

A simply die shrunk 1080 would have had more TFlops than what we have, but it wouldn't be as interesting of a product.

Exactly, the reason for this is that it is an RTX and it has a lot of tensor cores and ray tracing cores which you won't measure by looking at the amount of FLOPS.

Moore's Law really has 2 aspects - scaling and economic. The original quote states: "The complexity for minimum component costs has in- creased at a rate of roughly a factor of two per year". This basically says things scale at a certain rate (the time frame was changed from a year to 18 months in a later revision), but also that it is cost effective to do so. The scaling is still happening to some extent, but the cost per transistor has flattened out more so. Performance and power efficiency were never part of Moore's law, and are better covered by Dennard scaling

>Now TSMC and all Intel competitors straight up started lying through their teeth.

The new nm numbers simply try to extrapolate transistor density using an old numbering scheme that everyone is familiar with.

Id be interested to see the wattage per teraflop difference between the two.

Reply on news.ycombinator.com