Hacker News Reader

Comment by YetAnotherNick | Hacker News Reader

Wow, 480 GB per GPU! What happened to end of Moore's law?

I hope this improvement translates to consumer GPUs as 24GB is a big limitation.

Moores law stated transistor density per square inch roughly doubled every 2 years.

I see no end in sight for that specific law. As we can always go vertically if needed. It’s also held through so far in 2020[1].

The folks who conflated moores law to also mean doubling of compute processing capabilities of a CPU double every 2 years were wrong.

[1]https://en.wikipedia.org/wiki/Moore%27s_law

Smaller transistor means lower energy consumption, going vertical won’t solve this.

This ceiling will be hit much earlier than what process/technique allowed.

smaller energy per transistor, but if you're packing more in the package, the package's consumption will grow up. Also, I think leakage current (and the heat that comes with it) goes up the smaller the feature size.

Not to be pedantic but wouldn't stacking transistors have no effect on density per square inch? Since it would only increase density per cubic inch.

I view it as if you cut 1 square inch of a motherboard. That the every 2 years you’d expect to see roughly double the number of transistors in that cut out piece.

Scaling vertically would “technically” still meet the above.

I don’t think that’s what they mean by per square inch. They mean in a plane, not a volume. If you add a third dimension the law stays the same, because a volume is two planes and the density law applies to each independently. That’s why node sizes are a single value not a two dimensional value. A 3nm node is 3nm feature sizes, regardless of dimensionality.

The quote given on Wikipedia is:

> The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years.

If he was talking about the area of a single transistor, there would be more concise ways to put it.

Even a 1000ft thick motherboard?

That'll show Moore

More’s law: if there is more of it, it does more!

Stacking transistors increases density per square inch if it can be done on a single wafer of silicon, because its "per square inch of fab wafer silicon"

We include multi-story buildings when we calculate population density, why not include multi-story chips?

To be fair, if you stack them, density is not going up - only if ignore the number of stacks and take one of their areas for the total number of stacked transistors would it then go up. Plus, stacking is great, but with heat issues, isn't the industry going to many dielets with a massive interconnect?

Heat issues are very valid. But the "per square inch" density is relative to a square inch of fab wafer. So if it can be done on one wafer, it counts. If it's stacking discrete chipsets, not so much.

Stacking likely wouldn't save substantial cost compared to producing multiple different wafers. It could even increase cost if it decreases yield. That's very different from making the transistors smaller, where the cost per transistor decreased exponentially in the past. People focus too much in Moore's law (transistors per area), when the only interesting quantities are 1) price per performance and 2) power draw per performance.

If measured in square inch the 3rd and more importantly the 4th dimension are not accounted for and are basically free.

Another way to say it is to count the famous founder brown: ymmoore wasn't thinking 4th dimensionally.

For shame really

> I see no end in sight for that specific law. As we can always go vertically if needed.

Not a hardware person but heat dissipation becomes more of a problem when you go vertical IIRC.

> As we can always go vertically if needed.

I don't think you understand how difficult non-planar transistors are to engineer at scale

How difficult is it? Difficult or really really difficult?

Per your link, Moore's law also doesn't state anything about density. Density is just one of the ways "The complexity for minimum component costs has increased at a rate of roughly a factor of two per year.", i.e. Moore's law only ever stated transistor count per a given price roughly doubled every 2 years.

Moore's original article on the topic in 1965[0], and the same with additional context interview form 2005[1].

> "The original Moore’s Law came out of an article I published in 1965...I had no idea this was going to be an accurate prediction, but amazingly enough instead of ten doubling, we got 9 over the 10 years, but still followed pretty well along the curve. And one of my friends, Dr. Carver Mead, a Professor at Cal Tech, dubbed this Moore’s Law. So the original one was doubling every year in complexity now in 1975, I had to go back and revisit this... and I noticed we were losing one of the key factors that let us make this remarkable rate of progress... and it was one that was contributing about half of the advances were making. So then I changed it to looking forward, we’d only be doubling every couple of years, and that was really the two predictions I made. Now the one that gets quoted is doubling every 18 months...I think it was Dave House, who used to work here at Intel, did that, he decided that the complexity was doubling every two years and the transistors were getting faster, that computer performance was going to double every 18 months... but that’s what got on Intel’s Website... and everything else. I never said 18 months that’s the way it often gets quoted."

Anyways, See slide 13 here[2] (2021). "Pop-culture" Moore's law stated that the number of transistors per area will double every n months. That's still happening. Besides, neither Moore's law nor Dennard scaling are even the most critical scaling law to be concerned about...

...that's probably Koomey's law[3][5], which looks well on track to hold for the rest of our careers. But eventually as computing approaches the Landauer limit[4] it must asymptotically level off as well. Probably starting around year 2050. Then we'll need to actually start "doing more with less" and minimizing the number of computations done for specific tasks. That will begin a very very productive time for custom silicon that is very task-specialized and low-level algorithmic optimization.

[2] Shows that Moore's law (green line) is expected to start leveling off soon, but it has not yet slowed down. It also shows Koomey's law (orange line) holding indefinitely. Fun fact, if Koomey's law holds, we'll have exaflop power in <20W in about 20 years. Which should be enough for people to create ChatGPT-4 in their pocket.

0: https://www.rfcafe.com/references/electronics-mag/gordon-moo...

1: https://cdn3.weka-fachmedien.de/media_uploads/documents/1429...

2: (Slide 13) https://www.sec.gov/Archives/edgar/data/937966/0001193125212...

3: "The constant rate of doubling of the number of computations per joule of energy dissipated" https://en.wikipedia.org/wiki/Koomey%27s_law

4: "The thermodynamic limit for the minimum amount of energy theoretically necessary to perform an irreversible single-bit operation." https://en.wikipedia.org/wiki/Landauer%27s_principle

5: https://www.koomey.com/post/14466436072

6: https://www.koomey.com/post/153838038643

Thank you for the detailed writeup with sources, I enjoyed this. I'd somehow never heard of Koomey's law despite working in tech, this is very interesting and directly relevant to the widespread deployment of AI (biological neural networks still blow silicon out of the water for computations per joule).

An implicit aspect of Moores law has been that cost per transistor has been going down as the density is increasing. This doesn't seem to be the case anymore. The technology required to get higher transistor density is getting ridiculously expensive. We're not seeing the power benefit of scaling down transistors either, since leakage is starting to get too high. I guess there's one more trick in the pipeline with Gate-All-Around, but I don't think I see a path to get better gate control after that. And if we don't get power consumption per transistor down, then stacking transistors in layers to increase density isn't going to be very viable for compute chips, since you need to get the heat out of the chip. IIRC, Intel is working on putting the power metal layers on the back side of the chip, which grows the chip vertical in the other direction so to speak. And it helps wick away heat as well, so could open a path for a few layers of compute transistors. But all this adds a huge amount of complexity to manufacturing, so at some point it might not be worth the cost anymore.

I thought the power benefits of shrinking still hold up rather well, in contrast to cost. E.g. new Nvidia gaming cards have smaller GPUs for the same price as the respective old generation, meaning the cost per chip area doesn't stay constant for improved manufacturing nodes. So the price per transistor shrinks slower than the number of transistors per chip area grows. At some point in the future the price per transistor would go up rather than decrease. Then the value of shrinking structures could stem, at best, from lower power draw per transistor. For mobile devices. But even power draw per transistor may stop decreasing at some point. Then further shrinking the process nodes would be useless.

That's a common misconception and I'm not surprised it made it into Wikipedia. Moore's 1965 paper was the first time anybody had pointed out the exponential nature of progress in miniaturizing transistors and packing more of them on the same integrated circuit. But it wasn't until 1975 a the same conference where Dennard presented his scaling laws that the phrase "Moore's Law" was coined in an interview where someone was trying to explain Dennard scaling to a reporter. The original coining was ambiguous as to whether it meant more transistors, faster transistors, or smaller transistors and that ambiguity remained in its usage because from 1975 until about 2005 they all went together just as Dennard said they would.

And my lecture notes from a class I took on semiconductor physics in college had a photo copy of a memo from Moore himself endorsing this broader conception of Moore's Law.

End of Dennard scaling was the performance breakdown. Meant chip frequencies couldn't be cranked higher and higher as temperature dissipation became more and more of an issue https://en.wikipedia.org/wiki/Dennard_scaling

Except for GPUs, which are for highly parallel tasks anyway.

A bit self-serving but GPU scaling supposedly follows Huang’s Law (from Jensen Huang of Nvidia) which claims GPUs more than double (~1.7x) every 2 years

https://en.m.wikipedia.org/wiki/Huang%27s_law#

> more than double (~1.7x) every 2 years

To clarify in case anyone else finds this confusing. The linked article suggests a 1.7x annual increase, which compounds to 2.89x every two years.

I'm pretty sure this law does no longer hold, as new Nvidia GPUs show only meager performance improvements over their (in class) predecessors. Though this could also be due to the price per chip area increasing faster than performance.

4090 is twice faster than 3090 in almost all metrics.

It came out two years later, so according to Huang's law we would still expect more than that. Moreover, most other models have seen much smaller improvements, like the 3060/4060.

There's a 24 GB limitation for consumer GPUs, because AMD and Intel aren't competitive.

80 GB per GPU and the consumer GPU is purposefully lower memory to induce demand for server grade.

On top of contracts strictly penalizing utilization of consumer GPUs in data centers, at that! Even with the memory, bandwidth etc. handicaps, servers with 4090/3090s would have been competitive for many ML tasks.

big limitations for what? AI models? It's certainly not for gaming and we're not quite to the point of consumers running huge AI models on their desktops. The HN crowd is, as always, not representative of the broader consumer market.

Aren’t we?

We’re seeing ChatGPT plug-ins for games, to provide intelligent conversation — and we’ve seen DNNs in StarCraft and similar.

To me, the “next gen” of gaming is intelligent NPCs, combining those features to create realistic behavior. That will require that consumer GPUs get closer to supercomputer GPUs:

More tensor cores and higher memory.

Other than the holographic projection, it feels like we're in reach of the holodeck - you ask for a scene with a character or general backstory, and you go in. Fun times. Now on that energy to matter and holographic projection part...

It's not a holodeck but Google has an interactive display now that feels like an open window. It doesn't even register in my mind as a display, it feels like looking through a literal portal to another location in physical space.

I tried to Google for more information, but I didn't find anything. Can you share a link?

https://blog.google/technology/research/project-starline-pro...

That is amazing. Do you work for Google or are you part of the early access program? How does it work? I cannot believe we haven't seen this on HN before!

https://en.m.wikipedia.org/wiki/Deep_learning_super_sampling

DLSS needs less memory than rendering in native resolution, not more.

Are there more dense memory chips or is it more of the same memory chips? Putting more memory chips on something doesn't have anything to do with increased density, which is what moore's law is about.

Reply on news.ycombinator.com