Nvidia GPU roadmap confirms it: Moore's Law is dead and buried

We would likely get 2x transistor per dollar every two year for quite a long time.

Moore's Law was an extrapolation on transistor density which is a manufacturing process in a chip foundry. NVIDIA is a fabless company that depends on physical process improvements from companies like ASML & TSMC.

So the competition you're speaking of is happening more at the foundry layer which would be TSMC vs Intel vs Samsung vs China-chip-maker (e.g. SMIC). At the lithography layer, ASML's EUV has no competition from Nikon and Canon.

Roadblocks they've seemed to put on themselves, in the name of market segmentation. There is a huge market sitting between consumer and "professional" GPUs, and nvidia is trying to milk the market as much as they can before any of the competitors get their shit together.

This is just one example, but I would be surprised if that wasn't a consideration in more datacenter placements.

The problem of course is to find a place where electricity is abundant, but you also have easy access to the large fiber backbone network. If you have a lot of traffic, fibers aren't cheap, especially interconnecting them with others.

Can’t they just buy indulgences^W carbon offset credits?

Agree. It is only 'dead' because Moore's law is about the cost per transistor, and Nvidia wants to keep that price constant.

Double the transistors? Double the price of the chips. That's Nvidia's wet dream.

I hope this backfires on Nvidia and the standard Moore's law returns.

If you mean chips that do fast matrix-matrix multiplications, that definitely has a long history and was not "just started".

If you mean calculations in low precision, that's also already a decade old if not more.

Two years ago, I snagged laptops with a 32GB RAM, RTX 4070, and i9-13900HX for $1,500.

Historically I've been able to get a sizeable bump in CPU + GPU performance for around the same price point, but currently there's nothing on the market.

It _feels_ like the pace of development has slowed in the last few years.

I don't think I'll be upgrading again until consumer X3D CPU laptops become available for a non-egregious price.

Yes, there is truth to that argument—both in the limitations it describes and the vision it hints at. Let’s break it down:

⸻

1. Bandwidth Bottleneck with Horizontal Scaling

• True: As you scale out the number of computational cores (SMs in NVIDIA GPUs, for example), the global memory bandwidth becomes a bottleneck unless it scales proportionally.

• GDDR6, HBM, and even on-package memory (e.g. Apple’s M-series) are attempts to address this.

• Still, there’s a ceiling: latency and contention start dominating.

• Problem: Many workloads, especially in ML, are memory-bound rather than compute-bound. Doubling cores doesn’t help if data can’t be fed fast enough.

⸻

2. Local Memory per Compute Unit

• True and Already Happening: Architectures are increasingly moving toward giving compute units local memory:

• NVIDIA’s SMs have shared memory (very fast, but small).

• AMD’s Infinity Cache, or on-chip memory on M1/M2, is another example.

• TPUs and custom ASICs use SRAM close to the compute logic for matrix multiplies.

• This is reminiscent of scratchpad memory in SIMD systems or old-school Cell processors.

⸻

3. Workload Partitioning and Data Locality

• Critical Insight: Yes, mapping data locality to compute locality is the key.

• LLMs and other neural nets are mostly large tensor operations.

• These can be tiled and distributed to cores with minimal inter-core communication if partitioned right.

• This is why hardware-software co-design is becoming more important:

• NVIDIA’s Tensor Cores and transformer engine

• Google’s TPU with XLA compiler

• Apple’s use of unified memory for CPU, GPU, and ML cores

⸻

4. Hardware Matching Software Architectures

• Emerging Trend: We are already seeing this:

• LLM accelerators (e.g. Groq, Tenstorrent, Cerebras) aim to align hardware with transformer-like workloads.

• Neuromorphic chips and systolic arrays are trying to escape the Von Neumann bottleneck by mimicking data flow instead of instruction flow.

• Vision: Instead of generic ALUs doing many different things, you have domain-specific architectures (DSAs) optimized for matrix math, attention, etc.

⸻

Conclusion

You’re touching on a real and current transition in hardware design: as Moore’s Law slows and workloads become increasingly specialized (especially ML/AI), the bottlenecks shift from FLOPS to memory bandwidth, latency, and data movement.

Yes, scaling horizontally is increasingly limited by data movement costs, and yes, future architectures will (and already do) look more like the software they’re meant to run.

Would you like a more technical breakdown of how LLM workloads are actually split across GPU memory hierarchies or how specific hardware like Cerebras or TPUs tackle this?