[1] "Jim: Here it is. The concept is that the speed of light is finite and a nanosecond is a foot. So if you buy a gigahertz processor, it’s doing something every nanosecond. That’s the event horizon. But that’s in a vacuum, the processor is not a vacuum, and signals don’t go in a straight line and the processor is running at 3 gigahertz. So you don’t have a foot. You’ve got four inches. And the speed of light in a solid is less than that. So this is the event horizon. If something happens on one side of this thing, the clock is going to tick before to the signal gets to the other side. That’s why processors of the future have to be small and in fact golf ball– size. Why are they smoking? Well, because they have to run on a lot of electricity. The way you get things to go fast is you put a lot of power into them. So heat dissipation is a big problem. Now it’s astonishing to me that Intel has decided that this is a big problem only recently, because people knew that we were headed towards this heat cliff a long time ago. And why is it hairy? Because you’ve got to get signals in and out of it, so this thing is going to be wrapped in pins." -- https://amturing.acm.org/pdf/GrayTuringTranscript.pdf
We are now definitely near the end of the S curve. May be Another 10 years of node advance and small incremental IPC progress. I often wonder, what's next. Or are we... nearly done and its all software?
With silicon based physical chips, I imagine this is true. I would guess, like some speculative science fiction, that the next leap in performance would be an entirely new concept. Who knows what is possible?
The more we go 3d on compute we have to cut ghz even more to reduce power, and that puts more burden on the programmer. Which we'll probably do some day.