Each switch of a gate causes energy to be expended. When a transistor switches, there is some current flowing and some voltage across the source and sink. That voltage * current is greater than zero, and therefore requires power. This is opposed to the stable non-switching state where there is no current and therefore no power.

So, because each switch consumes the same amount of power, the power consumed is directly proportional to the switching frequency. Double the frequency = double the power.

As other posters have alluded, Intel tried to work on the power issue by decreasing the voltage. Unfortunately, they were not able to figure out how to decrease the voltage as they had in the past. With the high power required to run high frequency circuits, cooling would have been more of a problem than could be overcome economically.

If I understand correctly, Intel CPUs around that time were super deeply pipelined so you could meet timing closure at such high frequencies. So yeah you can keep pushing the clock-speeds up and up but now a mispredicted branch really shoots you in the foot. Makes sense at the time because people associated clock speed directly with performance. Higher frequency more better. Intel kinda gave up on that at some point and started over with the core architecture design which has a lower clock speed but does more each pipeline stage.

There is also another fundamental limit, the speed of light.

Because of the special relativity theory nothing including electrical current can transmit information faster than the speed of light (c = 3*10^10 cm/s). So if maximum size dimension of your CPU is say 2cm (I'm ignoring that they actually can have multiple smaller cores), then the maximum physically achievable frequency is upper bounded by size/c = 15 GHz.

Although it has not been reached yet, it has the same order of magnitude as the record frequencies achieved by overclockers using liquid nitrogen cooling (something a little below 9 GHz). Also it shows that reducing transistor size and as a consequence overall CPU size increases maximum physically allowed frequency.

Mostly heat dissipation. Assuming you hold voltage constant, heat increases linearly with frequency. Also, as process nodes get smaller, you increase the # of transistors per square inch, so a jump to a new node can also increase heat generation quadratically if all else stays constant.

As other commenters mentioned, you can counteract this with better cooling, but eventually the marginal $/watt of cooling isn't worth the tradeoff of lower speeds & multicore processors.

> Temperature? Stability?

Yes. Notice how after it was clear that P4 was a dead end they went and dusted off their P3 mobile line that they had been making more power efficient the whole time?

Technically it was heat but mostly it was due to (i) Economics, there was less demand for faster clock speed. Otherwise more research could have gone towards solving heat problem. (ii) Each cycle of CPU was more efficient with ability to execute multiple instructions in a single cycle and with more efficient instruction sets.

Surprisingly, power consumption also made huge impact. As tablets and laptops got more popular than desktop battery life became a major concern and thus TDP played major role in research.

Try this fun experiment: Underclock your CPU by half a GHz and see if you notice the difference in your day to day work.

heat mainly and the shift toward mobile devices which implies good autonomy and power efficiency.

Not enough time for electrons to move. Photons - another story.

They were scaling vertically because the GHz game was a marketing strategy and not an engineering strategy. But when the pipeline got so deep that it didn't make sense (branch misses would incur huge penalties) and AMD stole the spotlight, marketing finally gave way for the engineers.