There are many ways to dramatically lower power usage if you're willing to accept slower speeds. Intel has recently demoed some extremely low-voltage transistors that used power in the microwatts. You can make up for this dramatically lower speed by adding far more circuitry using 3D chip-stacking.

Another possible solution is to use 3D chip stacking to add dedicated hardware for a wide variety of tasks, most of which is turned off when not being used.

There's the intermediate step to 3D - 2.5D. it's when you lay all you dies side by side, on another chip, instead of a pcb. It makes relatively easy to manage heat and you can get much more bandwidth at much lower power(much shorter wire distances). And it's already in use for high end fpga's.

Plus, even if cooling between layers were figured out, we'll probably just end up with lots of transistors that we can't all power on at once, since we're already at practical power draw limits for most devices.

We don't just need to cram more transistors on a chip, we need those transistors to use proportionally less power (preferably even less than that).