On the other side, advancement in quantum computers would make current LLM inference much faster. Because of the extreme cooling needed, i dont think the energy demand would become less.
With AI companies talking about AGI, i sometimes wonder if they really need the machines for serving inference to customers, or they have a formula for computational capacity that could run an AGI, and they just want to reach that level.