There doesn't need to be infinite growth potential for there to be potential for massive expansion of civilization.

The energy that is being wasted every single second from the sun emitting its rays out into empty space is many orders of magnitude greater than what humanity collectively uses, and all of that energy could be harvested and even saved if civilization expands to colonize the entire solar system. Until it does that, the vast majority of the energy outputted by the sun is going to waste. So slowing growth is generally extremely wasteful.

And yet they and their ilk dominate (and thus drive most of the progress in) their respective areas. It's almost like not pre-deciding you're doomed and instead just getting on with it is the secret to getting anywhere.