From my reading of degrowth works the answer is: because now the global resource use induced by growth have accumulated enourmous negative externality costs, costs that will nedlessly increase further unless counteracted by degrowth policy. Food shortages, deadly heatwaves, property destruction. Direct dire costs to human health and welfare and rippling secondary effects. For example how would a scenario with hundreds of millions of climate refugees play out on top of current global political tensions?
I see the degrowth view not as "we should never ever have growth again" but rather the current modes of growth have very tangible imminent (on an astronomical timescale) negative effects that we must deal with right now. If we fail to step up to the current challenge of provisioning health and welfare for beings on this planet how likely are we to manage an all universe encompassing expansion without it ending in catastrophe on the largest scale? You could see the climate change challenge, and the degrowth plan to tackle it, as a necessary step to get our house in order, before expanding out.
Us attempting to expand into and shaping the universe is quite a task - so much could go right or wrong. I think Toby Ord's The Precipice[0] has a very useful framework for thinking about such risks.