Because billions of people still live in poverty worldwide.
> I don't want endlessly slaving to make the pie bigger (while ignoring second order effects, from quality of life, to environmental issues, to debt etc).
No one does. Economic growth, although not by itself, helps alleviate a lot of these issues. There are other factors that also impact this, and I'd argue a lot of the economic issues we face aren't from too much growth but from these other factors (taxation etc.) not being handled properly. So in general more growth is advantageous, but we have to other things going right as well.
> How about we learn to enjoy a moderate rate of economic growth, or even a steady economy, and focus on improving other aspects of life besides bank accounts?
Well a lot of people disagree about what things like quality of life and adequate environmental care are. One of the beauties of capitalism is that it allows people with wide disagreements about what's important in life to coexist and even benefit each other.
Edit: sorry I didn't see this before:
> Under the current model though. Whereas e.g. better distribution of wealth might help them more than more lending.
Maybe, but if that better distribution of wealth comes at the cost of shrinking the overall system (which there is a real reason to think would happen: this is why people and corporations bank outside of the US frequently already) then I think it's hard to conclusively argue either way.
Again this is not to say I don't see problems with our current economy, I just think "it's growing too much" is not one of them.