Stuff takes space, stuff adds clutter, some stuff requires administrative overhead, most stuff requires maintenance.
I prefer tiny apartments in central neighbourhoods than bigger ones in the suburbs. Stuff enters into that equation. I can't afford a big house in Kensington. So OK, if I could afford a big house in Kensington plus a couple of pied-à-terre, maybe I wouldn't worry so much about stuff. I could have an extra room for a corkscrew collection and hired hands to deal with it. Stuff staff.
I added the "certain level of wealth" qualifier to make it clear that I didn't mean to claim that being burdened by stuff is a universal experience. It's a problem tied to abundance, like obesity and information overload. This level of wealth is much lower than what most people would consider "wealthy".