Different people lose energy from different tasks. I.e. it may not tax your mind to have clutter around, but it can be a distraction for someone else. For some people (like me), clutter is fine but starting on a task takes a lot of energy. The important thing is knowing and accommodating for yourself to get the best results.
But you could just remove things from your existing install rather than starting all over. It really isn’t about clutter.
The “clean install” thing is a habit from when old Windows would have phantom issues if you didn’t start over every so often.
There are many tools I wish I had never installed. I’m now installing them on a VM first