Source: I'm from the south of Italy, and I have Nonnas :D
I've seen moka pots where the upper chamber is black with oily residue accumulated in years of not cleaning it to avoid this "sin". The resulting coffee is sour and bitter and generally disgusting (but again, that's what we are often drinking normally).
Part of the idea of not cleaning it is connected to the material (aluminum), which makes a protective layer of oxide which could be damaged by scrubbing; apart from the fact that using regular dish soap is not the same as scrubbing, the oxide will reform, you would need to eat a moka every 3/4 year for aluminum absorbtion to be a problem, and the whole point falls apart when using a steel moka pot.
I recommend trying out yourself in a double-blind test instead of trusting "received ideas": does the coffee after cleaning it taste better? worse? same?
[1] https://www.gamberorosso.it/notizie/caffe-bar-inchiesta/
I don't know how one can question taste... it's taste after all! It's subjective.
What I'm hearing is that you don't like bitter espresso with sugar. This is to my taste, and yet, I'm not questioning yours :)
Do as you please, eat whatever you will, if it diverges from tradition or received knowledge and you're happy about it go ahead and enjoy!