Its not about the fashion, its about looking put together. Its signaling that you care. Of course, there's the article which says that people who made it can stop signaling, so maybe that's you.
Don Draper "I don't think about you at all" elevator meme in real life.
Little known fact: before he was Pied Piper's cool attorney in Silicon Valley, but after his first job at a Superstore, Ron LaFlamme lived in NYC where he was told that he "wasn't thought about at all" by Don Draper. https://youtu.be/LlOSdRMSG_k
Sorry, no. There's nothing wrong with me for not putting effort into what aesthetic "signals" I'm sending, because there is no true signal. It's noise. If someone's unclean, that's one thing, because cleanliness matters to health. Fashion does not.
Arguments that fashion matters are all circular, which tells you something. (It matters because people think it matters, and is therefore worth taking seriously!)