I tend to do the opposite. I distrust people who wear suits, especially in settings where it's not common to do so. I see it as a possible indicator that they are trying to hide their limitations.

For example, people giving their talk wearing a suit at a scientific conference? Very high chance of bad science. If you do amazing research you don't need a suit to impress anyone, the ideas you present will be more than enough.