I think this is a minority view, I used to have to wear suits for work and its a pain making considerations for them.
Ideally, anyone could wear whatever they want whenever they want. In practice, it doesn't seem to work out this way, even at the tshirts and jeans companies.
What kind of company did you work for where this happened?