I tend to elicit the feeling you are mentioning often, without intending to. I have seen old women move away from me while we were waiting to cross the street. I have been stopped by police because “I was acting suspicious” (I was walking to the gym, after work).
It’s not a nice feeling, scaring people just by being around.
Because of this, I never wear the hood of my hoodie. In general I have to wear as unthreatening clothes as I can. And I avoid dark streets, because someone can see me as a threat and fear makes people do irrational things.
We would all benefit from less fear.
I am not “athletic” though :)
In the end it is about a social contract, the details of which may change with the course of time.
Locality and social norms definitely play into this conversation, here in Seattle at least there is a culture that supports nudism, whether it's the summer solstice bike ride, certain beaches that are popular for swimming in your birthday suit, etc.
I've seen art students making life drawings in a city square, for example.
If wearing too little is illegal in most places, why wouldn't it be possible to wear too much, in theory?
You might be cool with that, but I’m not.
Some forms of dress and undress have laws. Others don’t have laws but are still bound by social contracts. The person you are replying to is clearing trying to compare these and pointing out responsibility may come down to the wearer in both cases and wondering out loud that society may punish any extreme state of dress.
At which point did he say you should pass new legislation or that he agreed with it?