FWIW, it is dangerous to wear headphones in the city and listen to music, but you can always wear only one side. It is not comfortable, but that is how you remain safe without being an ass.
Let's say I'm out in the woods, being non social. And someone comes up the path, playing music loudly. Now I'm being annoyed by people again, which is what I was trying to avoid by being out in the woods. And they're usually on a motorized vehicle, even though motorized vehicles are prohibited on the path.
I'm not trying to tell people how to live their lives. If they want to apprechiate nature in silence, cool. If they want to listen to music, cool ... but it'd be nice if they used headphones and it would be acceptable if they had a speaker at reasonable volume, but when I can hear them before I can see them, it's really not cool.
If they want to walk with a friend and chat, that's ok too.
To keep with the example below, walking in the woods alone is "not social" but it's not at all "anti-social." Listening to music with headphones while walking through the woods is not anti-social. Blasting music on a portable speaker, not caring whether or not you're disturbing other people on that trail, is a pretty great example of anti-social behavior. As is having a speaker phone conversation on mass transit, or being visibly under the influence of drugs in public, or choosing to park illegally and block someone in because you'll "just be a minute" or any number of other things people do because it makes their lives 5% easier at the risk of making someone else's 10% harder.