Any volume which makes your music become my music too without my consent is at asshole level.
(Sure, I just called someone random on the web an asshole. I don’t mean it with any force. In London we get people riding busses playing their im-personal stereos loudly, sometimes. I often don’t like it either. I often use headphones for my own sounds but not the blocking kind, and will have to stop my music because of thwirs. One time someone got into the Tube/metro carriage I was in playing loud Brazilian music from a speaker on a trolley. At first it annoyed me, but after a few bars it got me grooving. Then I realised it was a funk-infused cover of a traditional capoeira song, so I steuck up a conversation with the other rider about Brazil and capoeira. Made my day.)
For me, I don't want to live in a cacophony of noises 24/7. That goes for music, non-stop ambulances, loud speakers, etc.
I tried and decided that those places are not for me, so I moved back to smaller and much quieter places (I very much prefer the sound of rivers, insects and wild birds to other people's sounds.)
It might make me an asshole but it's also quite natural to be drawn to peace, so there that.
However, if you are hard of hearing to the point where you are actually disturbing others, I would recommend headphones.
If we cared more for the hard of hearing we would reduce music volumes and make restaurants quieter. Our society doesn't care even though it pretends to.