I probably wore it more around school off my ears and around my neck as some sort of status symbol more than actually listening to music between classes.
Eventually my high school banned them when someone recorded test answers on a cassette and used their Walkman to cheat on tests.
The TV show "Square Pegs" captured the era somewhat accurately. The character played by Merritt Butrick was one of those wearing a Walkman around everywhere and sort of reflected the way it felt. To me anyway. YMMV. :)