Doing a bit of research, it looks like a majority - ~60% across studies - prefer the better quality in double blind tests at 128. At 256/320 it drops off dramatically, yes.

I know I personally have sought out better copies of some songs because the high end sounds awful, witch each hi-hat splash being a crinkly-codec failure.