>The difference was only evident in the very high frequencies, and I really had to focus to tell.
This becomes a curse if you practice it too often. Yes, if you strain your ears during cymbal crashes, you'll hear warbly bits in compressed audio, especially if that's all you're looking for. And then you'll start hearing it everywhere. In terms of casual playback, it's not terribly important, and if you don't tune your ears, it won't matter. (Lossless is important for archival copies though, IMO)
My favorite/worst example of falling into this trap occurred for me when I was mixing an EP by my band. We had terrible recording equipment, so I was trying my 22-year-old best to carve something workable out with EQ and panning and whatnot. On the kick drum, I wanted the 'click' of the beater hitting the head, but it accentuated this squeak that my pedal made every time I hit it. I spent a little too long trying to cut this squeak out, to little avail. It was like an icepick in my ears, but no one else in the band could hear it.
Later that day, I got in the car and put in a CD by the band Cake (this being the early 2000s) - one that I'd heard countless times. On this playthrough though, clear as day, I heard the squeak of the drummer's kick pedal.
I switched to pure electronic music for weeks just to cleanse my palette.