I don't doubt that you feel the difference between the two - but that doesn't mean there is a difference.
Without a double-blind test, it's just conjecture. Audio is notoriously subjective, and existing research shows that 320kbps audio is indistinguishable from lossless.