When I listen to music, I want to slow down and relax, and taking a record out of the sleeve is just so much nicer than fidgeting with CDs. I like that everything about playing records is analog. I like the crackle between songs when playing an old record, it's so much nicer than the random skipping when playing a scratched CD. When something breaks in my analog audio system, I can usually fix it myself with a soldering iron.
As for the sound, I think it's mostly because of a different style of mixing than the media itself.