Why don't you switch to CDs then? Something is telling me this isn't quite the full story.
I'm sure lots of people who don't really need to use Spotify use Spotify all the time, if you really do listen to just a few albums, why not buy those off Bandcamp/Beatport/Whatever then listen to those and stop paying Spotify? I'd easily switch away from Spotify if I no longer saw/agreed with the convenience, but hard to beat it for discovery right now.