Yeah, sometimes I like to just toss a album in a playlist and listen to it, and I always seem to end up skipping at least one song

Then un-tick it.

Or rate it poorly and factor ratings into your smart playlist.