That's exactly how a useful social information system works. I choose what I want to follow and see, and there's no gap between what moderation thinks and what I think. Spam gets dealt with the moment I see something spammy (or just about any kind of thing I don't want to see).
This is how Usenet worked: you subscribed to the groups you found interesting and where participants were of sufficient quality. And you further could block individuals whose posts you didn't want to see.
This is how IRC worked: you joined channels that you deemed worth joining. And you could further ignore individuals that you didn't like.
That is how the whole original internet actually worked: you were reading pages and using services that you felt were worth your time.
Ultimately, that's how human relationships work. You hang out with friends you like and who are worth your time, and you ignore people who you don't want to spend your time with, especially assholes.