Anything belongs to the front page by virtue of getting to it. You can argue that the community is broken because it upvotes stuff you don't want to see, but that's a battle you just can't win.
Basically, you're suggesting that in addition to there being a button for "I find this interesting!", there should be another button for "No, you fools! No!".
I'm not sure I buy that argument. A simple counter-example is duplicate submissions. Yes that story may be front-page news, but that doesn't mean that each submission about it should be. There are probably some other trivial examples too, and personally I think that low-quality news is low-quality regardless of how many people vote for it.
I have that button, it's called "flag," and that's exactly what it's for: Posts that are popular but toxic. If a post was unpopular, you wouldn't need to flag it.