This is a good point.
You know Sturgeon's Law? Something along the lines of "90% of science fiction is crap, because 90% of everything is crap?"
Well, if you more or less agree with that, there is just so much more of everything that there is so much more crap.
Another commenter expressed this as a signal-to-noise ratio problem. It is that, but it is also that O(N) is bad news for very large values of N.