Yes there is - because they aren't the best of the best.
I don't see a reason to sugar-coat it. Being rejected means that YC doesn't think that you're one of the 50 or so best startups who applied. Nothing more, and nothing less.
None of these are true.
There is, frequently, a probabilistic scale of quality -- there are works or instances which will, generally, be considered "good" and some which are considered "bad", and, say in an iterated experiment model you'll generally come to at least some level of agreement as to what's what. Sometimes there are even very clear winners and losers.
But there's a hell of a lot of variance. There are things which people, often many people, like, often a lot, which others simply despise. I am, for example, not a fan of rap, hip-hop, house, and similar music. I really like Beethoven's late string quartets. I'm aware that by popular vote my preferences are likely unpopular.
And even where there is widespread agreement after the fact on something, experts in the field may not come to agreement. The Beatles were panned by several early music critics and record labels. Rob Malda's initial review of a new cellphone was less than whelmed. Oh yeah, it was called the "iPhone", you may have heard of it.
Tests for categories as seemingly refined as classical music performance -- both auditions of performers, and trials of classic vs. modern instruments, and fine wines, show that there are huge differences in non-blinded vs. blind trials (for music), and that wine experts are incredibly unreliable in how they grade, or even describe the qualities of, wines. Though this does make for an entertaining way to spend an afternoon or evening with friends.
http://www.sfgate.com/business/article/Blind-Auditions-Putti...
http://freakonomics.com/2010/12/16/freakonomics-radio-do-mor...
In another interesting study, unreleased music was introduced to several different "worlds" of test subjects -- within a world the experiences were shared, but not between them. The result was that, from the same starting selection, different "hits" were selected by each.
http://www.npr.org/2014/02/27/282939233/good-art-is-popular-...
And that's the whole point.
In any competitive selections process, especially where there are far more qualified applicants than there are positions, there will be many qualified applicants who aren't admitted. The reasons for denial will vary, but ultimately the selectors must adopt some arbitrary mechanism. Lottery would probably be the fairest, say, by assigning a rough percentile ranking to each candidate (perhaps by decile), selecting all within the top category (the clear winners), none below some cutoff, and a random selection within the grey range. More often culture, bias, hubris, or other factors will lead to other bases for decision.
But thinking you can, perfectly and without error, assess the quality of something as both nuanced and capricious as the possible value of a technology start-up concept based on a few brief exposures is really pretty arrogant. And all but certainly wrong.
I never said that they could assess it "perfectly and without error". I said that it means that "YC doesn't think that you're one of the 50 or so best startups".
YC can't unambiguously tell whether or not a particular rejected candidate should have been in the top 50 or not, at least not for a significant number of those rejects.
My point remains: quality or success cannot be foretold in advance. Particularly when much of that success depends on passing prior selection phases.
In evolutionary biology, as a comparative example, there's no question that luck plays a role in evolution. Your ancestors had the bad luck to live in an asteroid strike zone, on a tidal-wave flood plain, or within the affected region of an ancient pandemic? Pity, you're not here now. So the influence on survival exists.
The relevant part to evolution though is that luck is not an inheritable trait. It cannot be selected for. It's part of the random noise.
Similar logic holds true for YC prospects: some are successful, some are not. Passing the bar itself comprises in significant part a random "lucky" event. Yes, that can be influenced, but you cannot eliminate the error.
True, but that's sort of ex post facto justification.
Theoretically, ten startups could apply and objectively be equally qualified, but YC may still pick only one. The key is that without completely exhaustive criteria for admission (which would not be practical), it's impossible to determine what defines "best" and how to become one of the "best".
EDIT:
This [0] explains it nicely.
Put yourself in the position of someone selecting players for a national team. Suppose for the sake of simplicity that this is a game with no positions, and that you have to select 20 players. There will be a few stars who clearly should make the team, and many players who clearly shouldn't. The only place your judgement makes a difference is in the borderline cases. Suppose you screw up and underestimate the 20th best player, causing him not to make the team, and his place to be taken by the 21st best. You've still picked a good team. If the players have the usual distribution of ability, the 21st best player will be only slightly worse than the 20th best. Probably the difference between them will be less than the measurement error.
The 20th best player may feel he has been misjudged. But your goal here wasn't to provide a service estimating people's ability. It was to pick a team, and if the difference between the 20th and 21st best players is less than the measurement error, you've still done that optimally.
But for the majority of applicants, there is a good reason - because YC didn't think that you were one of the 50 or so best startups. This doesn't mean that being denied means that you're not smart or dedicated or that your startup is bad. It just means that they didn't think it was one of the 50 best they received.
All of the information you need is contained in that above statement. All of this advice is basically just saying, "interpret that above statement literally, and don't make any false inferences from it".