True, but advising dozens of startups, some of which became highly successful, does. Especially when your investment money came from the sale of your own successful startup.
"Whim" and "arbitrary" have a connotation of carelessness; I don't think they fit here.
Still, there's truth in what you're saying: The large number of applications necessitates that they use heuristics and gut feelings to choose whom to interview. Their rejection letter admits that the process isn't perfect, and it would be unwise to give up on an idea on the basis of a YC rejection.