So, are you arguing against elite colleges being worthwhile, or in favor of them? Are we not supposed to use every opportunity we get? That's cute ;)
The problem with both situations (VC/entrepreneurs and employers/employees) is that there's a cost associated with evaluating a candidate.
I don't think anyone doubts that good opportunities (and candidates) exist at all at lower tier schools, but it seems very likely that there are fewer of them.
If your resources are already constrained by something other than the number of opportunities you can afford to fund/hire (e.g. time to evaluate/interview), then it doesn't make sense to expand your search if it means lowering your success rate.