Having gone to a school with an extremely high barrier of entry (in the terms of admission tests not financially) I would say that bright & motivated companionship was what pushed me forward. Pooling talent helps to develop it in many ways.
At the same time some of the best/most braised education systems come from countries with a "level playing field" public school systems such as Finland.
To me the main question is if developing talent by pooling it together is in conflict with achieving high average student development. And if so, which one of those should be the goal as a society?