There are very few people actively trying to prevent lower class people from getting into the upper class, but it's competitive to be in the upper class and there are only so many spots. You don't get to be prestigious by letting everyone in.
The plus side is that if you are from a lower class background and somehow overcome your environment, the admissions committees are tripping over themselves to let you in.