I met a ton of students at UOP who hadn't been as lucky as I had in going to a school like Yale (FYI, I was an athletic recruit so felt really grateful to be there). Employers literally ignored their resumes and would never get back to them. I was shocked because I would talk to some of these students and thought they were talented, ambitious, hard-working, etc. The thing that makes me passionate about HireArt is trying to overcome the obsession with credentials (which are very highly correlated with socio-economics) and moving toward a more meritocratic way of hiring.
Can I say that I am totally rid of the drive towards prestige? No. I fall into that trap. It's a hard trap not to fall into given how much society values brand names. Anyway, you pointed out a tension that I actually struggle with a lot. A recent article about the "college bubble" recently called for all Ivy-League graduates to 'renounce' their degrees to stop the bubble from continuing. It's hard in practice but the point of the article is that unless the people with Ivy-League degrees stop caring so much about them, the status quo cannot change.