Anecdote: at my wife's Ivy medical school, there are several events each semester in which students are encouraged to drop several thousand dollars in a single 4 hour event. One example is a charity auction in which faculty auction off experiences like a boat trip or tours of their wine cellar to students. Some of these items go for 5 figure dollar amounts. Events like these specifically exclude poor students or students with families by design. While attendance isn't mandatory, the events are planned and executed by the student body via assignment or election, so it is difficult to not be included in some form or another. I can understand how, after attending for a few semesters, one would feel that wealth defines the culture there and that being poor excludes you from that culture.
Edit: it's also important that while all ages and socioeconomic statues attend the Ivys, the vast majority of undergraduates are 18-22. As someone 10 years older and who has an adopted daughter that age, what you suggest isn't advice that will be widely adopted by people of that age group.