At all of these schools, while the tuition price tag might be $50,000 per year or more, relatively few families actually have to pay this sticker price. At my alma mater, Bowdoin, the overall cost is around $54k, but the average financial aid package (according to the College Board) is $41k, leaving the average Bowdoin student on the hook for only $13k a year.
If you're a family that otherwise would qualify for a large amount of financial aid, but you've saved up into a trust or other savings plan for the purpose of funding your child's education, you'll lose this free money. The institution will count this amount against your demonstrated financial need.
Some families with means are able to hide away these funds, and still get significant financial aid from a school like Bowdoin, but most families don't have the resources or savvy to do so.
Of course, if we're talking about institutions that don't have such strong financial aid programs – which is the vast majority – you probably want that savings. Deciding to save for your child's education is, statistically, usually a good decision. If your child ends up going to one of the above, though, it can be a costly one.