The problem is the spectacular arrogance of the professors and their contempt for students. Anyone who gets to the level of ivy-league professorhood has reached the top of their profession, and has acquired an ego to match.
I'm generalizing here, of course, not all of my professors were full of themselves, but large numbers of them were, and more to the point there was a culture of disregard for the opinions of students. Over the years the University has sought to control, limit or suppress student ratings of courses, and professors were never penalized for getting a bad rating. They didn't care about the opinions of students because they didn't have to.
It's a lot like government. A person in this kind of organization does not advance by serving customers well. In fact, they often bristle at the idea that students, or constituents, are customers at all. Rather, they are sheep to be led or punished as appropriate.
Most of my friends at Princeton were considerably smarter than the professors who taught us, but we were young, lacked knowledge, depended on the good graces of professors to get the grades we needed for our future, and so lacked the tools to fight back against the substandard product we were being fed.
This, plus public conversations about courses or professors tended to be dominated by the few but prominent students who fawned, obsequiously like the courtiers, over the brilliant professors who filled them with inspiration and hope. We all threw up a little bit in our mouths when we heard this sort of embarrassment, but what could you say?
Again and again, I encountered professors who would assign readings that were unreadable, would promote their own work over better work by others, drone on about drivel, advance idiot theories, and treat the slightest challenge as an affront.
Nominally, Princeton isn't like that. Unique among universities, it has something called a "precept", which is a small group meeting once a week, led by a grad student or sometimes a professor, to debate and discuss the issues. It's a nice theory, but most students say little in precept, aside from a few suck-ups. We were so overwhelmed by the firehose of un-curated content gushed at us that virtually no one had done all the reading, and risked being exposed as the one who didn't know what he was talking about.
Things won't change until professors start losing their jobs over bad ratings.
All that having been said, my physics courses were pretty ok.