In my experience it's more that more prestigious schools have a very talented student body, who are not at all used to failure. If something gets hard for the first time in their life, more of them are likely to say 'well that's not my thing' and switch, rather than say 'I guess I'm not a genius I'll have to work hard'. Calc III is Calc III -- you can look up the video lectures at MIT and see for yourself. The books, topics, pace, are mostly pretty standard at most universities, with some exceptions.
Also bigger, more prestigious universities seem to care less about their students and more about their research: see: TAs who are incapable of teaching, can't speak English, etc. When you're taking a difficult subject and lecture is a waste of time because your teacher can't communicate it's no surprise people drop out. They didn't sign up to self-study by themselves for a semester and take a final.