Why are they brutal when you have your notes, books, the web, etc. available? They're usually not based on spitting back answers you were given or can look up. You actually need to use what you've learned to produce something novel or, at least, very esoteric. e.g. I've had exams where I was asked to look up a paper and reproduce the unpublished proof to get from equation (x) to equation (y). Such proofs are often not published because the authors are freaking geniuses who view the derivation as obvious, and getting to the point where you can do the same proof with intense effort is one point of the course! These exams are often fiendishly difficult and/or open-ended, so you will be working like a crazy fool from the moment you get the exam until it must be turned in. This sometimes means pulling an all-nighter (or more). (I despise all-nighters as the last resort of poor planners, and hate being forced into them.) Time is such a constraint that, if you didn't study and absorb all the course material thoroughly prior to the exam, you will likely not be able to get much done in the time allowed. Frequently nobody in the class will actually finish, and it's really just a matter of how much you can do relative to your peers, and how well.
Collaboration during preparation was never forbidden in my field, although I hear other disciplines (especially humanities) can be quite different. Collaboration during the test is usually discouraged, but obviously students are on the honor system. If any actual copying is detectable from what is handed in you can bet it was egregiously bad. If Harvard was able to make charges stick to 70 students, you can safely bet that twice that number cheated their hineys off, but many were just a bit smarter about it.
Seriously, think about it, it just cannot make sense. Nobel price winners come from all countries. A university does not need to belong to the U.S. Ivy League to have smart alumni.
However... The Ivy League is really expensive and well-known. In fact people like George W. Bush studied there. It is almost like a self-amplifying loop.
The only stipulations for the exam was no collaboration and show all work.
University should not be about a memory test. Not memorizing all the formulas, etc but building on knowledge, doing something you would be expected to do 'in real life', and of course with all the resources you have.
Unfortunately it seems this kind of work is more appropriate to math/engineering/science type of problems.
Most of work in other areas is bibliographical research.
The reality is, there's a few "twists" (non-intuitive steps) in every proof. If you know the "twists" it cuts the time down to 1/10th. If you're working in a group, there's a clear advantage, but no way to prove anyone cheated (unless students are dumb enough to copy it out line by line).
If students, act according to an honor code, then professors can get on with the work of teaching.
If people have to spend all their time fighting cheating, which is a non trivial task, it creates a very different environment. I would go so far as saying that it ends up obviating the entire point and purpose of college.
----
This is what colleges were originally built around: Students came because they wanted to learn X subject, and the best place to do that was with the professors who were essentially THE experts in their field.
You want libraries and data sources bar none? Universities had them. Want to deal with and learn from like minded students and teachers? Few other places you could be to get that.
People assumed that you wouldn't cheat on your test because that would be like cheating on a MIT open courseware test.
Now things are different - for a lot of people, colleges have become "The next step" to get a job. People assume that the correlation now goes the other way - college MAKES you better at getting jobs.
As a result your population is also now filled with a lot more students who just don't care. The degree is your only way to get a foot into the door of a stupid system.
To steal a phrase from finance, college must be decoupled from "financial success."
Note: A better insight is avaialble from Panos Ipeirotis' experience dealing with cheating in his class a year or so ago. IT was discussed extensively on HN and the blog post, now pulled, was called "why I will never pursue Cheating again". If you dig around a bit though you should be able to find a copy.
I never ever wanted a take home final like that again.
Where take home exams are used, a) they're usually harder than proctored exams, and b) they're usually open-book, so you're not expected to "write it just off the top of [your] head".
The rule is you have to work alone, can't work with another student. That's the rule that these students broke, apparently.