Open letter to the Yale Community from Dean
yalecollege.yale.edu
yalecollege.yale.edu
I'm not sure what the policy implication is - there's no easy rule here. But it's worthwhile to keep in mind that "information wants to be free, Yale can't decide how it's used" (an attitude that's a bit of a straw-man, but not too far from what some have argued here) ignores the reality that the next time the Yale faculty debate whether or not to collect and collate evaluations (or whether to have a numeric score component), they may well decide not to. (Which, of course, is a principle that applies to other cases of data collection - sometimes it's best not to collect it, just because you don't know how it'll be [mis-]used...)
But... given an adversary's policy of "we will provide this data if and only if it goes unused", there's no reason to give that policy any weight in deciding whether to use the data. If you decide to use it, the policy will kick in and you can't. If you decide not to use it, it's not relevant that it remains technically "available" for other people to also not use. That's not a reason to consider what Yale wants, it's a reason to slam them for being weasels.
When I taught a college course several years ago, it was conventional wisdom that student evaluations were primarily driven by grades, thus encouraging grade inflation. A popular website called "rate my professors" included ratings for "easyness" and "hotness."
Female teachers were advised to anticipate receiving one or two rape threats in each batch of anonymous comments.
http://www.ratemyprofessors.com/SelectTeacher.jsp?the_dept=A...
The advantage of having the data come from Yale is that they can mandate that every faculty member must take class time to conduct the student evaluation. This ensures that [nearly] every student's experience in every class will be considered in the final results, which gives much more legitimacy to the results.
An example where this has happened: Some (many?) U.S. public libraries have stopped storing historic checkout data for patrons, in order to avoid having to turn the data over to a govt. agency. I recall being able to get a list of every book I had ever checked out. Not any more.
Yale can decide how the information is used. But if someone within the Yale community comes up with a Yale-beneficial use for the information, Yale should embrace that use.
I don't think this was a free-speech issue, so much as a missing-the-boat issue. Yale should be nurturing its hackers. Yale has an excellent CS program (I say this as a Yale alumnus), but in recent decades, it's not been much of a countercultural or hackerish place to be. Yale is top-notch at producing lawyers and investment bankers. It needs to get better at producing hackers. (By many recent accounts, it's starting to get considerably better. Maybe this episode will inspire a rethinking of sorts.)
To collect this data, Yale hands out surveys and students generate the content.
Yale needs the students to gather this data but the students do not need Yale. They could just as easily aggregate this data on an external website. It seems that they even have the ability to auth against Yale servers.
This content is generated by students and is supposed to be used to improve the University for the students. If Yale doesnt trust its own students with their own data it opens up the question of what the relationship between Yale and its students is and whether Yale is the kind of place for kids who understand how technology is shaping democracy, privacy, and our culture in general.
We've complained about this bitterly going the other way. How many app developers are frustrated when they get a one-star review that says, "This app couldn't sync to my online account unless I was connected to the internet. Changing to five stars when that's fixed"? How many Amazon shoppers have seen one-star reviews that say, "The package was damaged during shipping. The return was really easy and I got a replacement in two days, but I wanted it for an event that happened before the replacement came."?
This doesn't come close to being true in any sense. Wanting to do something and actually doing it are barely even related concepts.
The situation is the same here, at my school when viewing aggregate evaluation data, I find it to be VERY accurate to the actual experience I've had in those classes, and I think that's the issue here. Yale is protecting teachers from being called out for being bad teachers.
Regardless of subject matter, I learned the most in classes that had good teachers who received good evals.
However, the administration doesn't think that way; they think that it's more valuable to take the "right" course than to take a course with the right teacher.
In other words, if reduction of complex performance to a scalar value is invalid, then the entire grading system that defines academic success is bogus.
Incidentally, I think that likely.
Doing so at a college scale would be too costly. But you do have all the grades that go into your average, comments on each essay, office hour time, etcetera. The nuances of your performance don't matter in your GPA, that is, your Grade Point Average, but you do have a bit more information available to you, yourself.
I think she doesn't really know what free expression means. It does not mean, "you can say what you want as long as we agree with it." That's certainly how I read "more importantly, they encouraged students to select courses on the basis of incomplete information". Free expression means that the administration cannot control the message; people can say what they want. This is OK; win in the marketplace of ideas ("maybe I should evaluate my courses with more contextual information"), not by treating your students as children who can't decide for themselves how to consume information.
Also, this bit about reconsidering whether to provide the data at all is childish. "You're not interpreting our data the way we want so now you can't have any! I'm taking my ball and going home!" It's worth remembering (both here and in the thus-far imprecise and confused discussion of who "owns" the data) that the students are the ones providing the evaluations being aggregated here; at the very least, it's unclear why they'd be expected to continue doing so without getting some quid pro quo benefit from it.
Stop trying to control everything and trust your students to be smart and thoughtful.
Besides which, I think exposing rating systems for courses creates perverse incentives. It's been well studied that students who get better grades rate courses higher. Is that really what we want to be measuring professors against?
OK, but now you're arguing that Yale students shouldn't be exposed to free expression because they're incapable of handling its consequences. I think that's silly, but fine: it's a real argument. But let's stop pretending this kind of paternalism is consistent with even a modest notion of free expression, because it's very clearly not. It says "you can only say things we approve of, because we know best." Being right about knowing best doesn't make it not a limitation on free expression. Free expression as a principle necessarily doesn't hinge on whether you think the individual expressions are right. That's, like, the whole point of it. [1]
You're also arguing that 1-5 rating systems are silly and Yale shouldn't bother collecting them, but that has nothing to do with the YBB+ website or how students will interpret it. For example, wouldn't those perverse incentives you mention be in place already, given that providing feedback to professors was the whole point? Ratings don't magically transform from useful evaluation tools to misguided oversimplifications just because students have convenient access to them. Either they carry useful information or they don't.
[1] I'm not taking a free expression absolutist position here. (I'm tempted to, but it's totally unnecessary for this discussion.) The point is that to the degree to which you make rules about what kind of thing people can express is, well, trivially the degree to which you limit free expression. And limiting the ability of students to transmit truthful data about the courses they're taking is a very strong limitation, and it would be hard to formulate a consistent, principled policy that permits this kind of restriction and doesn't just say, "you can't say things we don't like".
Odd, I read jsun's comment to be claiming that the professors were incapable of handling the consequences, not the students.
I don't really have a bone in this fight, so to speak, but I do find it interesting that the Dean of Yale has such a problem with the way their students are using their data to express an idea, when their very own Freedom of Expression Policy says:
Yale's commitment to freedom of expression means that when you agree to matriculate, you join a community where "the provocative, the disturbing, and the unorthodox" must be tolerated. When you encounter people who think differently than you do, you will be expected to honor their free expression, even when what they have to say seems wrong or offensive to you.
Although, if we're being pedantic, their policy technically states that they expect their students and graduates to honor others' free expression, not that they agree to honor their students' free expression.
For example in Sweden we have laws that provide "freedom of speech".
It means that one can ALMOST say what one wants to anyone. One may however not say/or behave in a way that is intended to offend a group of people. Saying such a thing leads either to Jail or a Fine.
The law, of course, is not in play here, because Yale is a private institution not wielding government authority. But the degree to which it supports free expression is exactly the degree to which it lets people say whatever they want.
Perhaps I'm missing something, but I do not see an apology behind that link. A detail of Yale's reasoning, but not an apology.
> However, I disagree that Yale violated its policies on free expression in this situation.
I'm not sure what "[Yale's] policies on free expression" are, but my understanding here is that Yale blocked the IP address of a site because it was using some data that Yale published, and modifying it in a way Yale disagreed with. There might be an argument to be had under copyright, but I don't think it's important at the moment. Yale states this:
> the developers violated Yale’s appropriate use policy by taking and modifying data without permission, but, more importantly, they encouraged students to select courses on the basis of incomplete information.
Abusing the fact that you have power over the students' access of information to prevent access to information you disagree with is what people have a problem with; this is especially true when you label something like this "malicious activity". These concerns are not addressed by the dean's letter.
I would guess that they are referring to Yale's Freedom of Expression Policy: http://yalecollege.yale.edu/content/freedom-expression
EDIT: While I'm at it, here's what I came up with when I googled "Yale appropriate use policy":
It's the typical shoot first ask questions later philosophy that intelligent well educated teachers should never have used.
If they would have addressed their concerns from the beginning to the students i'm certain they would have complied.
"In retrospect, I agree that we could have been more patient in asking the developers to take down _information they had appropriated without permission_, before taking the actions that we did."
The entire note has an odd, arrogant read to it. If the new tool in fact provided complete comments as well as numeric scores, Yale would have cheered and saluted them ... but because they didn't, Yale shuts it down? Is that really the issue?
Universities tend to move very slowly, and I get that these situations can be hard for them, but the whole letter is disingenuous, at best. AFAIK, the students aren't guilty of "modifying data without permission", nor did they "encourage[d] students to select courses on the basis of incomplete information." That's like saying no one should use a course catalog because it doesn't have complete information. Of course scalar averages don't capture the entirety of the feedback on courses. But I bet there's a meaningful difference between 4 stars and 2 stars, or whatever the rating system is, and I'd like to think that Yale students are clever enough o understand this, and to be able to go through more detailed evaluations when it makes sense to them.
If the Dean had just said "holy crap, we didn't realize how easy it is to do this and we're freaked out about it, we need you to close this down for now, but please help us build the next generation system", people would have a very different reaction.
In other words, death by committee.
Maybe Yale is indeed in the right to try to control use of this data. But Yale most certainly is not in the right to unilaterally block access, regardless of the reasoning.
(NB., my concern is not with legal rights, but rather a higher standard of moral or ethical rights that Yale, as a purportedly high-minded institution, should hold itself to.)
Welcome to the 21st century. That single minor incident now escalated into a news item of several days. Lot of lost PR points. All because they feel the need to control the course selection process. Sad.
It's almost as if the entire purpose of their institution was controlling the course selection process. You know. By offering courses in the first place.
Just this weekend, we learned of a tool that replicates YBB+'s efforts without violating Yale’s appropriate use policy, and that leapfrogs over the hardest questions before us.
http://yalecollege.yale.edu/content/freedom-expression
And here is their Appropriate Use Policy:
http://its.yale.edu/forms-policies/appropriate-use
The most interesting thing to me about these policies is that the Freedom of Expression Policy explicitly states how crucial it is to be tolerant and accepting of people's ability to express ideas that are "provocative, the disturbing, and the unorthodox," and that this means they value the ability to "think the unthinkable, discuss the unmentionable, and challenge the unchallengeable." And yet, they are having a rough time because some students used their data to express an idea in a way they hadn't intended or thought was possible.
At the same time, I actually couldn't find a single clause in their Appropriate Use Policy that had been violated. The only restrictions I saw in the policy were that their data be used only for non-commercial and academic purposes (which it was). I didn't see anything about only being able to use data in its intended manner. But maybe I'm missing something (for example, maybe the actual YBB site has its own Appropriate Use Policy separate from their IT policy).
EDIT: I also just realized I was looking at the Quick Reference for their Appropriate Use Policy, so maybe there's something else in the full text that was violated:
http://policy.yale.edu/policy/1607-information-technology-ap...
* "we need to review our policies and practices" -- We're going to make it policy that these ratings are gone.
* "We will also state more clearly the requirement/expectation for student software developers" -- The requirements will become more burdensome and bureaucratic, and we will be able to better cover our asses.
* "and we will create an easy means for them to do so" -- (I don't buy this one.)
Yale is going to end up on the wrong side of history in this one. And the irony is that the dean is a professor of history.
Mary Miller, Dean of Yale College. http://yalecollege.yale.edu/content/officedetail/454
Can it be both?
Coming from a country where higher-education is covered by the government, I think it's a conflict of interest for an educational institute to be in the money-making game
I wish my university even had all those reviews we do at the end of the year available for all to see. They don't do sh*t with it. I once wasted my time by suggesting my university create an API for students to easily get any data about the school. B/c then we would all see how 'great' some of our professors are and what we each think about them.
Sure 3rd party website like 'ratemyprofessor' are good, but they don't have the ease of acquiring rating data like schools do that pass out reviews to each student at the end of a semester. They have to protect the 'wizard of oz' lest we all see who really is behind the curtain.
a post facto weak-sauce apology is about what I expected here. I get the distinct impression that what the Dean regrets most is that the story was widely publicized, rather than that her initial reaction was "heavy handed".
If they had done all the above, then I don't know what the fuss is all about.
Because Yale has options too - if a few students decide to make a principled stand, there are others waiting in line just as good who care slightly less.
This is, of course, crazy.
From simply a UI/design position, you can't easily display/sort on X different professors/courses each of which has 1..N detailed reviews that are blobs of text.
From a broader perspective sometimes you need to decouple data to be able to better process it. In fact, given so many students used YBB+ it is reasonable to conclude that YBB+'s presentation of the data is doing a job informing students than Yale's own systems.
Everything is a remix
[1] From the article: "The tool created by YBB+ set aside the richer body of information available on the Yale website, including student comments, and focused on simple numerical ratings. In doing so, the developers violated Yale’s appropriate use policy by taking and modifying data without permission, but, more importantly, they encouraged students to select courses on the basis of incomplete information."
mort importantly the ybb reviews are basically useless from a reality perspective due to the way the data is collected (gateway for letter grades) -- they never told you anything important no matter their format, and never will.
if you want to know what a professor is like you just ask around or drop in on the class during shopping period. ybb+ is not an improvement on that process.
digital equivalent of 'dropping in': coursera or open yale. digital equivalent of 'asking around' ... ? not ybb+
personally i think it would make sense to get each professor to record a video introducing themselves and the material. then skip that awkward first lecture. actual teaching video would be tops...
There are always hard, scary courses, and those classes are going to end up ranked lower because the material is harder or the professor isn't as "cool" or because a 20-year-old doesn't see the point in learning how to do proofs. Letting that become the dominant factor in course selection is not good for education (or intellectual freedom--what happens when a professor with unpopular views is ranked poorly and his/her classes become sparsely attended?)
> Just this weekend, we learned of a tool that replicates YBB+'s efforts without violating Yale’s appropriate use policy, and that leapfrogs over the hardest questions before us.
Not a huge victory for the students here. :/