Facebook and OkCupid Broke the Law When They Experimented on Users
medium.com
medium.com
Even better, the entire issue can be avoided by placing the decision in the customer's hands. Let them decide what kind of results they want. You can provide all the tools you want when it is the customer that is voluntarily choosing to use them. You might even learn something interesting from that choice, too.
/* now, to the thread in general, and not specifically the above comment /
What so many here seem to be missing is that this is about respecting the customer and not trying to go behind their back. They chose to use a service for some reason(s), and suddenly changing that without notifying them or asking for their input* is at a minimum highly unethical (and plain rude). As this article points out, sometime it is also illegal. Depending on the laws, contracts, and/or monetary transactions involved (if any) this kind of unilateral modification to the relationship may conflict with contract law or various business laws as well.
As some may people seem to have not read the article, a key quote:
"The central pillar of modern research ethics is that in most cases researchers don’t get to decide for themselves whether an experiment is worth it. Unless the risks are minimal or nonexistent, that decision belongs to the participants, not to the researchers. That’s what the Common Rule does: it systematically takes these decisions out of researchers’ hands, and gives them to participants and IRBs."
Yes, whenever humans are involved, the decision to modify ("experiment") on humans in any way cannot be made by the experimenters. Which bring me to the thing that I find most damning about this whole situation: the Facebook[1] and OkCupid experiments was fairly trivial. In in my not-an-IRB opinion, there is little risk to any of the participants; it's trivial compared to experiments like drug trials or other medical research. If they asked an IRB first, they could have gotten their approval quite easily. If, due to the specifics of the experiment, proper Informed Consent would have made the experiment useless, the IRB could have easily waived some of those requirements.
So no, various testing is not "illegal". You can try new things and see how it affects your customers. We have simply decided as a society (and try and ensure with various federal and state laws) that if it involves humans, you need a neutral 3rd party to make that decision. Yes, this slows[2] you down a bit, but again approval for an experiment like those done by Facebook and OkCupid should be fast and easy. Be glad you're not doing medical research with nasty risk.
Traditionally, if there is a question as to if approval is needed you should assume you do need it. If you are going behind the customers back to change something from the expectations[3] they initially signed up for without their approval, you absolutely need IRB approval.
[1] As I understand it, it may be Cornell that was the initially at fault for not getting IRB approval, so they deserve some of the blame in that case. Regardless, as the researchers Facebook has a duty (and sometimes, a legal requirement) to make sure that approval was done properly before they proceeded with the experiment. As an analogy, consider that before you do any penetration testing it is important make damn sure you have all your contracts and approvals properly signed. You don't rely on "but the CEO said so" or "We just tried some common passwords and did a few nmap scans - they shouldn't even notice", you get explicit approval first.
[2] If getting IRB approval too slow for the tech industry, they could try working with existing IRBs to try and come up with some new way of fast-tracking some types of experiments or they could create some sort of new mechanism that better fits the types of experiments would like to do regularly. The important part is that a 3rd party - not just the researchers - has vetted the experiment. The methods to get that approval can probably be modified.
[3] While it's not related at all to the topic of human testing and the IRB, HN readers my find the talk given by Joe Peacock at NOTACON 8 about how they (fark.com) screwed up their website redesign in the infamous "you'll get over it" incident. The point of the talk is about the dangers of violating the expectations of your users. Not only is it an ethical/legal issue in some cases, the attitude where the customer was not even involved in the change (even trivial) can cause them to leave you surprisingly fast. Simply asking or otherwise involving you users is the only real way to keep them in the long run. (it's fark, so mild NSFW language warning) https://www.youtube.com/watch?v=YnVeysllPDI
You could probably argue that legalese pushes the boundary of "understanding scope and intent", but included in every TOS is a contact link for a user to learn more and ask questions so I'd be surprised if there was legal footing for that complaint.
The ethical nature of what they did is certainly in question, but I just don't see how one could declare either action "illegal".
EDIT: s/signs/signed/
Two corporations that sit down and hash out a mutual contract have much less restrictions though.
(b) Unless otherwise required by department or agency heads, research activities in which the only involvement of human subjects will be in one or more of the following categories are exempt from this policy:
(6) Taste and food quality evaluation and consumer acceptance studies, (i) if wholesome foods without additives are consumed
Facebook and OKCupid are, at least not at the moment, part of the federal government.
> "Research subject to regulation," and similar terms are intended to encompass those research activities for which a federal department or agency has specific responsibility for regulating as a research activity.
Furthermore, the definition of "minimal risk" is given in part (i) of the same section,
> the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine...psychological examinations or tests.
The argument against Facebook and OKCupid are pretty slim here... How could you experience significantly more discomfort in daily life if OKCupid slightly changed their rating system? Any reasonable interpretation of this risk is negligible.
[1] http://mlis.state.md.us/2002rs/bills/hb/hb0917t.pdf [2] http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf
No, it says it incorporates the rules in the federal regulation, but applies them to all human research, expressly excluding all the other limitations in the federal rules that limit the applicability more narrowly.
(No additional citation, because your own citations clearly establish this, contrary to your description.)
There's "research" and there's "research subject to regulation." The federal law says that all research by or directed by federal departments is subject to this law. It also says that everyone, including you and I, who are doing research subject to regulation (which includes drug development) are also subject to the law.
That is, even without MD 917, all research in MD which is subject to regulation would have been subject to the law.
However, this leads to a gap. Anyone doing research, but not research that is subject to regulation, and not research which is funded or directed by a federal department or agency, is not subject to these ethical requirements.
The MD law (see http://www.oag.state.md.us/Healthpol/hb917letter.pdf) is meant to close the gap, and include research which is not subject to regulation by a federal department.
The "minimal risk" section you pointed out is for expedited review. That is, the research still needs an IRB, but the IRB can use the expedited review process instead.
OkCupid didn't have an IRB, so the minimal risk section doesn't apply.
This is a deeply offensive comparison. Reading the complaint letter makes me wretch.
For awfulness: 5/5
That definition is so vague that practically any sort of interaction with a person in which any information about the interaction is recorded and analyzed can be construed as research. If a Girl Scout troop makes a graph of which neighborhoods buy the most cookies ... boom, that's research on human subjects.
I'd be interested to know whether any entity has been prosecuted for violating HB 917. I can't imagine that the extreme vagueness of the definition would hold up against a determined challenge.
What puzzles me is why James Grimmelman, J.D., and Leslie Meltzer Henry, J.D., think that the ultra-vague definition of "research" is legally tenable. Like, can't they anticipate the objections to the definition -- that it effectively bans all sorts of everyday actions that no reasonable person wants to subject to these sort of legal hurdles?
OKCupid's blog is super interesting but they might have to discontinue it if it is going to generate legal challenges like this one. The only reason we know about their A/B testing is because they openly told us about it, which if they get prosecuted for it might be a mistake.
I kind of liken it to the "never apologise" rule many companies now have. It is when common sense/public interest run into cold legal restrictions.
I don't buy this, as long as all representations made to customers about the service are met, it seems perfectly reasonable to tweak the already unknown algorithms behind the scenes for whatever reason.
Exactly this. The thing that some people seem to be missing is that there is a (very) good chance nothing is banned. It's just that you have to actually ask the customer. And IRB could waive a lot of the usual requirements (that are primarily intended for experiments with a LOT more risk).
The Common Rule, in these cases, is primarily about enforcing that stuff like a boilerplate EULA does not count as properly informing the customer and getting their approval. When humans are involved you have to get explicit approval each time as a precaution, which isn't actually that hard.
(a) Compliance with federal regulations. -- A person may not conduct research using a human subject unless the person conducts the research in accordance with the federal regulations on the protection of human subjects.
(b) Scope to include all research. -- Notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research, subsection (a) of this section applies to all research using a human subject.
My problem is, I'm not sure what MD 917 (b) means. I'll start with §46.102 (a) (1) at http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#... :
> (a) (1) Research that is conducted or supported by a federal department or agency, whether or not it is regulated as defined in §46.102, must comply with all sections of this policy.
Is MD 917 (b) supposed to mean that §46.102 (a) (1) applies to all human research in Maryland? I believe that's the case, because §46.102 (a) (2) only applies to those organizations which are regulated by a federal department, so would already be covered under the law.
Section §46.102 (b) has a list of human research which is exempt. The most relevant for Facebook and OkCupid appears to be:
> (2) Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless: (i) information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and (ii) any disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation. [Note: does not apply to children.]
If A/B testing falls under 'educational tests' or 'survey procedures' then it's exempt.
Otherwise - and this is another tricky part - §46.102 (c) says "Department or agency heads retain final judgment as to whether a particular activity is covered by this policy."
Who is the department head for Facebook and OkCupid? §46.102 (a) defines it as:
> (a) Department or agency head means the head of any federal department or agency and any other officer or employee of any department or agency to whom authority has been delegated.
MD 917 doesn't have the information for how to interpret §46.102 outside of the federal context for which it was written.
An alternative interpretation of MD 917 (b) is that no human testing is exempt, including §46.102 (b) (2) which allows surveys. Instead, all testing must go through an IRB.
However, if that's the case then the other exceptions of §46.102 (b) are also no longer allowed, including
> (4) Research involving the collection or study of existing data, documents, records, pathological specimens, or diagnostic specimens, if these sources are publicly available or if the information is recorded by the investigator in such a manner that subjects cannot be identified, directly or through identifiers linked to the subjects.
which would require IRB review for every historian working out of previously published materials.
Therefore, I don't think this alternative interpretation is correct.
Which means I don't know if MD 917 really applies to OkCupid or Facebook. (The Facebook research was published in a journal which follows the Common Rule compliance; I presume the journals have a way to resolve the question of what "department head" means when there is no department head - it must have come up before. I only investigated MD 917 compliance here.)
_EDIT_ : The MD attorney general clarified this at http://www.oag.state.md.us/Healthpol/hb917letter.pdf .
> Implicit in the federal regulations, and therefore in House Bill 917's mandate for compliance with these regulations, is the requirement that a person have in place a reasonable process for determining whether research falls within one of the exempt categories.
So determining if A/B testing is exempt comes down to if there's a reasonable process for determining if it's exempt. It's not clearly illegal.
Probably Annotated Code of Maryland Section 13-1601(b), added by House Bill 917, means exactly what it says. "Notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research, subsection (a) of this section applies to all research using a human subject." (emphasis added)
> §46.102 (b) has a list of human research which is exempt.
But those exemptions are a provision of the federal regulations which limits the applicability of the federal regulation to certain research, and so are expressly excluded from the part of the regulation incorporated into Maryland law by Section 13-1601(b).
> Otherwise - and this is another tricky part - §46.102 (c) says "Department or agency heads retain final judgment as to whether a particular activity is covered by this policy."
This, again, is a provision of the federal regulations which limits the applicability of the federal regulation to certain research (to those things not excluded by the relevant funding department/agency head), and so are expressly excluded from the part of the regulation incorporated into Maryland law by 917(b). So there's no need to figure out how it applies -- it doesn't.
> However, if that's the case then the other exceptions of §46.102 (b) are also no longer allowed
Note that, if you go through the link you posted and click on the final passed (Third Reading) version of the bill and note the edits from the prior version, that prior version of the bill had express language incorporating the exceptions in 45 CFR § 46.102(b) as exceptions to the Maryland law, and this was stricken in favor of the language now present expressly disregarding any exceptions in the federal regulations. This makes it pretty clear that, whether or not this is desirable, the legislative intent was very expressly not to incorporate the exceptions in 45 CFR § 46.102(b).
> Therefore, under the applicable federal regulations, a person may conduct research in an exempt category without, for example, obtaining review by an institutional review board. A Maryland researcher likewise may do so under House Bill 917, because such a person is conducting the research “in accordance with the federal regulations on the protection of human subjects” and, consequently, does not violate the prohibition in § 13-1602(a) of the Health-General Article.
This is dated May 2, which is after the bill passed both houses and before it was signed by the governor.
This seems like the exemptions in §46.102 (b) are all allowed under MD law.
When things like this are published it just serves to dissuade them from PUBLISHING the results of the experiments but not from actually CONDUCTING THEM.
I would much rather know when they do things like this than have them continue to conduct them in secret.
https://medium.com/@JamesGrimmelmann/illegal-unethical-and-m...
the author explains more clearly his position:
"Federal law — primarily the so-called “Common Rule”— ( http://www.hhs.gov/ohrp/humansubjects/commonrule/ ) regulates research on people in the United States. The details are complicated, the gist simple. If you engage in “research involving human subjects,” you must have two pieces of paper before you start. You need a signed informed consent form from the person you’re experimenting on, and you need approval from an IRB (short for “institutional review board”)."
The "Common Rule" appears to be a rule for the Federal agencies, and it is a rule, not a law.
Long jump from that to the illegality? But I'm certainly not a lawyer, whereas the article author seems to be "James Grimmelmann, Professor of Law, University of Maryland."
Strange.
And is the Maryland law relevant? By the author (from the article I've linked previously):
"But wait, you may be saying, Facebook isn’t in Maryland, and neither is OkCupid. True. But they have users in Maryland, and given the size of the experimental groups, it’s overwhelmingly likely that they experimented on residents of the state. Facebook manipulated with hundreds of thousands of News Feeds; that’s thousands of Marylanders. OkCupid gave bad recommendations to about five hundred users."
Well... "likely."
However, looking around, it sure sounds like this would not normally cover A/B testing, a business changing its prices and tracking results, or similarly "local" investigations.
Here is one IRB's interpretation of that term: http://www.irb.wsu.edu/definitions.asp. Others seemed similar.
Changing the way content is ranked by your private service is not in the same ethical territory as the Tuskegee Syphilis Study. It's just not.
That said, there is an interesting discussion here, which is: where is the line? Netflix runs an experiment (and whether it chooses to collect data) every time it changes its ranking algorithm, but I think we'd agree that it's ethically OK in that case. What about those sleazy mobile game companies that use data science and heavy experimentation to find and target "whales" who have an in-app-purchase addiction problem? Not ethically OK. I don't know where the line is, but it's clear to me that some tech companies are on one side and some are on the other.