Hacker News Reader

Comment by at_a_remove | Hacker News Reader

Twenty years ago, this incident would have been met with a "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." A decade back, "play stupid games, win stupid prizes" would have won out. Now, I think The Kids would say "Fuck around and find out" (this is often accompanied by a stencil of an open-mouthed alligator).

No matter the form of the reaction, I am still left wondering at the proportions of disregard, hubris, naivete, and perhaps sheer lack of consideration of consequences that went into this whole affair. For the life of me I simply cannot imagine anyone patting me on the back and saying, "Good show on that paper, old boy! Smartly done!" Was there nobody in the whole group who had reservations? Were any expressed or were they simply hidden away? I am sincerely curious as to how it managed to get to this point.

View on HN

I'll be the odd one out and say that I see nothing wrong with their study. There are much more skillful, much better resourced actors who can do what the UMN did much more discretely.

I mean, the UMN flaws were egregious and blatantly obvious. They should have been met with ridicule at first glance. If they aren't (and they weren't), there are problems that need to be addressed.

But there's a torch mob about this -- a very misled, and I would say foolish torch mob -- and as a result comments like mine will invariably be moderated down to invisible. I guess if we just brigade against some junior researchers it'll help defend the kernel from actually malicious actors...

If UMN’s ethics review(s) has approved this experiment in cooperation with Linus/Greg, there would not be anything to be outraged about.

The outrage, and the core ethical violation, centers around intentionally wasting the time of others for selfish benefit without appropriate review and consent.

> If UMN’s ethics board has approved this experiment in cooperation with Linus/Greg, there would not be anything to be outraged about.

I do not think that would help. This was done on public mailing list and deceptive behavior was also against third parties (other reviewers). I do not think Linus/Greg can give consent to that.

Within the scope of the project, Linus and Greg absolutely could deem this acceptable on a project basis. Individual contributors might then leave the project once this comes to light, but if the project owners say “We need you to submit an 8 hour video of paint drying with every commit”, that’s their right to do so, and if they say “We chose to allow researchers to waste hours of your collective project time for a experiment”, that’s their right to do as well. If they want to guide the project down seemingly-unproductive paths because they truly think it’s worth it, then they will.

The difference is one of these gives you the opportunity to consent or opt out of the wasting of your time, the other ones doesn't. You aren't describing ethically equivalent things.

With an experiment of this nature, it is not always possible to get universal consent-or-refusal, due to the nature of the experiment. If you're doing an experiment with footpaths and you want to close specific paths to see how people's behavior changes, you do need to seek permission from the owner of the footpaths, and you do need to perform some sort of ethics review to ensure you aren't creating severe mobility issues for those being experimented upon, but you would never be expected to seek permission from each person traveling those footpaths to waste up to a minute of their time by closing the footpath they intended to traverse. Whether or not that time waste is acceptable is ultimately the decision of the owner/operator of the footpaths, not of the individuals using it.

So, then, it is similar with a 'secret experiment upon unwitting people' and the Linux project. The owner/operators are Linus and Greg, and as the experiment cannot be pre-announced without tainting the outcomes, they are the ultimate "go or no-go" decision makers — just as the owner/operator of the footpaths would, too, have a right to refuse an experiment upon their participants. The individual Linux contributors who participate in the Linux project have no implicit authority whatsoever in any such consideration, and would not be offered opt-in or opt-out consent prior to it being performed. If the good of the project requires pentesting the processes that contributors operate, the project has every right to do so; it's for the good of the project, and contributors' time spent will have been net valuable even if the specific contributions are felt to have been "discarded" or if the contributors feel that their time was "wasted".

This lack of individual authority in many respects is not comfortable or appealing for open source contributors to consider, but it's critical for us to confront it and learn lessons from it. We do not perfect authority over how open source projects use our contributions, whether in time, money, or code. Some percentage of our contributions will always end up being discarded or wasted, and sometimes that will be upsetting to contributors. These are real aspects of project participation regardless of whether secret experiments are approved by the project owners/operators or not. I hope that this event helps us develop better empathy for large projects, such as Linux or other operating systems, when they make decisions that benefit the project rather than contributors.

With an experiment of this nature,

I think the thing I'm objecting to is that this is a valid or meaningful 'experiment'. Placing there, to me, is already inaccurate framing.

In a previous thread a user suggested the following:

Get consent from the project leaders and announce publicly their intentions and a time window of a few months. Then randomly submit the patches as originally outlined.

Although this would not prove as strong of a result, it is far more ethical and similarly effective. Companies use this kind of method all the time.

Wouldn't Linus/Greg be heavily criticized for such cooperation then? The nature of this experiment prohibits it from total consent with the process been experimented on.

They likely wouldn’t have cooperated with study, but advised that there is no point to it as there is no mechanism to catch the behavior they described.

It sort of like asking “What if we volunteered for the parks department and slipped a load of salt into the fertilizer?” - of course bad actors can find new ways to circumvent security.

"We found out that we have no process for confirming that we're applying the correct fertilizer to the soil in question, and accidentally salted the earth of a small patch of flowers during the test. While not ideal, the damage is small and contained and repairs are underway."

"The Parks Department responded by saying well duh, we could have told you that there was no such process, and then you would have avoided killing the flowers and wasting our time by making us check on everywhere else you were allowed to apply fertilizer".

No one’s discussing that, and I think that’s the most valuable question of all. Was this experiment so worthwhile that it ought to have been approved? Or was the experiment itself so irrelevant that ethical compliance or not, it shouldn’t have needed doing? Is random testing of gatekeepers an appropriate process in Linux development?

Actually I think this is a very good point you're raising. Maybe the kernel community has a whole should consider a way of checking their processes. Now randomly submitting buggy patches is probably not the right way, but there very well might be some interesting research (on the processes) that could be done. So maybe a document that laid out what would be acceptable and what not could be helpful.

There is no such thing as UMN ethics board, although they do have an ethics office: https://integrity.umn.edu/ethics. I don't think they get individually involved in specific studies unless there's been some sort of compliance violation.

It's also questionable whether the IRB should have considered this human subjects research and not given them an exemption. It's also questionable whether, if the IRB had done that, the IRB would have stopped the study or asked for revisions to the study design (they would if they were paying close enough attention).

Professors at universities are typically given large amounts of freedom to conduct studies without heavy prior approval, it's a tradeoff.

And it's a trade-off that the Linux maintainers have now justifiably called them to task on for making the trade-off in a way that pushed the cost onto external parties.

As the top level stated, "fuck around and find out."

Okay, “UMN’s ethics review(s)” encompasses whatever the exact ethical review process(es) are. Updated.

ethics is mostly self-regulated; when you apply for grants, the University ensures you're not proposing to do anything illegal (risk protection for the U), but subsequently doesn't actively monitor most researchers to ensure they are being ethical. Same for publications. In those cases, the response is entirely reactive, doing investigations and actions after the problem hits the press.

> If UMN’s ethics board has approved this experiment in cooperation with Linus/Greg

No, informed consent must be with all participants and maintainers reviewing the patches. Why does Linus/Greg get to decide that for others?

That is not as cut and dried a decision as you frame it to be.

California emissions testing for vehicles includes licensed smog test stations and a process where undercover inspectors bring cars that are in violation to those stations. If the smog test station is incompetent, they will be cited and perhaps stripped of their operating license.

If another state decided that they’d like to start performing random tests upon their network of smog test stations, without any retaliation to those stations, then it would not be a violation of ethics for that state to send undercover cars through the stations.

It would be unethical to punish those who fail undercover tests, unless the state had announced that random undercover testing was beginning and that punishments would be applied for failures.

The researchers were not attempting to modify the behavior of the participants, nor did they seem to be interested in naming and shaming specific maintainers, so it’s not as simple as “anyone who comes into contact with the experiment must be fully informed”.

A professor is not a government. Also, all governments will use uncover officers without warning first.

The analogy is from California to Linus/Greg, not UMN.

California controls the licenses for smog test stations. I would imagine there’s a clause in the contract that says “California, at any time, may do random undercover inspections of the smog testing facility to ensure compliance” which the owner of the licensed smog station would be aware of.

Do you see how that differs from an academic randomly experimenting on an open source project with no notice or warning?

Retail store owners/managers contract out “mystery shoppers” to test compliance with retail store policy and procedure. This example is also nothing like the UMN experimenting on Linux, since there’s a contract and both parties are aware.

A similar example to the UMN/Linux situation would be an academic doctor deciding to randomly test blood donor screening by sending in HIV positive people to lie about their status in order to donate tainted blood and only telling the Red Cross or whoever after the blood has been donated.

I guess that depends on whether you consider this a sociology experiment or white hat work.

I'm not sure that I agree that sociology experiments have 'informed consent' the way you appear to be thinking of it. Yes, you know you're in an experiment, but if you know what the experiment actually is, then your reactions are not authentic and you skew the results (which always makes me wonder about clever people in experiments).

In white hat stories, it's not always the case that everyone knows ahead of time, but 'enough' people know. Those who do know bear part of the responsibility of ensuring that things don't 'go too far', and they give organizational consent but not personal consent. Although I confess that OSS might be a little fuzzy here because I didn't sign anything when I started. You can't tapdance around informing me by pointing to some employment agreement.

You are free to disagree. Obviously, not every scenario can be navigated using an arbitrary policy for conduct, which was what clearly happened here. 'Informed consent' in the context of cybersecurity research is described in the Menlo Report [1].

And fyi, not all white hat stories are clean in their approaches, that in itself remains a controversial topic for another discussion. Furthermore, employees in an organization are under a different set of contractual obligations, full of caveats, to their employers. In some ways, they've already "consented" to specific bare minimums(white-hat can be framed as security awareness training required in your job role).

Open source contributors and reviewers are individual third party actors. No one has established any tolerance limits. So "enough" people doesn't really apply here because no one was made the arbiter source to decide that.

[1] https://www.dhs.gov/sites/default/files/publications/CSD-Men...

With senior endorsement it would be easy to recruit a pool of participants.

Yes, opt-in informed consent from maintainers and reviewers of the patches.

Somehow I feel that wasting some developer time in a situation where informing all the parties involved would have probably undermined the research itself, is a lot less of a ethical breach than full blown coerced human experimentation with permanent effects on those people's lives, which this research is being compared to. The outrage on HN about this experiment has been mind-boggling to me, how sacred people hold keeping the Linux kernel and its maintainers' time pristine. And ultimately it seems that the research was useful, given the blast radius it produced.

Time wasted is a permanent effect that cannot be recovered, so your argument falls apart.

What a rhetorical power move it is to preemptively decry "torch mobs" for having your comment disapproved of. You might also consider the possibility that you'll get downvoted for suggesting that "ridicule" is an appropriate response in a code review.

Maybe you haven't been paying attention, but the "torch mobs" have been pretty active as this situation has unfolded. There is nothing "preemptive" about it. The zeitgeist is pretty obvious.

As to the "ridicule", putting aside the out of place attempt to moral high ground about a rhetorical turn of phrase, yes their commits were so egregiously flawed that ridicule was the only response.

When a state level attacker carries out a successful attack via the kernel, then the developer community will hide behind shocked_pikachu.jpeg and call for more social engineering and red team training for the Linux foundation. Until then, people are happy being complacent and any attempt to change the status quo will be met with hostility.

Your comment will be downmodded because you mentioned it in advance and you haven’t actually stated why you think it was fine to abuse the open source maintainers other than “someone else can do it”.

“I beat my kids so some weirdo on the street doesn’t!”

Their particular problem in this case is that they are researchers at a reputable university. They are not supposed to be caught doing this, ever. Poorly supervised students coupled to disinterested faculty advisors is the recipe for this sort of non-sense.

Actually the supervisor was directly involved in this. I think it's much more the high pressure publish or perish culture, possibly coupled with the pressures of tenure-track hiring (I'm not entirely sure if the supervisor was already tenured).

> I'll be the odd one out and say that I see nothing wrong with their study. There are much more skillful, much better resourced actors who can do what the UMN did much more discretely.

Puting these patches under false pretense is deceptive behavior to other people, that is problematic regardless of whether the intent is malicious or just doing research.

I agree strongly. People demanding this be pursued in criminal court are seriously getting bent out of shape. "We can mob these people because they're super in the wrong and also utterly helpless to defend themselves against my righteous fury," is ridiculous.

They made an ethical blunder, but this wasn't Tuskegee 2.0.

Seems more ethical and positive experiment to have the students to work with historical data, you can find a metric on how easy it is to slip a bug into the kernel, maybe they could identify problematic trends like time of day/year when bugs can slip in, or time before a release.

Do we really need someone to prove that a malicious backdoor can be added? since real bugs happen then it is clear that the system is not perfect,

Imagine this would be allowed, then all contributions would need to be checked not for the regular kind of problems but for all those stealthy tricks, so instead of working on the kernel you would need to attempt to find on how each commit is truing to trick you , or maybe this is fine on it's own but is part of a bigger plan.

> and as a result comments like mine will invariably be moderated down to invisible.

Please don't downvote-bait or superciliously posture like that. It degrades discussion noticeably and breaks the site guidelines: https://news.ycombinator.com/newsguidelines.html. Note the second-last one. Also this one: "Please don't sneer, including at the rest of the community." Also this one: "Don't be snarky."

Okay, but he gets a comment from you and avoids a faded out comment. If his comment was simply faded out, you would not have replied. Frankly, you need to recognize that there is a mob mentality on this topic and that legitimate comments are getting downvoted.

The way to respond to a mob mentality is to respectfully supply correct information and patiently make correct arguments. That may not work, mobs being what they are, but the solution is not to lash out in frustration, call names and fulminate. That's worse in every respect: it inflames the mob, damages the commons, and gives readers even more reason to reject whatever truth you're advocating (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...).

Discrediting your minority/contrarian view while at the same time damaging the community we all depend on is self-defeating behavior. The only reason we do it is because it's extremely frustrating to feel outnumbered by people who hold a wrong view (or one we feel is wrong). We all need to find ways to process that frustration in ourselves, before we can't stand it anymore and get propelled into the comments to seek relief. The relief of venting is short-lived and soon comes back worse, in the way that addicts need a higher dose the next time.

I don't mean to pick on you or mumblemumble personally. We're all addicts in this sense, and it's in all our interests to find a better way. This experience of frustration is extremely common, as we all know—the internet being, to a first approximation, wrong about everything.

Thanks for the reply. I respect what you are trying to do with shaping the conversation.

Yes, you are right about the frustration and you are right about the addicting qualities. I did comment in frustration.

I would say, however, it is additionally frustrating when the overall tone of the mob comes across as one of invective, and yet an invective comment that is mob-dissenting gets the comment from the moderator.

It is not just about stating correct facts on this topic. There is a value judgment and moral judgment being made here. Goodness knows I don't feel comfortable commenting counter to the collective opinion here.

Again, I agree I commented in frustration and likely unproductively and I appreciate the intent to moderate good discussion.

I'm not sure which is the greater chilling effect, the way downvoting works out in practice on this site, or having the site moderator dismiss you as "sneer"ing and being "snarky" if you openly acknowledge the downvote's chilling effect.

Downvotes on HN are capped at -4. Go to Reddit and notice it's unbounded.

The comment seemed fairly reasonable until that last bit, which was unnecessary. I felt the moderation to be appropriate in that context.

Your GP comment is an example both of what we're trying for on HN and what we're trying to avoid. The first two paragraphs are fine, on topic, and make a good argument. The third paragraph goes into snark and resentment, and ends up in reductio ad meta. Your post would be strictly better with just the first two paragraphs. Is that so hard to see? As for chilling effects, I wish we could chill the latter kind of stuff more effectively, or better yet, freeze it out—it would make this site remarkably better.

Maybe the most common quality-degrading thing people do in HN comments is to resentfully represent oneself (or one's view) as the unfairly excluded underdog, the noble freethinker surrounded by a deranged mob, striving heroically to hold up the truth for a brief moment before the rabble crushes them. This simultaneously supercilious and victimish pose is something people do on all sides of every topic. I'd like this community to get a little more conscious about how badly it degrades the threads. That's the intention behind the guideline, "Please don't sneer, including at the rest of the community. Perhaps we can find a better wording that will make this clearer.

I don't mean to pick on you personally! It's extremely common, as I said, and the purpose of these moderation replies is not to scold (and certainly not to shame) individual commenters, but to try to shift the culture another notch in the direction of interesting discussion, which is what we're all here for.

I don't feel picked on. However, I do think that the way that downvoting works in online communities deserves some examination. And, given the basic structure of how HN works, community members don't really have a clear way to discuss it except by complaining about it in comments like these. Which isn't the best way. But still.

"Freezing it out", as you say, has the effect of shutting that conversation down completely, which I think is potentially harmful to the long-term quality of the discourse, and in a way that is rather more insidious than the rather more banal quality problem that you're trying to guard against.

In particular, the ever-present threat of getting voted down to -5 for presenting an idea that goes against the popular opinion presents a strong disincentive to share opinions, particularly for less thick-skinned individuals. And it creates a constant low-grade anxiety for some contributors. Pre-emptively complaining about the downvote is one symptom of this effect, but it's not the only one. Others, less visible, symptoms are that people just don't share their opinions at all, or don't share them as thoroughly or eloquently, when they're worried that doing so might result in the electronic equivalent of a smackdown.

And the fact that this phenomenon affects certain people more than others does put a spin on the conversations that take place on Hacker News. Not necessarily one that has the effect of elevating the quality of discourse over the long run.

Well, that's a pretty decent reply. In any case, we can't freeze it out, so if you're right about how bad that would be, at least we're escaping that particular fate.

I wasn't meaning to say that your efforts at freezing out conversation about the downvote are having that effect. I'm saying that's the effect of the (current) downvoting mechanic itself. What you're trying to freeze out is people expressing, however indirectly, a legitimate concern about the social environment being created by the current mechanics of the site.

In other words, I think you're being a bit too quick to the trigger to dismiss these sorts of comments as just being boring conversation. My impression is that they're actually a rather banal tip to a much more consequential iceberg. You can't see the rest of it because it concerns things that are mostly happening out of view, out in meatspace, and typically only manifest themselves indirectly in what's happening on Hacker News. Especially if you're trying to "freeze out" the more direct manifestations.

You completely ignore the issue of consent.

The UMN has placed itself on the same level as those miscreants, because they screwed up in several different ways.

Finally, maturity is, in large part, knowing the difference between having a capability and exercising it. As an institution, the UMN has revealed itself to be immature. What they do about it now determines how they will be treated in future.

They acted just like actual malicious actors by not letting anyone know or asking for consent on their experiment.

Try robbing a bank and then claiming you were just testing their security and it was just for research, see how well that works out.

And it's not too rare, either. Tests of "supply chain attacks" have actually uploaded sample packages to package repos like NPM and PyPI, and the professional researchers collected bug bounties. Meanwhile the volunteer repo admins often take the blame and have to clean up all the fake packages too. https://arstechnica.com/information-technology/2021/02/suppl...

The difference is that the bug bounties are being collected from an endorsed bug bounty program.

Fake and malicious packages jam public collections regularly, and are not part of white hat research. It's a trash fire which seemingly only gets better with exploits forcing people to change.

The bug bounties are from companies who use the repos, not the repos themselves.

> Was there nobody in the whole group who had reservations?

I've seen very well-intentioned people do very stupid, seemingly callous, etc things that would make anyone who didn't know them question their quality as a human being.

My answer to these kinds of situations has been to remedy it and move on quickly. The more you dwell, the more possibilities and random connections pop in your head, and the easier it is to believe everybody is just a selfish, evil, etc person; which is far from the truth if you've chosen to live life a little.

The academic ivory tower is real. I have seen tenured professors with tenuous grips on the reality outside of academia. Something like this kernel affair seems pretty plausible to me

There are plenty of examples of the entire decision tree being tone deaf from the real world not just in academia. Often times, this turns out to be because the group involved is not very diverse so the group "experience" is not as wide. How many times has something with a double meaning been used in marketing where the world gasps and asks "was there not one person involved of ____ race/sex/age/etc?"

Wonder if anyone involved was on the tenure track. If so I assume they can kiss tenure goodbye.

It looks like the lead faculty is likely going up for tenure in a year or two.

>> Was there nobody in the whole group who had reservations?

Based on my experience (not at UMN), reservations expressed by juniors are dismissed and the seniors keep their reservations to themselves, lest they also be dismissed.

Reply on news.ycombinator.com