Open letter from researchers involved in the “hypocrite commit” debacle
lore.kernel.org
lore.kernel.org
I am a Red Teamer and work with companies to understand how their detective/preventative/recovery controls and processes are working. Here's how you resolve this:
You work with maintainers to get their coordination on the research. You work out a mechanism to prevent submitted patches from being merged (e.g. maintainers are notified before bad patches accepted by code review processes are merged).
You do not tell them when the patches are coming. You do not tell them which identities are going to be used for the patches (e.g. from which email addresses). You do not tell them which area of code will be targeted. You set rules and time bounds for the study.
You wait some amount of time before submitting such patches (weeks to months). Realistically this is all that's needed. If hypersensitive, set this up earlier and let it bake longer.
At this point, you submit patches from a variety of addresses (probably not associated with your university - it is easy to create many such identities). You also can coordinate with other researchers, universities, and companies to submit patches under identities as needed. You also study submitting from yandex, gmail, .cn and other email addresses (because isn't that interesting to know?).
The premise that there's some ultimatum between working with the community and performing the research is on its face incorrect. This is either ignorance or laziness on behalf of the researchers. Clearly, they hadn't taken the time to work with the community to work out an approach that could be mutually acceptable.
OK, how do you test SocEng vectors? Do you obtain consent and coordinate with every employee that might be targeted to receive your e-mail?
>You also study submitting from yandex, gmail, .cn and other email addresses
No, the point was submitting from a known and respectable entity, which might affect the level of scrutiny. They weren't testing a whole patching process, but a specific human component of it.
If it were an obvious phishing URL, like a variation on the company domain, then fine (maybe). But it wasn't.
1. (Windows specific) Opens up a Windows file share, which causes the person to authenticate to the file share, which through PtH/Responder results in their enterprise credentials being stolen.
2. Exploited a XSS or CSRF attack on an internal/management endpoint. Which in turns allows a pivot from external to internal access.
3. Steals a web session, cached password or authentication token, resulting in compromise of employee credentials to be used elsewhere (e.g. reused to access enterprise VPN).
These are just some not-a-browser-0day examples of a single click being game-over dangerous.
How do you do this without a browser vulnerability (and assuming it's not also XSS/CSRF like the previous point)?
Also to distinguish between #3 and, XSS in #2 was intended to mean "persistent stored XSS" as opposed to "reflected XSS". In the case of reflected XSS, this can be chained with CSP bypasses and insecure cookies to grab out e.g. bearer tokens.
My overall point is that heap fung shui 0day not required for 1-click ownage. In practice, I've not had to burn browser 0day to compromise organizations or their customers.
1. It sounds like they'd have to be pretty well targeted against the precise systems of that particular company in order to work. Which would tend to suggest more targeted spear-phishing attacks and extensive recon being done against the company systems somehow before anybody launched a real black-hat attack.
2. At that point, it feels hard to blame the individual employee versus whoever misconfigured those corporate services in the first place. Though I would guess it's fairly common for those kind of things to happen due to many systems being set up without the help of true experts and the unlikeliness of a real attack against them without either a highly-skilled black hat targeting them or securing the services of a skilled and prices pen test team.
2. I agree. Individual employees are not at all to blame. Companies who are blaming their employees for getting phished are doing it wrong. The correct action to take is to inform employees and build the other kinds of mitigations mentioned elsewhere in this topic tree.
To test if employees are easy to pish, or was it for real (black hat)?
I'm usually not testing only whether employees are easy to phish (the answer is pretty much 100% yes). I'm testing end-to-end: can you as a company prevent me from phishing through email protections? Can you detect when I'm phishing your employees? Will your employees report potential phishing emails? Can you figure out (without me telling you) which employees were targeted and which attacks were successful? Can you figure out which credentials/machines would need to be quarantined/rotated/examined?
Even more fun if you were allowed to social engineer your way into the office and steal someone's powered on not-screenlocked laptop :-)
You're talking about vulnerabilities in components, or other software, here. The user, by himself, is doing nothing wrong.
Why don't you just restrict employees to the intranet, then? Why do you give them free roam on the whole internet, but then you tell them "don't click the wrong link!".
Phishing happens when the user does something which is actively wrong. When the user opens an Word/Excel with VBA from an untrusted source and bypasses security restrictions. If they execute/install something unsigned and untrusted from some random site.
Click = fail is just wrong. Links are how the internet works. You aren't teaching anything.
It's particularly annoying where I work, as the company itself sends out a completely unreasonable amount of internal spam every.single.day - often with bad spelling/grammar, and very often with the contents being a single image with rendered text (why?!?!).
Yes, clicking on links is dangerous[0].
0. https://www.bleepingcomputer.com/news/security/google-fixes-...
How can I tell whether I can click on a link? Sometimes there's even something like linkprotector.outlook.com/[very_long_url] in corporate emails.
My usual approach if I'm unsure whether a link is malicious would be to open it in a private window (and probably in a different browser from the one I usually employ), or if I really think it's phishy, I would open it from within throwaway VM.
So, the blanket "click and fail" policy seems pointless to me. If I enter some login/PII, then I can agree I've failed the test. But a click on a link cannot be considered failure.
They weren't, though - their own paper explicitly says they used newly created Gmail addresses for the patches...
"We submit the three patches using a random Gmail account to the Linux community and seek their feedback—whether the patches look good to them."
It is my understanding that this happened and that no bad patches were actually merged.
I also empathize with the plight of the researchers — Linux is a bit different than normal Red Team engagements, in that a normal organization has a hunch of administrative / management layers who typically do not participate in the operations of the system being tested. A VP of engineering at a medium to large company is unlikely to be committing code, much less maintaining the build pipeline etc.
This is not the case with Linux. The people “at the top” are also reviewers, and so it’s pretty likely that notifying them will result in a change of behavior.
I wonder if there is some way to build some sort of Red Team consent “blind trust” organization, such that willing open source projects could agree to responsible attacks, and the attackers could register their work (including disclosure / mitigation plans) with the blind trust ahead of time.
It is not like community members are not for look out for bad commits on every new commit from most committers any way, since at least 2003, I think substantially earlier.
In your experience, do you create a "fail safe"?
So for this study, some way for the researchers to prevent any of their patches from ever being released.
"We sincerely apologize for any harm..."
While there are other requirements, a sincere apology cannot in any way entertain doubt about the fact that there WAS harm.
Truly acknowledging the harm done is foundational to a real apology, and most of us (myself included) end up sneaking in weasel words or phrases like this.
Psychologically, its nice for the apologizer, since it allows one to think "i'm being good by apologizing, but maybe I didn't do anything bad after all?".
But from the apologizee standpoint, these phrases are often devastating and can make it clear that the apologizer has no real recognition or care of what happened.
Personally I've worked pretty hard to try to remove these sorts of phrases from my apologies. It's not easy. It makes you feel much more vulnerable and you really have to let whatever you did sit with you in a very uncomfortable way. But it's worth it.
Someone will always be apologizing wrong for some. Some interpretation of what harm there was, is not necessarily the same as my interpretation. There are too many ways to construe what harm there was or may have been according to others to satisfy everyone addressed. This is an efficient wording that doesn't explicitly satisfy your (and many people's) specific issues out of "the community", which illustrates the point.
I understand doubting the sincerity of the authors, but this argument hinges here on them saying "any" instead of "all" and they are often used interchangeably in casual conversation.
Honestly, for all we know this letter could be meaningless. A real good actor would also reveal the other commits in order to have a full disclosure.
Trust isn't based on promises, trust is based on past actions. Therefore they should be treated as such, as their actions are evidence for being a potentially malicious actor.
I'm still convinced Greg did the right thing here, as it's better to be safe than sorry in this case due to the sheer scale of an attack vector that the actors might have introduced.
Even if other commits of good actors at the same University are now treated with more attention to their code, I think they are a casualty that was predictable in the moment the ethics commitee signed off the paper's research procedure.
Honestly, based on the way they handled their "research" in the first place, sneaking weasel words into an open apology letter is entirely par for the course...
To clarify: I'm not arguing this is a good or bad apology; just that the justification provided here looks flawed. Human speech isn't a programming language; it's not well defined and it's more than the sum of its parts. One can't derive conclusions about a text by analyzing a tiny subset out of context.
Based on my experience with the general public, with academics, and with industry folks, the criteria for what constitutes a sincere apology is not known by the majority. Furthermore, many of those who have heard the criteria do not accept it as a gold standard, and disagree with it (even those who are not being looked to for an apology).
The recipient can always choose to accept or not an apology. However, I find it quite distasteful to attribute intentions to someone simply because they did not follow a recipe (even if the choice not to was intentional).
While it's common in colloquial German to use the one-step-absolution and skip over the possibility of the other party not absolving you, it's also often considered rude when it matters and has a taste of "but not really". It's fine when you accidentally stepped on someone's foot, but not so much when you've stolen their car.
https://news.ycombinator.com/newsguidelines.html
I think it's both unfair and non-constructive to pick apart a apology letter based on one word like that. Let's assume good faith, especially when the writer's English might not be their first language (based on their name).
The "hypocrite commits", according to the researchers' own paper, originated from "random Gmail addresses", and the researchers continue to claim none of those got merged (though since they haven't told us what they were, ...)
The additional claim is that some of the commits not covered by their "hypocrite commits" (and thus, submitted from their UMN addresses) contained security bugs, deliberate or not, and the loss of trust in the researchers is sufficient to justify reverting all of their commits until they can be reviewed.
The apology is also specific about what they did wrong despite their intentions. It really is a good apology after reading past the first six words.
If you want to apologise, "all harm" admits you caused harm, which seems necessary for an apology. "Any" is a bit like those "I'm sorry if you were offended" non-apologies, though in this case the rest of the message does better than that.
"We sincerely apologize for all the harm our research group did" would be a little less unnatural than "...for all harm...", but it's still an unusual choice of wording that ends up sounding like you want to emphasize that you did an unusually large amount of harm.
A more standard phrasing that includes the word all would be "We apologize for any and all harm...".
The standard form I know is actually "[I am deeply sorry] for any harm I may have caused..."; the letter here does not use a modalized verb. (Compare their "We sincerely apologize for any harm our research group did...".) In that sense it's more definite than usual. The criticism above is strange.
(The reason for the modality in the standard form isn't really to leave open the possibility that you didn't do any harm. It's to leave open the possibility that you did harm you don't even know about -- and therefore can't apologize for specifically.)
[1] In general, any occurs only in negative sentences, though in the details there are several types of sentences that are sort of "honorarily negative" for the purposes of allowing any and other words that obey the same restrictions.
It wouldn't have occurred to me to discuss it above, because I thought I was contrasting any with all, and neither is present in that example. But I'd rate it above most of the alternatives discussed (while still below "for any harm").
2) "We apologize for any harm that we caused"
3) "We apologize for all the harm that we caused"
(1) is the least apologetic. This could be interpreted as saying "we might or might not have caused harm, and we think we didn't but you think we did, so we're going to apologize for your sake, but we're not really sorry because we didn't do anything wrong from our perspective".
(3) is the most apologetic. You acknowledge that you did something wrong, and that you're apologizing for it. This might be followed up with a specific list of the things that you did wrong, and that you are apologizing for. That would make for the most sincere apology. This could be interpreted as "we caused harm and we're sorry, whatever harm you think that we did, we agree that you are right, and we are sorry for all of it".
(2) is partway between (1) and (3). You acknowledge you caused harm, but you won't enumerate the things that you did wrong. So, you leave yourself a little bit of wiggle room. This could be interpreted as "we caused harm and we're sorry, but we didn't cause that much harm, we think it was actually quite little, you think it was a lot, but we're saying sorry, so let us go with this apology".
I am surprised to see people hung up on this, because the rest of the letter acknowledges specific harms done. To me, it really seems like a minor thing, and something that I might have written (as a native English speaker.)
This is a pretty weak assumption. Someone's name (and physical appearance) don't give you an accurate information about anything.
And to be honest, I kind of agree with them. I don't really see what they did here as particularly bad. They demonstrated a very serious vulnerability in the linux kernel development process. I guess the harm they caused was wasting maintainers time, a bit? But what we all got out of it is the knowledge that real bad actors could easily have done this too. It's odd to me that people are focusing on like, the etiquette here when such an important vulnerability was demonstrated.
This is a risk, but it's exposing vulnerabilities is always preferable to leaving them in place.
> Or of intentinally buggy patches making it into the wild and being exploited?
There was no real risk of this. They specifically did not allow it to make its way into real releases.
If they had allowed it into actual releases, I would have a serious problem with that.
But I'd argue it is his own fault to use unsupported patches.
Says who? I don't think that's true according to IRB standards in the US. Nor is it true for websites who A/B test according to the GDPR
Me and common sense. I don't believe consent is relevant when there is no risk of harm to the subject. IRBs and the GDPR are overly aggressive on this point, probably as a reaction to real and important violations of privacy. But the idea that A/B testing the color of your CTA button on a landing page requires informed consent is absurd.
Of course there are. But what specifically are the harms that are going to be caused by either this research or a landing page A/B test without a click through pop up? The existence of theoretical harms for broad categories of potential research does not have a whole lot of bearing on these specific lines of research.
> If you're interested why these ethical principles exist in the United States, I suggest you at least skim the Belmont Report
I read it. It was interesting, but I don't think it's particularly relevant here. The only prong of its test that would be relevant to this experiment is "respect for persons". The idea that somehow not revealing the bug or the experiment was intrinsically harmful as a violation of a person's moral autonomy.
I don't buy that line of reasoning, and I can't really think of any valid consequentialist justification for it, and the report itself does not attempt to justify it either, as far as I can tell.
A/B testing on major platforms is much more sophisticated than that[1].
It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
So, yes, it should definitely require informed consent and opting in.
On some it is, and some it isn't.
> It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
This is just a long and emotionally laden way of saying "changing images and text to see which works best".
> So, yes, it should definitely require informed consent and opting in.
Guess we're just going to have to agree to disagree then. I don't see any reason whatsoever to think that this practice is harmful to the subjects being experimented on.
At worst, it's harmful to society in general because it incentivizes consumerism and potentially self destructive behavior. But that is completely orthogonal to the issue of informed consent. If you got perfectly informed consent from 10,000 people to tease out the perfect pitch text, and then deployed it against the rest of the population, the effect would be exactly the same, whether or not you got consent.
It's just a scammy move at best. To me it's borderline criminal to sabotage a project like that.
https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-...
For me, it reads like this "With the best intentions, we were helping you. Unfortunately, you are too stupid to not realise this. We are sorry that we hurt your feelings, but please let us continue in helping you."
When you get such an apology, best thing is to avoid such people. Because it's clear they do not understand where they went wrong.
They should have either apologized with "we made a very big mistake that had a negative impact, it did more harm than good". Or they should have argued with real evidence on how they improve the Linux kernel, like refer to real exploits that they fixed.
This is just a "we're sorry that you don't realize we are helping you"
Indeed: "The Court held that "the word 'any' is to be considered all-inclusive"
https://www.michbar.org/file/generalinfo/plainenglish/pdfs/9...
A more substantial criticism is that blending we’re sorry with excuses and mitigating explanations is what makes it lose impact.
But really, they acknowledge the harm they did and they said sorry and I hope you give them some credit for doing so.
I would have listed the specific harms in a concise manner instead.
Maybe save that word “any” for the end, maybe and only as a future tense of profusely avoidance of future harm.
However, I find something very problematic. This quote shows it:
"We have learned some important lessons about research with the open source community from this incident."
This is something I don't like. This is not something about "research with the open source community". If anything, they should have learned something about treating human beings as persons and not as involuntary guinea pigs. They should have learned something about not breaking anyone's (not only any open source community) good faith and trust, and respecting that.
They behaved like jerks and they still cannot see that.
They just come off as disconnected academics. Samething with the experiments with algorithms from Facebook. They are so wrapped up in what they are doing, they no longer see the "users" as human beings. There are some times this is almost required to survive like being an ER doctor. If you lose that many people, it could be debilitating unless you were able to disconnect. That's far and away much different than these robots.
They didn't decide to conduct this research on a whim. They had full approval of their university ethics board as well. They published a paper and had it peer reviewed without (as far as I know) anyone immediately calling for their heads.
They can't immediately turn heel face and believe their actions are wrong, always were wrong, and the wrongness should have been obvious to them.
Yet, despite this they recognize the hurt they've caused and are genuinely apologetic and will never do it again. Asking them to dismantle their world view in a week is a bit much, even criminals are given a few years of quiet contemplation before being asked to tell a parole board they have changed their hearts and minds.
If they lack knowledge of why it was bad, what will prevent them from doing another garbage study next time.
This study could have been done with consent, with limited bias, but they chose to go the idiot route.
Are we not allowed to discuss the possibility of someone hacked an university email address and then submit a patch?
> we are very sorry that the method used in the “hypocrite commits” paper was inappropriate
This reads more as "sorry you were offended" than "It was inappropriate and we are sorry".
> As many observers have pointed out to us, we made a mistake by not finding a way to consult with the community and obtain permission before running this study; we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches.
Bringing up why you did something in an apology is very shaky. Like in this case, it can sound more like justifying / excusing.
Why is that a problem?
Which I'm guessing is why they made it an open letter. This isn't about acknowledging their misdeeds and trying to fix their relationship with the OS community, this is them trying to sneak in spin and justifications to a public announcement to cover their collective asses, to put forth a false narrative that shows their actions in a better light than they acted in.
After all before a ban, there where complaint(s) made regarding this paper, by other researchers and community members.
Perhaps the entire "tech" industry needs to learn that lesson. Non-technical end users should be entitled to that same level of trust as nerds. I can download free open source code, extract a tarball and build the software without worrying too much about scanning through all the files first for phone home/telemetry/OriginTrials nonsense.^1 However non-technical end users who use programs compiled for them by "tech" companies with ads and surveillance as their "business model" are not entitled to the same trust. I cannot think of any justification for the difference.
Audit studies are conducted all the time where researchers lie to people and waste their time.
> Factors Determining Callbacks to Job Applications by the Unemployed: An Audit Study
> We use an audit study approach to investigate how unemployment duration, age, and holding a low-level “interim” job affect the likelihood that experienced college- educated females applying for an administrative support job receive a callback from a potential employer. First, the results show no relationship between callback rates and the duration of unemployment. Second, workers age 50 and older are significantly less likely to receive a callback. Third, taking an interim job significantly reduces the likelihood of receiving a callback. Finally, employers who have higher callback rates respond less to observable differences across workers in determining whom to call back. We interpret these results in the context of a model of employer learning about applicant quality
http://www.econ.ucla.edu/tvwachter/papers/audit_study_Farber...
How else do you prove your process can stop real infiltrators by state actors?
There are many crimes in the world that are only crimes if you do it without consent.
> How else do you prove your process can stop real infiltrators by state actors?
One of the common security controls against infiltration by foreign nation states is espionage being a capital offense. Well obviously not appropriate, i think that its pretty obvious that these researchers would have not succeded if they faced the electric chair like a spy would. So i don't think the comparison is apt.
If you have approval from one person in an org of 100,000 is this still okay?
It has to be, because getting explicit consent from every person in the org at that size would be untenable, and make them artificially extra vigilant during a potential audit window.
Well now scale this to open source. The typical web project has 2k+ random nodejs libs in their dependency tree. These are all separate orgs/individuals.
Real world bad actors are backdooring these projects constantly. I specialize in this area of research.
If every white hat has to get permission from exponentially large dependency chains, they will never even come close to being able to compete with black hats here finding the weak links of the chain.
The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
You have to get consent from someone who has legal authority to give it to you.
> If every white hat has to get permission from exponentially large dependency chains
They don't. They just have to get permission from someone in authority. Different open source projects have different governance structures. Sometimes that is a single person.
> The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
If you are hacking someone elses system without their consent (not to mention for your own personal gain), you are a blackhat. By definition. Pretending to be a researcher doesn't change that.
there lies the difference, duh.
Or the the thousands of brew packages that are blindly merged unsigned by 800 people with access?
Who pays to give a white hat as much freedom to find supply chain attack vectors here? Who gets consent from every random student whose code, if compromised, would compromise every major company?
We have created a massive mess, and I don't know that we will be able to hire enough researchers to fix it.
We are going to need unpaid volunteers, and a lot of them.
> We are going to need unpaid volunteers, and a lot of them.
that's absolutely orthogonal to the question at hand, it's not a matter of paid or unpaid, neither being volunteer or not: it's a matter of consent.
if you don't sought consent beforehand either via a contract relationship or a sponsored bug hunt program or the likes, you're a racketeer, not a white hat, and you should (and will) be treated as such.
According to other people in the conversation, this is already taken care of by reference counting (https://lore.kernel.org/linux-nfs/20210407153458.GA28924@fie... ) and the patch apparently does nothing. The commit doesn’t reference any specific tool they’re using, or any bug they faced.
Looks innocuous, but I guess past behaviour from this group left enough of a bad taste for Greg KH to be suspicious.
https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah...
You can see that Greg KH is skeptical that the patch was generated by a tool. It’s also unclear to me if the patch is actually harmless or not. It would be good to get some definitive clarity on that but the lkml discussion of it seems inconclusive.
I would add that the student’s tone in this message feels really familiar as an open source maintainer: this is the tone of someone doing something they know is bad being called out on it and trying to deflect with outrage. The last time I got that tone, it was someone who had created multiple sock puppet accounts to try to discredit my project and force moderators to reverse or apologize for a (completely justified and mild) moderation action. So I can’t help but feel that something fishy is going on here and the UMN research group still isn’t being honest about it.
Apparently, they sent html mails, which the list refuses to deliver.
The idea of the kind of research they previously did is to submit patch requests with some kind of trick or hidden agenda first, then do some analysis of the results, and then later on publish a paper explaining what they did.
Now here they are again submitting a weird looking and seemingly poorly conceived patch. Who knows what they're really doing? Perhaps they're working on some kind of new paper, with who know what purpose. Maybe they'd find out in 6 months. Maybe it's a failed line of research similar to the previous ones which they won't actually publish. Or maybe they just have no idea what they're doing. Either way, it seems like a waste of time at best. The mass ban and revert sounds like an appropriate move.
I hope I'm just being overly sensitive here.
The core issue here is that the system under which they were working, and the researchers themselves, did not consider this work to amount to unethical human experimentation - even though the work directly involved infiltrating and exploiting humans and human social systems. There is no mention of this nor of any substantial desire to understand or address this systemic failing and, until there is an unconditional acceptance of responsibility for the harm caused and a real actionable change for the better with accountability, I don't see how we can consider this matter to have been apologised for.
This bit just reads like "sorry you're upset".
I would have expected something along the lines of "sorry, we should have asked the maintainers and we understand why it was wrong not to", instead of "sorry, we didn't ask the maintainers, but this is why we didn't".
Legitimately these folks have some kind of personality defect. They observed something that everyone already knew and then decided to act on it and pretend they were doing something interesting and new rather than just being twats. They should be treated exactly as should be based on their actions, regardless of their claims of being researchers. This would be a perfect cover for being on the take from a government agency. They should be investigated to ensure they aren't actual malicious actors.
It takes almost no imagination to think of hundreds of "security vulnerabilities" in the world, yet the vast majority of people realize that it's dumb to act on those observations.
[signed personally, by every involved researcher, professor, IRB member, and all of the relevant university leadership including the board]
Something like that would do nicely, in my opinion. Then they'd have to absolutely follow through on it. Ideally, they'd also ask involved members of the community so harmed, among other experts, to independently review and watch the process to ensure no further harm is being done, progress is made, and public accountability is kept.
I wouldn't trust those researchers again, not for anything. Free software depends heavily on trust. It's astonishing that it works so well; but if you break trust, it's very hard to repair the fractures.
The legal arrangements around free software are fairly fluid, because there hasn't been much case-law, because there's not much money sloshing around for Free Software developers to pay lawyers. Banning pull-requests from that college is much cheaper than sueing.
I hope the whole uni stays banned until the ethics board issues their own apology.
Then let it go, I say. People make mistakes. We shouldn't blame everyone for the mistakes of a few.
But we have to protect the Linux kernel; billions of people depend on it's correctness.
While I had a bunch of notes about what makes the sentiment behing an apology meaningful, it's really separate from the issue that this should have been done privately. A public apology is just another thing about them and their message. They're performing and trying to draw in sympathetic members of an imaginary audience. There is no humility in public displays. I would be surprised if anybody asked for it, and doing it without asking is just more of the same type of behavior. There is no such thing as a public apology.
That said, I have no doubt they have suffered personally over this, and have some compassion for that suffering itself, but not sympathy for this performance. There is no gesture that restores trust. When their contributions and acomplishments become more remarkable than this issue, they will have restored it, but like most things, there is no "back" to go to. Maybe that static analyzer will be the ultimate bug finder, as succeeding at that is probably their only out.
You seem to be implying that one apology is as good as another and that is absolutely not true. To me is is absolutely unfair to expect that people will treat a zero effort "sorry your feelings were hurt" apology the same as a deeply heartfelt apology by someone who feels deep remorse.
By it's nature, an apology should be an act of making amends to a person your have wronged. If the words of your apology don't do that, you have failed, even if you had good intentions.
If the author of the apology isn't a native speaker of english, this seems like a document of sufficient significance to merit asking a friend or colleague for a proofread. (Which you should do in a case like this even if English is your first language.)
Software is complex. Wasting maintainer's time is harmful to the maintainers and to those who use the software. If the goal is to explain what sort of exploits to be on the look-out for, just write the paper without doing the exploits.
As for the letter, their goal is to recover their reputations, not to apologize. I suppose a lawyer helped them in spots, but there is still a certain truth that shines through: they want to continue on doing this sort of thing, because they are oh-so-clever and their work is oh-so-valuable. Nice try, but the academic community may be better off without these folks doing this sort of research and influencing students to follow their methods.
I'll be interested to see how they react to feedback / responses.
Either this is a bald-faced lie or they didn't think through the implications of their research, neither of which bode well for their trustworthiness in future. However, if it's the latter, then they possibly deserve additional leeway.
On the other hand, I wonder whether this letter would have eventuated without the ban - if not, then they've likely been forced to write it, in which case any remorse is either selfish (i.e. they've been professionally reprimanded and feel bad about getting caught, or making such an egregious error in judgement), or hollow.
> I agree, but let's give them a little extra benefit of the doubt. I thought as I was reading it that it seemed stilted and forced, then I wondered if the author(s) don't speak / write English as a primary language.
https://www-users.cs.umn.edu/~kjlu/
It looks like you might be right about this, but I think that reading it as stilted and forced is still accurate.
Context is everything, of course. I think you're right that it's tainted by the fact that they absolutely did not have a choice, and coming from people who've deceived the same tribes they're now trying to apologize to.
They university, and indeed, the researchers, only acted once the ban was imposed - i.e. they were getting away with it without reputational damage, and only once it was stained did they actually do anything.
ETA: GKH on reverting the patches[1]
> This patchset has the "easy" reverts, there are 68 remaining ones that need to be manually reviewed. Some of them are not able to be reverted as they already have been reverted, or fixed up with follow-on patches as they were determined to be invalid. Proof that these submissions were almost universally wrong.
[1] https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...
https://lwn.net/Articles/854064/
What the authors did is wrong, so did GKH actions. Let's be honest in calling BS out.
Yeah, the authors are claiming those past 190 patches are fine... but they also essentially did the same thing when they submitted the intentionally bad patches too.
So the question is, how much do you trust them right now? And given they just submitted intentionally bad patches, I don't think it's very surprising they don't have a whole lot of trust.
And if you don't trust them, then you've got 190 patches active from an untrusted source with a history of submitting bad patches. That's not a particularly good situation. Why wouldn't you revert them?
The specifics of the situation make the whole thing substantially murkier, in my (entirely unrelated) opinion. But take a broad enough view, and this is pretty much the default path. However the specifics make this a bit of an oddly exceptional situation, all things considered.
Blocking an entire University seems extreme; but just think of it as a sanction on the University's ethics board. I think a (short, and very explicit) apology from the ethics board ought to suffice, to get the Uni off the hook.
And I think those researchers should not be allowed near Free Software again, unless their pushes are going to be rigourously scrutinised.
- "We are sorry for the harm we caused" instead of "We're sorry for any harm we caused"
- Not trying to explain their actions in the first paragraph of the apology! I think most folks are aware of their intent by now, and leading with yet another explanation just makes the whole thing feel disingenuous
- Avoid saying things like "this has been painful for us as well"
Unfortunately, it has so many hallmarks of a non-apology, and it's hard to look past those given the context.
As others have mentioned, they can clear this up with their actions going forward, but it will take time to rebuild trust.
Is that a thing? Like there's some non-apology Bingo card you can fill out? I don't see a connection between the criteria you listed and genuine-vs-false contrition.
You may perceive these things one way, but ultimately you can't know the minds of others well enough to tell if they are sincere or not about anything. You don't get to just declare yourself the arbiter of their feelings because they used "any" instead of "the".
I would expect it to be there and would minimize attempts to marginalize my apology with direct, precise, and inclusive language.
For every word created or omitted to those ends, the number of negative comments will be reduced.
Fact is people take it how they take it and feel what they feel.
There really is no "can't" in any of that.
...which is why the consistent feedback to those ends is here in the discussion.
How else is this to be done and be meaningful, not easily gamed?
Serious question.
I tend to have pretty dry affect at certain hours of the day, which has caused considerable frustration for myself and others when I know I'm sincere but they don't. Nor does it make much sense to me for that sort of mind-reading to take place democratically. What do we then gain from it?
At best, you get an apology for a misunderstanding over the previous apology, plus a second draft that the masses might like better. Then you still get that lingering contingent that says "you just did that to placate people! Now we really know you don't mean it!"
The cultural apparatus for "Saving Face" is completely broken on a build failure for missing dependencies.
The first two sentences seem sincere. This reads as a real apology.
As far as giving benefit of the doubt, it's likely not done out of malice in the first place. Just a combination of poor reasoning and doesn't exactly clear up if they even considered alternatives to control their variables. Unfortunately, it seems like they were already given the benefit of the doubt from their first offense, but did not take any lessons away from that, so it would be understandable if the maintainers kept their ban.
In sum, they show not remorse for doing wrong.
I completely agree their experiment is unethical. However, it's not actually clear cut to most researchers the ethical bounds of their work, especially for study papers that's never really been explored before. Ethics in of itself is largely a active subtopic for many areas in CS, not only security research. AI is one area where qualifying potential harm to human beings remains largely controversial. Ask any ML scientist, and they'll tell you that determining the ethics of a project is not their responsibility.
The ethics around research that involves deception have been pretty well established and are are several good comments here explaining them.
Every scientist is personally respsible for the ethics of the research they conducts. Full Stop, no caveats allowed.
If your research is in an area where the ethics are controversial or grey, that means your need to spend MORE time considering the ethics of your research, not that you get a free pass from being responsible.
If a any scientist espouses the opinion that determining the ethics of their projects is not their responsibility, they should be permanently barred from recieving grant money.
Ethics in computing research remains an active research area. This incident will be used as a case study in the future, but it's not that well established. Many people have been using anecdotals, which honestly don't fit the scenario because so many variables and parameters distinguish other types of pentesting from this. And disappointingly, not a single post has actually produced the documents that establish this.
Arguably the first set of guidelines for ethics in computer security research [1] was published in 2012 and not yet widely taught in Ethics lectures (I only know about it because I learned Computer Security from one of the authors).
On identifying harms:
> "Challenges identifying harms in ICTR environments stem from the scale and rapidity at which risk can manifest, the difficulty of attributing research risks to specific individuals and/or organizations, and our limited understanding of the causal dynamics between the physical and virtual worlds. As with all exploratory research, it can be challenging to articulate benefits such that subjects can make informed decisions. In ICTR our ability to qualitatively and quantitatively foresee the probable benefits is particularly immature."
On this type of research:
> "Research of criminal activity often involves deception or clandestine research activity, so requests for waivers of both informed consent and post hoc notification and debriefing may be relatively common as compared with research studies of non-criminal activity."
This isn't a huge change from 30 years ago since Moor [2] wrote his thesis on Computer Ethics, see:
> "A typical problem in computer ethics arises because there is a policy vacuum about how computer technology should be used. Computers provide us with new capabilities and these in turn give us new choices for action. Often, either no policies for conduct in these situations exist or existing policies seem inadequate. A central task of computer ethics is to determine what we should do in such cases, i.e., to formulate policies to guide our actions."
Researchers themselves are far from educated on this topic; you won't ever explore this in depth unless you're in this particular sub-field. IRB/REB boards are considered the most qualified but are possibly too outdated to navigate around this. It's a whole mess, there is currently a lot of questionable research in many areas of computing, but the clock moves forward.
[1] https://www.dhs.gov/sites/default/files/publications/CSD-Men...
[2] https://web.cs.ucdavis.edu/~rogaway/classes/188/spring06/pap...
These guys are malicious clowns, who came up with an idea that would hurt Linux, but advance their careers, and they went all in on it.
The words are there. It's not up to us to decide if they're "genuine". No one's a mindreader.
The tendency to view apologies as fake seems more often to reflect how harshly we view the one making it.
It's like telling your partner "I'm sorry if I hurt you". No, you hurt them. Apologize for hurting them.
Agency returns as trust does.
Regardless, you can't genuinely apologize for things that aren't in your control. (You can and should empathize and sympathize with their feelings about it.) It may be a convenient fiction at times, but there are many downsides to this type of boundary blurring. I think you do the recipient a disservice by implying they have no agency—especially if they believe you. You also face the danger that your apology seems insincere or you misjudged their feelings, which could actually make matters worse.
If we really had complete agency over our feelings then I guess we’d all just choose to feel great all the time. Doesn’t seem to work that way.
In my view, the extent to which we can influence the emotions of another is the extent to which the recipient themselves has given implicit permission to do so; which is to say, they have agency. I don't think that simplifies into being able to choose to feel great any old time we like. I also don't think it at all excuses bad behavior or justifies a lack of empathy. But I will say that I feel a lot more good moments and less frequent and intense bad moments since I adopted this view and took responsibility for my own emotions.
There is no universal rule that can be applied to determine when one vs. the other is appropriate, but seemingly innocuous actions can have negative impact, and apologizing for the innocuous thing may not always make sense.
To be clear, this is not universally true. There are situations where apologizing for the action makes sense, and situations where the action that led to a negative outcome are either inconsequential on their own, or there might be a myriad of factors leading to the need for an apology.
Apologizing for the action can also come across as disingenuous or passive aggressive in the wrong context. This is especially true when the action is well-intentioned, but has the wrong effect. Taken to an extreme, this sounds like "I'm sorry I tried helping".
The phrasing is such that it acknowledges harm was done. It's an apology, full-stop.
You have to acknowledge (in full) what it is that you are apologising for. That's the most important thing. Promises are meaningless.
It's not possible to "be genuine"; especially in a public post, nothing is genuine, everything is PR and smoke. Both the researchers and their ethics board need to grovel. If they did that, this tornado-in-a-teacup could be over in a month or two.
But keep those researchers banned.
There is no need for the researchers to be in any specific state of mind or even apologetic as long as they don't try to submit bad faith patches again.
Really the only reason there is even a need for a public apology is to signal how much pressure they are under so other people think twice before doing the same silly thing. Otherwise they could just apologise to the maintainers directly.
To me, it's always a distinct sign that an apology isn't sincere when it hedges apologizing for "any harm" that they might have caused. In my mind, a sincere apology clearly acknowledges harm that has been caused and apologizes for that.
Well constructed apologies are hard to seethe over.
This one could be better.
Many, myself included, believe it should have been.
Listing the harm you caused is also an important part of the public record, so that anyone else considering a similar scheme in the future can see what consequences were agreed to have happened the last time.
[0] https://dave-dittrich.medium.com/security-research-ethics-re...
Other ways to resolve it include collecting data without deception: instead of introducing flawed or malicious patches themselves, researchers identify such patches that have historically been submitted and then review the processes that led to their acceptance or rejection. This is more difficult, but might arguably produce better results. In the case that there are few or no such cases on record, then I would question the value of doing the study at all: deceiving people to study a phenomena that doesn't appear to occur at an appreciable rate is difficult to justify.
There's a simple heuristic: if you're studying a group of human beings that you are not a member of, and for which no members are consciously participating, you must be extremely careful. The general rule in anthropological and sociological research is that you do not lie to your subjects. There are cases where the value of the research is sufficient, and for which no other options are available, to break that rule. But they are rare and the utility must be clearly shown and carefully reviewed by a qualified third party. This experiment doesn't come close.
There will certainly be those those willing to argue that this isn't human experimentation and thus does not require ethical review. If your experiment depends on misleading human beings---directly or by omission---then it requires an ethical review. It is unethical to waste people's time to no purpose. In the case of an open source project where volunteers are donating their time, it is particularly egregious: they were squandering volunteers' time. In effect they were destroying part of the contribution people made to a project they care about. That requires a very clear justification, which this particular project absolutely does not provide.
I recall a conversation with other IRB members shortly after the Sokal Hoax became known. Our general consensus was that it was hilarious but absolutely unethical if considered as an experiment.
Can there be ethical possibility (highly remote one) where an study (assuming it is objectively justified) conducted with deception, but without prior informed consent at all. (eg. human subjects will not know that there's time is used for another purpose)
E.g. I the researcher ask if you consent to spending 30min completing a series of tasks to sort objects by their shape, presumably because I want to study your ability to recognize shapes. However, what I am actually studying is the group dynamics, of how well you and others in the group cooperate or have conflict over your tasks.
Yes, this happens all the time. But note the salient features: (1) the subjects are aware that they are research subjects and have agreed to participate (albeit without full knowledge of how the collected data will be analyzed). They have agreed to be studied, and have agreed that their time may be used in pursuit of this research. (2) All such studies undergo a very stringent ethical review and are usually monitored closely by third parties (at least since Milgram made it extremely clear that this was a necessary policy). These issues are complex and difficult to navigate---which is precisely why we have review boards. Every experiment has to be evaluated to balance the requirement to act ethically with the value of the research data to be collected. Skipping that requirement is unacceptable.
In my experience, the moment a research team starts looking for reasons not to classify what they are doing as human experimentation is the moment when it becomes extremely clear that they need board review.
Some relevant cases; the Facebook case[0] suggests it is justified because of EULA, and court determination test[1] is complicated for me to comprehend (to be honest), but seems like most relevant to this discussion.
[0] https://www.theatlantic.com/technology/archive/2014/06/every...
I've learned over several decades that any simple pronouncement that "X is ethical/unethical" is an effective way of ensuring that you will be wrong about some particular case. Yes, I think there might be cases where it is justified, but it would be very rare and require extremely careful monitoring and review. Two areas where it might come up would be medical research and research on children or adults of diminished capacity. In the former, there might be situations where the value of the research to the collective health and safety of the entire community would be sufficient to balance the use of such deception. In the latter, it may not be possible for your subjects to give informed consent (side note: this is, of course, also a ethical problem in experimentation on animals). The ethical dimensions of parents or guardians consenting to experimentation on their charges is extremely complex.
I will go out on a short limb and assert that ethical experimentation using that sort of deception depends a great deal on the details of a given case. Very subtle changes in the experimental protocol could easily change its ethical acceptability.
P.S: And, of course, there are a vast number of things that are completely legal but absolutely unethical.
[0] https://drive.google.com/file/d/1z3Nm2bfR4tH1nOGBpuOmLyoJVEi...
So it's sort of hard to take them seriously as human beings, and not just caricatures.
I can imagine research on a "process" made up of humans as having some value. For example, testing EMTs on whether they correctly diagnose injuries, or testing TAs on whether they catch exam cheating. But I would expect the researchers to ask someone (in those cases, maybe the manager of the EMTs or the TAs, in this case I suppose Linus) whether this research is wanted, and for guidance on how to go about it. For example they could submit the questionable patches during slow times. Does anyone know if that happened? I assume if it had, they would've mentioned it.
With that kind of permission, I wouldn't have a problem with this research and I don't think most people here would either. Without, this is the academic equivalent of those "just a prank, bro" youtube videos.
This shows one thing to me, clearly, they still don't understand how such studies are conducted. not authorized penetration testing is illegal, to a very large degree.
For example, as far I understand, and as many pointed out; you get permission from members for an study without specifying time or any details of a study, then conduct study in 6 month.
-they apologize for the three patches in 2020
-they claim asking permission would have defeated their research
-they claim the other 200 were legitimate patch attempts (a sample of which were, from my personal reading, from 'innocuous but useless' to 'slightly harmful').
Hard to believe, but plausible.
But surely this proves malice aforethought. If you interact with someone under false pretences to deliberately mislead them, then you are effectively lying to them. In fact, if you are doing so in order to receive something of value from them (such as their time reviewing your code) then it could even be seen as fraud.
Did they tell their IRB that their research involved deception? Before declaring the project exempt from oversight, the IRB should have required that the researchers answer a question like "Does your research involve interacting with people who are not aware of your project and who would act differently if they were aware?"
Until the IRB process at the university contains at least this level of protection against such ethical failings (not just in Computer Science research but across all areas of study) I think it is right for the kernel team to refuse to interact with members of that institution.
Punishing these researchers this harshly about bad manners is creating a chilling effect that will scare researchers away from evaluating potentially one of the single biggest vulnerabilities in our industry.
To be honest if anyone went around anonymously trying to merge security exploits all across well used open source supply chains and always told everyone when they were successful immediately after and helped everyone become more vigilant in code review, I would call them a public servant even if they were almost universally hated for it.
I don't think most people realize just how easy supply chain attacks are and how widely they are being exploited by very dangerous organizations.
If something does not change fast to dramatically increase the level of scrutiny we give to code contributions, it is going to get much worse.
I have gone very far in pentests. Planting malicious USB cables, modifying keyboard firmware, straight up taking unlocked laptops and walking off, sniping recently expired domains to do XSS attacks, obtaining password reset links for the email accounts of maintainers of highly depended on third party dependencies.
Even with consent from high levels at orgs it still upsets unknowing people that are tricked at lower levels in the org. I don't apologize for this because it is my job.
I have seen real and successful social engineering attacks by state actors up close and you would way prefer a security researcher with bad manners to break you of your your survivors bias over being hit by the real thing.
The reality is big companies can afford to pay people like me. Open source projects by random solo maintainers that the security of almost everyone on the internet relies on... can't.
We should be very thankful for people that risk public rebuke to do research like this in open source at minimal cost to the receiving organization.
I strongly agree that supply chain attack is a huge deal and worse attacks will come to light eventually but what should be done at the level of oss maintainers?
Like you said, I feel like Open source projects by random solo maintainers that the security of almost everyone on the internet relies on... can't do a lot of things.
What's the result of that? Lots of known bad patches being sent in, wasting everyone's time, because now every researcher finds "do they spot my intentional bugs" as an easy way to get something published?
There's no reason why you can't run these studies with their consent. Ask the maintainers. "But it will tip them off", yeah, in a "they would be aware that somebody might send faulty code at some point in the future" kind of way, which they certainly already are. To get consent, you don't need to tell them exactly when you will be sending it, who will be sending it, or what exactly it would be.
If it did put them into permanent hyper-vigilance mode, then ... goal achieved?
Maybe this helps to raise awareness, even though in reality I will do tests like this to a tiiiiny percentage of repos over the next decade.
I don't really know what purpose this would serve other than making people realize everyone got the same message, and everyone going back to being just as complacent as they were... but now we technically informed it was going to happen so it is fine now?
Do any security researchers ask permission before finding other vulnerabilities in random open source code? I have never observed this.
I feel like this type of thing should be assumed to come from whitehats or blackhats always.
I for one teach everyone around me that if they can compromise me, they can have the bitcoin private key I leave on my personal systems. They earned it, and I will see that outgoing transfer on that address as a canary.
I have been compromised a couple of times in this game when multiple people colluded, and it only leveled up my opsec and threat model. Good.
IMO we should double down and raise grants to offer as bug bounties for people that successfully get potentially malicious code past code review in majorly depended on open source projects. We need to encourage a -lot- of this behavior to train people.
So...contrary to what you suggest in your first paragraph, all you need to do is send out a handful of warning emails at the appropriate juncture. What’s the problem? For sure you shouldn’t be submitting malicious code to a project without forewarning. That's just a basic ethical no-no. Even if you were right that this kind of unethical research somehow had beneficial results, the ends wouldn't justify the means.
It will make it hard for things to improve, so millions will continue to get hacked due to supply chain attacks that almost no one is doing anything about.
At what point does the ethical fault of negligence become so great that intervening to stop said negligence without consent becomes justified? We don't have much in the way of well defined law in this area yet, and it may take a long time to catch up.
I suggest the lesser evil in the short term is to allow maintainers to suffer occasional mild embarrassment from time to time because they merged bad code. They get a valuable lesson out of the deal with no one getting hurt, no privacy violated, etc.
If a maintainers don't want to review code with the understanding it could be malicious, then they have the right to never review or merge anything.
All code contributions should be considered potentially malicious.
Now, we also don't want to see repos bombarded with bad commits either if we can help it.
Maybe a middle ground could exist where open source authors can signal a willingness to be tested once a year, and can signal if a test for this year has happened already.
I think most projects would not do this, but it might be a step towards wide consent long term if someone cares to build such a system.
In the decades before that though, there are not really any good options.
If someones house is on fire and it has the potential to burn the neighborhood down, I don't imagine you should be faulted for doing what is required to put it out.
Emergencies sometimes warrant dispensing with niceties. I think the supply chain attack risk is approaching emergency levels at this point. Negligence is the norm.
There's tons of potentially useful research that's impractical because of ethical considerations. Just think of all the incredibly useful medical research that could be done in the absence of ethical constraints!
In the end that's just tough cookies. If you can't do your research ethically then you can't do it. Period.
If you really want to help the Linux kernel, offer your time as a patch reviewer. That would achieve far more than any attempt to deceive and embarrass others with pointless "research". (I say that it's pointless because the results are antecedently obvious – of course you can deceive reviewers if you spend enough time and effort on it.)
In your ethical world, it seems that people who offer their spare time to review kernel patches are "negligent", while people who do nothing but carp from the sidelines are heroes. In my view, the situation is exactly the opposite.
That's not what happened. Nobody is asking (or is expected to ask) when they look for vulnerabilities. The researchers were trying to introduce vulnerabilities. Had they only looked at the tools that maintainers use and given those a good shake to see whether an attack vector is lurking there, everybody would have been happy.
> IMO we should double down and raise grants to offer as bug bounties for people that successfully get potentially malicious code past code review in majorly depended on open source projects.
The likely consequence of that would be maintainers shutting down public contributions because nobody has time to deal with automated submissions from scripts that are bounty hunting.
There also might be middle ground where people submit benign code that clearly demonstrates highly risky behavior the reviewer should have noticed.
Maybe I introduce code that covertly exfiltrates memory from a non sensitive address, but it is enough to understand I could have chosen a more sensitive one. Or code that simply executes a ping to my server, etc.
Anyway, the point is, intent matters.
It is different in that looking for vulnerabilities in code only costs you time, trying to introduce vulnerabilities costs the other person's time as well, and often producing vulnerabilities scales much better, e.g. you could write some tool that mails them patches, while they'd have to individually judge the merits.
> There also might be middle ground where people submit benign code that clearly demonstrates highly risky behavior the reviewer should have noticed.
I don't think so. The issue isn't so much with trying to get code reviewed, it's with wasting people's time. That they did so by trying to introduce bugs is just the icing on the cake.
If you get consent, you can have all your tests and strengthen the infrastructure, without eventually breaking the system because every other "researcher" submits bogus code every day in hopes to have one slip through and write a paper on how they proved that project x is exploitable.
I've noticed the same with general bounties. When you have a security.txt or are on hackerone etc, you'll get lots of useless automated submissions that are wasting your time. Once that happens, you'll either shut down the program, or start rejecting reports automatically which includes the possibility of a false positive.
The Linux kernel project needs and deserves the best security methods but (a) likely can't afford the people and (b) bringing more money aboard might draw unmotivated people only after the paycheck which could objectively deteriorate the project's code quality.
Dual cryptographic signoff still doesn't address any social issues (or I'm grossly misunderstanding it if it does). F. ex. a state actor can threaten or bribe two, five or thirty people into merging a harmful patch.
So what can actually be done?
Hundreds of people have blind push access to brew. Almost no one signs code. One bad commit and you backdoor every company in silicon valley that blindly executes the unsigned code from the brew repo. Pick a favorite Linux distro. All but a handful work the same way.
Right now an low skill level malicious party just needs to phish one of the ~90% of maintainers that don't have strong (or any) 2FA enabled anywhere in a dependency graph while they are on vacation or are not paying attention to a project and sneak in a commit or two into history while notifications are turned off. Attacks just like this have happened in the wild and not gone noticed for months. How many have we still not noticed?
It is insane we let it get to this point. I feel like we are in early 1900s medicine where only like 1/10000 doctors see the value in washing hands and tools.
Dual cryptographic signatures on all code from author/maintainer plus signatures by security-only reviewers from a pool not known in advance before new releases would be a very high bar for an adversary to defeat and a high risk of exposure.
I have helped deploy variations of this approach with clients/employers in the past, and will publishing more detailed recommendations along these lines publicly soon.
If the remaining weaknesses in this threat model and posture became our worst attack surface, we are in dramatically better shape.
I pushed for more security and better practices until I was fired from 3 companies. It's quite obvious that business only cares about PR; if a big security incident happens they are more interested in how to reduce liability and not how to prevent the problem in the first place [the next time around].
Until programmers become an elite clique with a huge barrier to entry and our own independent income -- completely disconnected from the interests of those who want to use our services -- then we have our hands tied and are at the mercy of the paycheck and its innate conflicts of interest with good craftsmanship.
I feel I am happier and have a wider net impact now that I can choose to invest less time with clients that can't afford to be super interested in security right now and invest more time in those that are.
Weirdly companies seem to listen to security consultants more than their own internal full time security advocates.
Having got their University banned and overturned the effort it took to write 190 previous patches from others which have now been reverted, it seems likely to me that they are under internal pressure. In all it makes me doubt the sincerity of any of this especially given the tone in the last email with the maintainer. A lot of sudden learning seems to have taken place between then and now.
Since when it is a ok to acquire data through unethical means and then publish it in a journal ?
Uhh.. but that was the exact intent of the paper. And they did it successfully. So.. mission failed successfully?
What a bizarre attempt to save-face. This is academic misconduct and they're trying to save their asses. finding and fixing security vulnerabilities.
https://sfconservancy.org/blog/2021/apr/20/how-to-apologize/
What makes you think this isn’t research?
Not justifying what they did but this apparently is still research.
I pulled this from the website of Oakland 2021
Since 1980 in Oakland, the IEEE Symposium on Security and Privacy has been the premier forum for computer security research, presenting the latest developments and bringing together researchers and practitioners. We solicit previously unpublished papers offering novel research contributions in any aspect of security or privacy. Papers may present advances in the theory, design, implementation, analysis, verification, or empirical evaluation and measurement of secure systems.
I was pointing out that this is like defining whether or not something is “art” not based upon the work itself, but instead based solely upon whether somebody bought it.
Incidentally, my recipe for apple pie was printed in a newspaper, and so therefore my recipe is “news”.
This doesn't admit to harm done, and sounds like an apology that doesn't admit guilt.
“They introduce kernel bugs on purpose” - https://news.ycombinator.com/item?id=26887670 - April 2021 (1902 comments)
UMN CS&E Statement on Linux Kernel Research - https://news.ycombinator.com/item?id=26895510 - April 2021 (313 comments)
Others?
There were months of opportunity for contrition.
They seemed a little too proud of their paper for Oakland'21, and this note comes much too late, for the message to be sincere.
And which party then -- to add insult to injury -- didn't seem to appreciate the offense, when called on it.
IMHO, to examine this incident very seriously is appropriate and beneficial. If we ease up on the casting stones, and look at the problem a bit more generally, some of the lessons learned might hit close to home.
> we are very sorry that the method used in the “hypocrite commits” paper was inappropriate.
The single mention of the IRB and lack of apology for doing human behavior research without permission of the test subjects is still problematic. I suppose being involved with an industry that does A/B testing on users might be an excuse.
Also, "any harm" should be "the harm" in a sincere apology.
Please, you've contributed enough already, really.
If that means changing jobs, most of us have been there. Looking for a job sucks, but if you screw up that badly, then you do have to consider stacking shelves in a supermarket; you're not fit to work on a public project of such importance.
> Unless the researchers are lying (which I've not seen a clear indication of), the 190 patches you have selected here are nothing more than collateral damage while you are completely missing the supposed patch submission addresses from which the malicious patches were sent! This all really sounds like a knee-jerk reaction to thier posting. I have to say, I think it's the wrong reaction to have.
https://lore.kernel.org/lkml/18edc472a95f1d4efe3ef40cc9b8d26...
Greg Kroah-Hartman’s poorly thought out bulk reversion is the bigger waste of time here.
Banned umn mail.
Because that’s going to reduce risk?
This particular act is about avoiding the risk of researchers wasting kernel developers time.
As evinced by the response, it seems really unlikely that other institutions would think it’s a good idea to perform experiments on the kernel devs in the future.
The policy is not about them being physically prevented from emailing from not-a-umn-researcher@yahoo.com.
It’s a symbolic policy, but I would be extremely surprised if a university flouted a clear and explicit ban on their participation.
So the idea is to collectively punish the mostly innocent people who wrote the 190 good commits, spending a huge amount of developer time checking them or losing the fixes to prevent other institutions from doing this?
Does reverting nearly 200 good patches seem out of proportion relative to 3 unmerged hypocrite commits?
I don’t have any special insight into the kernel team, but I think the response is as much about making an example of the university as it is about removing any plausibly contaminated commits, in which case it makes sense to be somewhat extreme.
Others are saying the entire research team should be fired.
It’s interesting that the former (reverting the commits) is something the kernel devs can do to publish the university, while the latter (firing researchers) is something the university can do to punish the researchers.
Who messed up? The researchers or the university who approved their research?
Probably enough blame to go around.
Punishing the right people is another.
If you’re an UMN patch submitter who has just seen your work thrown away because of the actions of some researchers you don’t know and some kernel maintainer you don’t know, you’d be rightly upset.
Punishing the entire university for the actions of a few. I think it’s a bad idea: https://en.m.wikipedia.org/wiki/Collective_punishment
Maybe my research is flawed, but I don't think the bulk revert is currently hitting anyone but them.
[1] - https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux...
So for example we have a 2019 commit by Wenwen Wang being reverted. Wenwen is now at UGA. The commit is “ALSA: usx2y fix a double free bug” review by Takashi Iwai confirms it’s a good fix. I don’t agree with Greg K-H threatening to trash Wenwen’s work, his reputation, and add bugs to the Linux kernel out of spite toward researchers than Wenwen hasn’t worked with in years.
Note that Aditya Pakki denies being a part of the hypocrite commits on email, maintains his denial in the apology, and has written papers on static analysis.
Also, vast majority of those commits are being re-reviewed and found good.
Greg Kroah-Hartman accused him of intentionally submitting bad commits. That accusation appears false. Greg should apologize to Wenwen and Aditaya.
This is a broad brush. It’s hitting a bunch of good commits and we know the 3 hypocrite commits aren’t there .
Assuming [3] came from a reasonably trustworthy source after Greg's initial distrust and frustration, combined with Aditya's reply of being extremely offended that anyone would doubt his work (which, if we assume for a moment he wasn't aware of the researchers' prior work poisoning LKML's opinion of the lab, could be a reasonable reaction), it's not surprising that his conclusion was "rip out all the patches for now and re-review them as we have time".
People keep overlooking that the plan was never "permanently remove the patches", it was always "remove the patches for now and then re-review them all".
And now it's working as expected - people are re-reviewing the patches proposed for removal and going "hey this is fine", "hey this is harmless", or occasionally, probably "hey this is bad". (I have not read anywhere near all the replies to the thread, I'm just assuming that the people who were complaining about the patches were also operating in good faith and not just making up complaints.)
I don't really see another reasonable way to have acted if you suspect a group of people has been generating and getting committed poor patches, whether out of malice or ignorance, than removing them for now and re-reviewing them.
If it turns out that the patches were (probably, since I don't think there's any absolute confidence to be had here) merely bad and not malicious, then sure, an apology would be warranted for claiming malice where there was none. (And for those who don't think he ever claimed malice, like I did when I started writing this reply, see [4], specifically 'Commits from @umn.edu addresses have been found to be submitted in "bad faith" to try to test the kernel community's ability to review "known malicious" changes.')
But I don't think "okay we need to re-review all these patches (and the usual thing to do if we need to re-review patches is remove them for now)" warrants an apology in itself.
[1] - https://github.com/torvalds/linux/commit/799bac5512188522213...
[2] - https://lore.kernel.org/linux-nfs/YIAta3cRl8mk%2FRkH@unreal/
[3] - https://lore.kernel.org/linux-nfs/YH+zwQgBBGUJdiVK@unreal/
[4] - https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...
In the end, we largely agree.
It is exactly the accusation of malice when what occurred was normal error that deserves an apology. You teased that out well.
As you note, we don’t have certainty. But we observe:
Aditya is not an author on the hypocrite commits paper.
Aditya withdrew his “garbage” commit when made aware it was not correct.
Aditya is author of other papers on static analysis.
Aditya’s other commits have generally survived this bulk review. Leon Romanovksy has offered no follow up for his claim of security holes nor explained his claim that the commits are part of the hypocrite commits research.
Aditya maintains his claims even as other members of UMN have explained their role in the hypocrite commits.
Aditya’s commits come from a UMN address where as hypocrite commits came from gmail.
These observations make it overwhelmingly likely that Aditya’s commits are from a good faith somewhat buggy static analysis effort.
This poor guy has worked for years on the Linux kernel and Greg Kroah-Hartman’s thoughtless rush to judgement threatened all that effort. Greg should apologize. Leon Romanovksy should too.
They mixed up and couple things and got carried away by their emotions. It happens. They should work to undo the damage.
It seems like he withdrew it in response to stumbling over [2], wherein Al Viro points out that the correct thing to do with buggy commits is to request they be reverted. (Based on, among other things, the fact that said LWN post is cited in the revert.)
> Leon Romanovksy has offered no follow up for his claim of security holes nor explained his claim that the commits are part of the hypocrite commits research.
He did offer the patch I cited as "[2]" in my prior reply, as well as pointing out [1] the commit where they reworked the buggy logic from it.
One commit does not a sinner make, but it's not correct to say he offered no data.
> They mixed up and couple things and got carried away by their emotions.
I'm not the biggest fan of GregKH for other reasons [3], but other than maybe explicitly requiring and verifying a list of broken commits beforehand, I'm not sure what I would have wanted him to do differently. If you have prior reason to suspect a group of behaving maliciously, and someone you trust attests that they appear to have behaved maliciously, what do you do differently? "I promise I'm not part of that research" doesn't work if you think the group might lie, and as I've pointed out other times, the only ones caught in the temporary patch removal were members of the relevant lab, and I further claim that "patches temporarily removed for re-examining" is not that severe a punishment.
[1] - https://lore.kernel.org/linux-nfs/YIMDCNx4q6esHTYt@unreal/
Fair point. Not sure what became of the other alleged security holes.
> "I promise I'm not part of that research"
This is a gross oversimplification of the evidence that Aditya is not malicious.
I offered a lot of evidence above that Aditya is likely not malicious just clumsy.
The lwn post you pointed to reaches the same conclusion “ I am quite certain by now that patches had been crap in good faith; the odds of that being the penetration testing, take 2, are IMO very low.”
So possible that Greg wrongly judged Aditya based on trusting Leon or other bad reasoning. Fine.
Now it’s abundantly clear the Aditya was not making malicious commits and Greg should apologize to him for the false accusation.
I agree with you that Aditya is probably not malicious, just clumsy, to be clear. But I feel like it's probably reasonable for someone in GregKH's position to have a higher threshold to clear than "probably" once the question is posed.
I also wasn't trying to claim that "I promise I'm not part of that research" encompassed all the evidence, just remark that personal attestation of innocence would not have been a useful input at that juncture.
I'm not sure, however, at what point an apology is merited now - anything significantly less than 100% of someone's patches seems like a poor choice given the small absolute number of bad patches that were used in that paper, but we're choosing by pseudorandom sample, but that's biased by how active/familiar the maintainers are with different parts of the code...etc.
So I don't know what the right confidence level should be for concluding a prior judgment of "you might be malicious" was wrong. (Maybe enumerating+examining all the "malicious" commits remarked upon by Leon, since that's what sparked the fire?)
The most recent patch, which triggered the bad, has not been explained in good faith and it seems likely there were mistruths involved in that exchange as well.
So while it is absolutely a waste of time to have to go back through those 190 commits, the responsibility for that falls squarely on the researchers who broke that trust, not on the Greg fro refusing to re-extend that trust in the face of continued sketchy behaviors.
Given how few of the 190 have turned out wrong, given the timeline of the research relative, given the existing static analysis paper. Did Aditya Pakki write two entire papers about static analysis as cover for continuing this hypocrite commit research? Or did Aditya Pakki submit a bad patch in good faith and Greg Kroah-Hartman misunderstood and then overreacted with this mass revert? I think the latter is more plausible.
Otherwise, why stop at 190? The hypocrite commits came from Gmail. Logically, we can’t trust the researchers so we need to revert all Gmail commits. Wait they could Be infiltrating other universities, we should ban university commits everywhere. That would be a lot of work. So Greg picked a medium set of patches that expressed his power and outrage but had no real effect. The point is the particular set of commits he chose aren’t risky (by inspection we know they are good) and aren’t related to the hypocrite commits. So many innocent people are having their work thrown away because Greg is angry.
Note that we have only seen the simple to revert patches. There are 68 complex umn patches that have not been reverted. I expect they won’t because this is mainly theater. The commits are good but Greg wants to be show how angry he is.
My impression is we are just living through some kernel maintainer lashing out because he was embarrassed by the failed review process and mixed up the buggy static commits with the hypocrite research. Admitting you’re wrong is hard, so I don’t expect Greg to do that.
So even if all the the commits are good and even if Greg believes that, he has still been forced into these actions by the violations of trust committed by the researchers.
You are correct that the line as to what patches is a bit arbitrary, but it was always going to be. The research was conducted by the University so it seem the most reasonable place to draw the line to me.
I have still yet to see any explanation from the researchers as to why that "bad patch" happened, didn't include a reference to the automated tool as required, or what tool it even was.
I do think Greg is angry, but I don't think your insistence asscribing Greg's anger as his primary motivation is fair to him or consistent with HN guidelines on comments. I think you could make points about reverting commits without what seem to me like unnecessary personal attacks.
The Linux kernel team had to revert changes in over 200 files. They will also have to go back and review hundreds of commits to make sure they do not contain any malicious code.
The technical effort in addition to all the time spent dealing with the backlash of this is a waste of time for a lot of people doing important work that benefits everyone.
So, what else can be said... Play stupid games, win stupid prizes. To Mr. Lu, Wu, and Pakki: enjoy your ban and worldwide press coverage.
No, they didn't have to. That seems like quite a bit of an over-reaction by mr Kroah-Hartman.
Can someone please explain why someone who is working with others on a common goal for years suddenly conducts a breaching experiment? What were they thinking?
(Also, there must be experience within Linux about malicious commits, three-letter agencies from all over the world must have tried before. Why not try archeology?)
First, you just apologize for what you did and that's that. Ball is in the other side of the court.
I haven't seen the message from the Linux Foundation to the university- assuming that's private and I didnt miss it somewhere?
I haven't done kernel work since BSD4.3 so my opinion isn't particularly interesting. With that said, I would agree that the researchers were naive and their approach has caused collateral damage. However, I'd also argue that the (apparent) topic of research is quite valid. There are a lot of extremely well funded adversaries who are quite talented at obfuscation. "Be careful out there."
The main factor in doing this ethically is obtaining consent from your research subjects. I believe that some Red Teams test precisely these sorts of security mechanisms in commercial software projects and there are solid comments else discussing procedures they use.
For you security people out there, how do red teams handle this issue?
Basically, warn in that you’re going to do it at some point in the future. Then do it in a time-frame of maybe 6 months or so from another ID. Maybe get some of the higher-ups to sign off (say Linus or Greg in this case) beforehand. At the very least, engage with the community...
> Thank you for your response.
> As you know, the Linux Foundation and the Linux Foundation's Technical Advisory Board submitted a letter on Friday to your University outlining the specific actions which need to happen in order for your group, and your University, to be able to work to regain the trust of the Linux kernel community.
> Until those actions are taken, we do not have anything further to discuss about this issue.
Was this letter publicly released?
The commission / board that approved this experiment will never suffer any consequences.
And IMO they are just as guilty. They didn't do their job well.
https://itwire.com/open-source/torvalds-says-submitting-know...
—-
Thanks for sharing.
The statements in the link are wrong. I will make some clarifications.
1. We have never intentionally introduced any bugs in Linux. 2. The project that investigated the issues with OSS patching process was done in November 2020. We did get an IRB letter for the research. The purpose of the research is to improve the security of the OSS patching process. 3. One of my students is working on a different project which has nothing to do with the project mentioned in 2. The student aims to fix problems in Linux instead of introducing problems.
If you are interested in the project mentioned in 2, please find more details here: https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc....
I hope these clarify the misunderstandings.
Kangjie Lu Assistant Professor Department of Computer Science and Engineering University of Minnesota https://www-users.cs.umn.edu/~kjlu
—-
- consent by selected few
- buy-in on all major parties
- documentation trail, ongoing
I’m sure there’s more but ...
A rather vague and strange way to put it. When did the Linux community adopt the language of safteyism?
It's a 190 vs 3 bad, against ~50 vs 143 bad.
Let's hope the maintainers can correctly identify the good ones, now with the given hints (August 2020 only). Or if they still keep staying in witchhunt mode. Word against word. Some of the latest patches do look wrong, but I'm not an expert. Analysis will need a few weeks.
One does not conduct security research or experimentation on mission critical infrastructure without the informed consent of the persons maintaining or responsible for said infrastructure.
The stark difference between white (or even gray) hat and black hack security operations is whether those operations are done with the informed consent of the owners of the system(s) being manipulated. To knowingly and deliberately attempt to manipulate critical infrastructure without the informed consent of the persons responsible for that infrastructure is, in my eyes, clearly unethical black hat hacking. It does not matter how many resources were expended nor does it matter if there was no personal injury or destruction of property. A quiet, relatively uneventful outcome of a black hat operation is nothing more than a dose of good luck.
There are many other circumstances in which this sort of clandestine operation would be obviously unacceptable. Consider if a researcher wished to surreptitiously tamper with vaccine supply chains, automotive or aircraft supply chains, fuel distribution supply chains, chemical manufacturing supply chains, etc. Even if the researcher claimed to have the best of intentions, and had a written plan to carefully back-out or otherwise reverse (or prevent from going to production) the intentionally flawed manipulations or modifications of critical supply chain components, I think very few people would consider the research ethical. There are an enormous number of issues, defects, and mishaps that occur even when a group of people tries their best to implement and maintain critical systems. When a clandestine operator decides to interfere, using whatever justification, then reliability of the system can be expected to suffer.
I do not believe the written apology addresses the underlying problem, which seems to be that the University of Minnesota does not exercise sufficient governance over the researchers in its employ.
In fact the argument could be made that there is an ethical duty for every software project, and possibly every individual, to economically boycott the University of Minnesota until they put in place independently-verified changes to their IRB process such that research like this will not be approved again.
If the only "cost" to the university is that one of its researchers has to write a two-faced apology message, then there is nothing stopping future abuses of non-consenting humans. Moreover, with no real punishment meted out, it creates an incentive for other universities to similarly weaken their IRB process to the same level so that they can approve more research projects and remain competitive.
and that's fair! (also completely unrelated to what i said.)
and has nothing to do with individual word choices that they're being picked apart on, which would be subtle to even non-college educated english speakers.
> And since the researchers clearly have the ability to write papers in major English conferences this doesn’t seem to be a problem.
you're reading different conference papers than i, native-level english is not generally a requirement.
and even the letter was heavily reviewed, the moral thing to do is to give a professor and two grad students leeway in interpretation of their english. (i'll grant that their past leeway in interpretation of their actions, but those aren't the same.)
that doesn't mean it can't be a bad apology, it doesn't mean you have to accept or like it. just, come on, find more substantial issues than their adjectival choices.
Whatever the reasons, this open letter is a joke.
This, and the previous mail that I found pretty insulting too and seemingly written in bad faith but I'd give the benefit of the doubt, one can react badly to a difficult situation.
But what’s the appropriate proportional response?
I honestly have a hard time thinking firing a bunch of people over this would be a good outcome for anyone.
What’s the end goal? I feel like already there’s been enough action to deter this kind of sneaky research in the future.
But at some point you’re just going to scare away the ethical researchers too.
Determining how to conduct your research ethically is a core part of being a researcher. Basic research into pen testing and security research would have revealed how unethical this study was.
I see firing as completely justifiable given that was a failure in a core part of thier job, that not only caused a public relations mess, but has directly negatively affected other members of the research institute because they have been banned from Kernel contribution.
What further incompetence do you think is required to justify firing this academic?
Yes, only if it's unrelated to their jobs. People get hired, not fired, to do unethical research in industry labs. Ethics is breached all the time in industry - rarely even considered. Google tried to make amends, but decided caring too much about ethics was a roadblock to their goals. Tesla markets their "FSD" at the risk of other drivers in the road. The only difference here is that companies have a bigger, better legal and PR team. Comparing the moral compass between academia and industry is absolutely ridiculous.
If you are trying to argue that publicly funded research institutions don't think ethics are a core part of research, that seems like something that needs to be addressed.
And to be more precise, ethics IS a core part of research. There's an entire subfield dedicated to it in academic labs. Graduate research is very specialized, and researchers do not have the intuition for a difficult topic outside of their expertise. Then there are IRBs, who are most qualified, and yet we see it remains a problem for computing ethics because it's a completely different environment.
We can continue to blame this lab, fine. I agree, huge ethical blunder. But don't forget that this passed IRB, a responsibility of the university, and several phases of anonymous peer review for a major security conference. The security conference that most definitely has reviewers from industry. Was there a lapse of judgement on ALL the scientists and engineers, all with immense experience in their respective fields, who touched this paper and participated through the entire process? Do you seriously believe that? Should they all be fired? Is it really hard to believe that computing ethics is actually not as clear, communicated, and well understood to MANY? There are so many caveats, and not much policies of conduct exist for the breadth of situations.
[1] https://www.theatlantic.com/technology/archive/2014/06/every...
In each case, a vulnerability was disclosed. With Linux being used everywhere you can be sure intelligence services etc are likely sending in bad patches. If the Linux kernel requires only patchers with good intentions, and doesn’t have other means of catching this stuff, we are screwed.
There's a big ethical difference between trying to exploit a piece of commercially produced software, and trying to exploit the time and actions of humans who are producing software which is given away for free.
I don’t think the free vs commercial aspect is the main issue here; lots of kernel devs are paid for their work after all...
You're right that the OP's analogy breaks down if the researchers were unsuccessful in getting their malicious patches accepted, because then there is arguably no vulnerability to report, and OP said "when security vulnerabilities are disclosed".
Steelmanning that analogy, though, what the researchers were doing was "probing for possible vulnerabilities", which some vendors also complain about, especially if the target is an online service rather than software running locally on the researcher's machine.
In that case, the main flaw in the analogy is still the difference between exploiting software and exploiting humans, so I probably should have focused on that. Nevertheless, there is a small ethical difference in some circumstances between software that is bought and software which is freely downloadable (regardless of the licences involved), since if you paid for something which is defective then you might deserve a refund.
They’re not sweeping issues under the carpet. They are actively addressing those issues.
They then published a paper and thought that this was acceptable behavior from a research/ethical standpoint.
OSS security should not rely on ethics and trust. The default assumption should be that malicious actors are using every dirty tactic they can to get their changes accepted, and that someone@something.edu is no more trustworthy than internal.security.agency@china.gov.cn.
Feels like the OSS community got caught with their pants down and is now overreacting.
So much incredulity and negativity in the comments here. The indignation is totally counter productive.
The apology is fine, folks. Just accept it and move on.
Maybe we can find or make up something unrelated but unsavory about these characters objecting to those pissing in the font of Linux... That's the normal way these things are handled now, right?
(scorching sarcasm intended)