We do know since the researchers have told the linux community, the university and IEEE what those patches where. Please do not spread further misinformation about the case.
Please read the IEEE statement and the full Linux TAB review.
https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_St...
Maybe "believe" would be better used here? Knowing would mean to know precisely what each of these changes does and whether they open up new vulnerabilities and then having confidence that all is well. Gaining this confidence requires work. And unless you put that work in, you are left to trusting/believing.
(Edit: what I mean here is that the researchers could be totally nice and ethical people, but the Linux devs would still have to either take a risk by trusting them OR put in the work to check it all OR decide not to put in that work)
maybe the issue is we aren't using static analysis enough / don't have enough static analysis tools?
A thorough review of the IRB documents revealed potential problems in the description of the experiments, and concluded that insufficient details about the experimental study were provided to the IRB. [0]
[0]Read the above linked IEEE response latter.
> Investigation of these patches revealed that the description provided by the authors in the paper is ambiguous and in some cases misleading. The experiments do not provide convincing evidence to substantiate the main claims in the paper and the technical contributions of the work need to be revisited.
Interesting that they list ethical considerations added to the review process, but are not adding content quality considerations to the review process. I think that's at least as embarrassing to PC. You can say that they erred in assuming the uni covered the ethics review, but what are they doing if they're accepting papers without checking that the papers support their own claims?
This makes them invalid. Their entire claim is based on malicious code entering the kernel, not anonymous/fake name commits (which is a separate issue). If you take that away there is no actual paper, just a hypothesis which everyone already knew and thus does not warrant this much fanfare. They added nothing to the research field, bothered people with it while contributing to no-one but themselves.
This is what we around here now call "Diederik Stapelen" (since he is a infamous example here): faking you data, using people to do/in your work, presenting it as true and gaining from it. It is omnipresent in some fields and in my opinion should be met with severe repercussions as it damages everyone else who was not part of it.
> Based on the overall positive reviews and the recommendation of all reviewers to accept the work, the PC did not discuss this paper during the online PC meeting; [...] When, after acceptance, the authors tweeted the abstract of the work in November 2020, several people expressed concerns about human-subject research featured in this work. At that time, the PC chairs discussed these concerns [...]. As a result of these discussions, the PC chairs asked the authors to clarify the experiments with the University of Minnesota Institutional Review Board (IRB). We now acknowledge that this offer was a mistake
Basically: they did not review it and once it was published and people complained, they reacted. What's the point of the PC then?
- Researchers submitted paper to IEEE.
- Researcher twitter about it.
- Tweet was deleted, because people pointed out it was bad humans subject researcher. (consent and deception)
- Other researchers not from UNM, filled complaints to IEEE.
- Researcher mislead (so far seems like) IRB, arguably IRB failed to do a job and just rubber stamped human subject research exemption, after research was conducted ...
- Paper got accepted to IEEE.
- Researchers push more patches to Linux kernel.
- Plonk email from Greg.
- UNM response latter indirectly blaming only researchers but not IRB.
- Paper get retracted from IEEE
- IEEE Response letter.
- We are here.
The fact that they didn't communicate _clearly_ with the kernel about exactly which patches this was about at the time when they announced their paper is extremely icky, and makes my sympathy for later misunderstandings/misinformation very limited.
The researchers conducting the research thinking this was a good idea, the IRB review process at the UMN, IEEE accepting the IRB exception after ethical concerns where raised, and the overreaction from gregkh on the LKML.
Hopefully there is a silver lining and we see better research collaboration between the kernel devs and researchers going forward. IEEE has a job to do around all of this going forward.
I don't think it was an overreaction. I think it was a very valid reaction.
Here we saw the emperor go on a full war path against the entire place of origin of a couple of misguided individuals. Not a proportionate response.
The IRB is a gatekeeper, yes, but it is also a resource. You should be working with the IRB to make sure everything you are doing is above the board, because ultimately, if you do something unethical or harmful to individuals or society, that's still on you, even if you got your plan stamped by the IRB. You shouldn't have an antagonistic relationship where you use the vague and technically accurate but misleading descriptions of your research an an attempt to get a "get out of consequences free" card.
Which suggests an entirely new line of inquiry: testing IRB procedures to see whether unethical proposals will be approved if described in misleading terms.
While we're at it, we should check to see whether proposals ostensibly from people who are not faculty or students, or who perhaps don't even exist, are ever approved.
Regardless of the results, we can conclude that having to sign a statement promising to never do unethical research before submitting a proposal, and insisting on identity verification before proposals are reviewed, will lower the chances of unethical research proposals slipping through.
However yes, review of the patches was proper but all this could have been done with less unfounded accusations from gregs side.
Really curious who makes unfunded accusations ...
And the resulting conversations between Aditya Pakki and Greg. Aditya was never part of the hypocrite commit research, accusing them for this is just bad.
Greg's concerns proved to be overblown, but at the time they were raised he had reason to believe them valid.
At the time, UMC had not "debriefed" the kernel maintainers they lied to in the "hypocrite commits".
And then Pakki didn't mention that these new patches were generated by a tool, and submitted a bunch of "nonsense" patches with no explanation.
What reason did GKH have to trust anyone associated with the "hypocrite commits" at that point?
(ps. don't think you deserve all the downvotes here...)
Yes, they are bad patches and the research is also questionable. But this is not bad faith nor malicious.
Do you want to take a moment and explain how your broken static anylizer isn't involved?
Or, y'know. Click on the profile.
I did click your profile, but this is a post about deception. Trust no one.
They lied to their IRB.
What evidence do you have that they're telling the truth now?
Replace researchers with hackers, CIA, North Korea, China, etc. Do you still have the warm fuzzies?
The fact of the matter is they broke the trust and your expectation is that since they've been exposed they can be trusted again?
No, if anything the event has shown that additional vetting and layers of scrutiny might be needed to protect against bad actors in the future should they be malicious.
* I type good.
We should just trust the liars when they pinky swear to tell the truth? There's a certain saying about what you win when you play shitty games that comes to mind. Being deceptive causes people to lose trust in you.
Or, my personal favorite: fuck around, find out. UMN fucked around, now they're finding out.
Let's assume that most people are giving their opinions to their best knowledge. We shall be careful when telling someone to stop spreading misinformation as this is how fascism starts. "I am right, you are wrong, stop talking!"
Throwing around accusations of spreading misinformation to further your opinion in an argument makes it harder to call out real misinformation.
And for full disclosure, I'm one of the four authors of the original complaint to IEEE back in December about the research. I fully believe all the facts have been put forth and there is no reason to spread misinformation about the incident.
They wanted to prove that others are too trusting, they got exactly what they wanted: heightened suspicion to things that are normally expected to be done in good faith.
I understand that inside the academic world the status imparted by "researchers and students at a University" is significant and important. For those of us outside the academic world, it's prudent and has no downside for us to drop our perception of them below the floor you'll continue granting them.
But season this old recipe heavily with some “fuck around and find out” and things get pretty spicy.
[1] https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_St...
And pointless?