Banned umn mail.
Because that’s going to reduce risk?
This particular act is about avoiding the risk of researchers wasting kernel developers time.
As evinced by the response, it seems really unlikely that other institutions would think it’s a good idea to perform experiments on the kernel devs in the future.
The policy is not about them being physically prevented from emailing from not-a-umn-researcher@yahoo.com.
It’s a symbolic policy, but I would be extremely surprised if a university flouted a clear and explicit ban on their participation.
So the idea is to collectively punish the mostly innocent people who wrote the 190 good commits, spending a huge amount of developer time checking them or losing the fixes to prevent other institutions from doing this?
Does reverting nearly 200 good patches seem out of proportion relative to 3 unmerged hypocrite commits?
I don’t have any special insight into the kernel team, but I think the response is as much about making an example of the university as it is about removing any plausibly contaminated commits, in which case it makes sense to be somewhat extreme.
Others are saying the entire research team should be fired.
It’s interesting that the former (reverting the commits) is something the kernel devs can do to publish the university, while the latter (firing researchers) is something the university can do to punish the researchers.
Who messed up? The researchers or the university who approved their research?
Probably enough blame to go around.
Punishing the right people is another.
If you’re an UMN patch submitter who has just seen your work thrown away because of the actions of some researchers you don’t know and some kernel maintainer you don’t know, you’d be rightly upset.
Punishing the entire university for the actions of a few. I think it’s a bad idea: https://en.m.wikipedia.org/wiki/Collective_punishment
Maybe my research is flawed, but I don't think the bulk revert is currently hitting anyone but them.
[1] - https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux...
So for example we have a 2019 commit by Wenwen Wang being reverted. Wenwen is now at UGA. The commit is “ALSA: usx2y fix a double free bug” review by Takashi Iwai confirms it’s a good fix. I don’t agree with Greg K-H threatening to trash Wenwen’s work, his reputation, and add bugs to the Linux kernel out of spite toward researchers than Wenwen hasn’t worked with in years.
Note that Aditya Pakki denies being a part of the hypocrite commits on email, maintains his denial in the apology, and has written papers on static analysis.
Also, vast majority of those commits are being re-reviewed and found good.
Greg Kroah-Hartman accused him of intentionally submitting bad commits. That accusation appears false. Greg should apologize to Wenwen and Aditaya.
This is a broad brush. It’s hitting a bunch of good commits and we know the 3 hypocrite commits aren’t there .
Assuming [3] came from a reasonably trustworthy source after Greg's initial distrust and frustration, combined with Aditya's reply of being extremely offended that anyone would doubt his work (which, if we assume for a moment he wasn't aware of the researchers' prior work poisoning LKML's opinion of the lab, could be a reasonable reaction), it's not surprising that his conclusion was "rip out all the patches for now and re-review them as we have time".
People keep overlooking that the plan was never "permanently remove the patches", it was always "remove the patches for now and then re-review them all".
And now it's working as expected - people are re-reviewing the patches proposed for removal and going "hey this is fine", "hey this is harmless", or occasionally, probably "hey this is bad". (I have not read anywhere near all the replies to the thread, I'm just assuming that the people who were complaining about the patches were also operating in good faith and not just making up complaints.)
I don't really see another reasonable way to have acted if you suspect a group of people has been generating and getting committed poor patches, whether out of malice or ignorance, than removing them for now and re-reviewing them.
If it turns out that the patches were (probably, since I don't think there's any absolute confidence to be had here) merely bad and not malicious, then sure, an apology would be warranted for claiming malice where there was none. (And for those who don't think he ever claimed malice, like I did when I started writing this reply, see [4], specifically 'Commits from @umn.edu addresses have been found to be submitted in "bad faith" to try to test the kernel community's ability to review "known malicious" changes.')
But I don't think "okay we need to re-review all these patches (and the usual thing to do if we need to re-review patches is remove them for now)" warrants an apology in itself.
[1] - https://github.com/torvalds/linux/commit/799bac5512188522213...
[2] - https://lore.kernel.org/linux-nfs/YIAta3cRl8mk%2FRkH@unreal/
[3] - https://lore.kernel.org/linux-nfs/YH+zwQgBBGUJdiVK@unreal/
[4] - https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...
In the end, we largely agree.
It is exactly the accusation of malice when what occurred was normal error that deserves an apology. You teased that out well.
As you note, we don’t have certainty. But we observe:
Aditya is not an author on the hypocrite commits paper.
Aditya withdrew his “garbage” commit when made aware it was not correct.
Aditya is author of other papers on static analysis.
Aditya’s other commits have generally survived this bulk review. Leon Romanovksy has offered no follow up for his claim of security holes nor explained his claim that the commits are part of the hypocrite commits research.
Aditya maintains his claims even as other members of UMN have explained their role in the hypocrite commits.
Aditya’s commits come from a UMN address where as hypocrite commits came from gmail.
These observations make it overwhelmingly likely that Aditya’s commits are from a good faith somewhat buggy static analysis effort.
This poor guy has worked for years on the Linux kernel and Greg Kroah-Hartman’s thoughtless rush to judgement threatened all that effort. Greg should apologize. Leon Romanovksy should too.
They mixed up and couple things and got carried away by their emotions. It happens. They should work to undo the damage.
It seems like he withdrew it in response to stumbling over [2], wherein Al Viro points out that the correct thing to do with buggy commits is to request they be reverted. (Based on, among other things, the fact that said LWN post is cited in the revert.)
> Leon Romanovksy has offered no follow up for his claim of security holes nor explained his claim that the commits are part of the hypocrite commits research.
He did offer the patch I cited as "[2]" in my prior reply, as well as pointing out [1] the commit where they reworked the buggy logic from it.
One commit does not a sinner make, but it's not correct to say he offered no data.
> They mixed up and couple things and got carried away by their emotions.
I'm not the biggest fan of GregKH for other reasons [3], but other than maybe explicitly requiring and verifying a list of broken commits beforehand, I'm not sure what I would have wanted him to do differently. If you have prior reason to suspect a group of behaving maliciously, and someone you trust attests that they appear to have behaved maliciously, what do you do differently? "I promise I'm not part of that research" doesn't work if you think the group might lie, and as I've pointed out other times, the only ones caught in the temporary patch removal were members of the relevant lab, and I further claim that "patches temporarily removed for re-examining" is not that severe a punishment.
[1] - https://lore.kernel.org/linux-nfs/YIMDCNx4q6esHTYt@unreal/
Fair point. Not sure what became of the other alleged security holes.
> "I promise I'm not part of that research"
This is a gross oversimplification of the evidence that Aditya is not malicious.
I offered a lot of evidence above that Aditya is likely not malicious just clumsy.
The lwn post you pointed to reaches the same conclusion “ I am quite certain by now that patches had been crap in good faith; the odds of that being the penetration testing, take 2, are IMO very low.”
So possible that Greg wrongly judged Aditya based on trusting Leon or other bad reasoning. Fine.
Now it’s abundantly clear the Aditya was not making malicious commits and Greg should apologize to him for the false accusation.
I agree with you that Aditya is probably not malicious, just clumsy, to be clear. But I feel like it's probably reasonable for someone in GregKH's position to have a higher threshold to clear than "probably" once the question is posed.
I also wasn't trying to claim that "I promise I'm not part of that research" encompassed all the evidence, just remark that personal attestation of innocence would not have been a useful input at that juncture.
I'm not sure, however, at what point an apology is merited now - anything significantly less than 100% of someone's patches seems like a poor choice given the small absolute number of bad patches that were used in that paper, but we're choosing by pseudorandom sample, but that's biased by how active/familiar the maintainers are with different parts of the code...etc.
So I don't know what the right confidence level should be for concluding a prior judgment of "you might be malicious" was wrong. (Maybe enumerating+examining all the "malicious" commits remarked upon by Leon, since that's what sparked the fire?)