The one positive of this environment that when you see someone who has succeeded in publishing a substantial amount, it's safe to presume that their work is not entirely without merit. That's not a 100% guarantee, but it's a good heuristic. A post like this implies that the author's personal judgement is more insightful than a process that I know to be enormously thorough and harsh, and yet it provides only vague assertions and no details on methodology. At the same time it's hard to ignore that the post is written by someone who is (understandably) a bit bitter and probably isn't being entirely objective in their evaluation.
In general it's not my impression that this sort of post produces reliable feedback that outsiders should rely on, and (unless it's very carefully anonymized, which it isn't) it also leaves bitterness in its wake.
Fascinating, as I did not find this at all to be the case in general. I imagine in some cases it could be true, but for the most part I found most research in academia to not be of much interest to many people, one way or another. Hardly anyone takes much time to either validate research or to critique it.
I'm glad to hear that you managed to find something to research that caught the attention of many people/mechanisms, even if only to judge it negatively. It must mean you were doing something important, which I think is the exception and not at all the norm.
Of course you can argue that all this academic competition selects for the wrong thing (that's subjective) but the competition and testing is never-ending. There is a reason that a lot of people leave for industry, simply because it's so much less stressful.
This has not been my experience. In my own field, I've found that quite a few well-cited peer-reviewed papers have severe flaws that are not difficult to find. I can't call the process "enormously thorough and harsh". I view current academic processes as more of a facade than anything else. Sure, the process probably filters out absolute garbage, but I don't think most people need peer review to recognize the worst quality research.
My point wasn't a complaint about a few papers, rather, important papers that presumably have had a lot of eyeballs go over them in addition to being peer reviewed. If those papers have serious flaws, then I think that does say something about the standards in academia broadly.
One reason why I don't work in academia is that too many academics resist the idea that there is a quality problem.
Also: I didn't say that my field was broken or anything like that. It's not optimal, but for the most part it's okay. That has little to do with peer review.
> someone who has repeatedly survived a publication competition with a ~20% success rate and then got other serious researchers to cite and build their work is probably not a total piker
Sure, it's evidence that the work is good, but it's not particularly strong evidence in my view.
> whatever your impressions may be from the sidelines.
If you're implying that I haven't published in a peer-reviewed venue myself, you're mistaken. I've never had a problem publishing. I think that's because I agree more with Gauss: "few, but ripe". Unfortunately "publish or perish" reins supreme in academia, which probably explains why so many people have trouble as they seem to try to produce "least publishable units". Borderline papers like that of course will have trouble.
The most effective thing I can do is encourage you to find problems in published papers yourself.
Another factor in my reluctance is that explaining each case would take a significant amount of time. (Brandolini's law, basically.) I can point you to places where I already explained problems I've found, for example: https://pubpeer.com/publications/95455FA4147A9CBD5EAA5185D21...
That paper has over 300 forward citations over about 25 years. It was published first in a conference, and later in the top specialized journal in my sub-field. I went through a large fraction of the forward citations and could not find anyone who noticed the major error I point out. And it's not difficult to spot if one does basic spot checks.
You are also unhappy that you can’t get a response from the first author [initially I read their Google scholar as indicating they’d left the field, but they haven’t and that was my mistake reading Google Scholar in Korean.] Meanwhile you have not contacted the third (typically senior and advisory) author, who appears to have publications as of 2022?
Yet even if you do the basic legwork here and contact all the authors, I’m not sure quite what result you want from this? I presume you’d like an acknowledgement that an old paper has errors? You’ve already received that through the acceptance of your own paper. Would an admission by the third author help? Do you demand a retraction of that ancient paper and every single paper that cited it since? And how is this the devastating critique of all science that you seem to think it is?
My link was just one example. As I indicated, I don't have time to give more examples, and I didn't expect to convince you. I think few academics are actively looking for errors, so unless you are, you may not notice them as often as I have.
> I can only assume that very few people were relying on this stupid model, since the flaw wasn’t noticed for decades.
There are quite a few papers that directly use the criticized model, from the 90s until recently. It does seem that many people blindly accepted a clearly wrong model. With that being said, the other model in the paper is responsible for most of the citations.
> You are also unhappy that you can’t get a response from the first author [...] I’m not sure quite what result you want from this?
The first author is the senior author here. (The conference paper version does not have the other two authors.) I contacted them because if I'm wrong, I wanted them to tell me how so that I could avoid making a fool of myself in my paper. I don't want their paper to be retracted, though a notice that it has severe errors would be helpful to readers.
I know a few people who have PhD's that had fantastic support, counselors and teachers, and they still had questions on the validity of their degrees and processes. Here, this person's experience seems to be a money grab by a school unqualified to provide him a PhD in his area.
Going public though, maybe a hard decision, and may hurt him, but who knows? Maybe there are people out there that want someone that is willing to stand up like this publicly?
{Takes on Quantum Computing grad student} + {has insufficient Quantum Computing expertise in group to even advise} seems... pretty bad.
PS: And being from a tenured family and having grown up exposed to academia, I'm inclined to take author at their judgement here, given that taking on students in hot topics meshes with how the incentives of grant funding, publishing, and prestige work.