Citation cartels help mathematicians-and their universities-climb the rankings
science.org
science.org
As a semi-related phenomenon, I also feel like mathematicians are culturally prone to under-citation. I don't just mean that nobody cites older mathematicians like, say, Leibniz or Euler, for techniques they invented. But pretty frequently I'd read papers that really seemed like they were heavily indebted to a paper that came only a few years before but didn't appear in the 5-10 citations. Maybe there it wasn't mandatory to cite those papers, but it made it hard to chain backwards into the literature to get more context on methods and ideas. And sometimes it came off as the author trying to make a tool appear ex nihilo. I'm sure this happens in other fields too.
This shows the authors are familiar with the field, avoids inadvertent plagiarism, allows them to make it clear precisely how their contribution contributes to knowledge, shows people from funding bodies that this is a cutting edge and important field of research, and points readers in the right direction if they want to see earlier ideas, or ideas in a more applied context.
If among those 92 citations should be a few papers from your boss and your colleagues, nobody will see that as unusual - as long as they're at least marginally relevant.
I would in fact argue that the trend, in "applied" fields, of justifying the importance of a piece of work by pointing out that a lot of people are doing similar work is in fact self-fulfilling. That makes it somewhat useless as a measure of importance.
Scientific context should be critical, not just descriptive.
There is one filter on this, though: you have to get funding for the work, which is a different process than getting a paper accepted. There is of course some overlap, as "people are working on this so it must be important" can be useful in arguing for funding, but funders at least potentially have other criteria in play too.
When you write your paper you say exactly this in the introductory context. Here is what other people have done, here is a problem not yet solved in this area, we solve it. It saves the reader an enormous amount of time by not having to do a literature review themselves to see how the paper fits into the larger thrust of the field, and what the novelty is.
How can you justify the novelty other than by comparing to other people's work?
Which claim do you disagree with? The claim that the trend I describe exists, or the fact that it is bad?
Note that what you describe is what I advocate: you explain that the question exists and hasn't been answered. This is not an argument that is based on the volume of work that exists.
It is also an argument you cannot (as an author) be trusted to make. Even if you cite everything that has been published that is tangentially related to your claimed contribution, there is no way a reviewer will know all of it, and no way a reviewer will be able to go and read all of it. So they can't determine whether your claim of novelty is correct unless they already know the entire field. The only defense against this is to encourage crisp and clear descriptions of claimed contributions (to knowledge or practice) and violently reject any overinflated claims. It is not to include an entire survey paper in the introduction of every piece of work that pushes the state of the art forward.
It does mean that the average paper is less accessible to the non-expert. It also encourages the regular publication of surveys whose role is solely to critically and exhaustively compare recent advances, and of textbooks whose role is to describe historic developments and their context. This is not something every paper should be doing.
So while I agree that generally citations are a statement of claim and the reference given allows people to see if that statement of claim is actually supported, within the paper itself good authors also explicitly weigh the "truth value of a proposition".
People do this in science, politics, and church. It’s a call to act only (or mostly) on that person’s testimony. An act of faith. It’s both a red flag to watch for and a necessary aid for people.
A red flag because it’s often a sign of fallacy. I default on looking that person up along with folks that might disagree with them. Every success adds credibility. Failures mean I’ll default in a different direction if I hear their name come up. Might even dig into what other claims they have which are spreading.
Those sources who keep getting the job done in specific areas can graduate from red flag to reliable enough to default on. Only in that area for claims not to far from what they’ve been good at. If outside or a big claim, still double check. Also, be able to provide verifiable evidence of why you trust the source.
Edit: So, if anyone wants to build on Fermat's last theorem, it's not enough that they cite Wiles' paper proving it, but they should have to re-derive the proof themselves?
I think humans operate on faith in others’ testimonies way more than actual science. As a follower of Jesus Christ, this doesn’t bother me since I believe we learn the truth in multiple ways. However, science is rooted in empiricism which says any belief must be proven, independently reviewed by skeptics, and we keep using it while it seems correct. Dissent is always allowed in real science since our models (beliefs) might be useful but incorrect on some level. Whatever survives the most review with the most utility is what we should keep using.
What happens is often different. People believe something just because it was reported somewhere or had a name on it. That’s pure faith (or argument from authority). Many works aren’t just unreplicated: they can’t be replicated. Many institutions have political biases affecting their work. Many don’t allow dissent on some topics. People good at one thing often fail at another. I also see big jumps in extrapolating a small success to future, larger successes.
While common (even on HN), none of that is the scientific method in action. It’s more like a mix of science, unproven dogma, speculation, political maneuvering, and a popularity contest. The ratios of these vary by field and institution. Don’t call anything science or “proof” unless it fully follows the scientific method. And builds on premises that did, too.
There was a chemistry colloquium that I went to occasionally, partially because I had some friends in the chemistry department who were into the same quantum chaos stuff I was into, but it was clear that a certain faction of the chemistry department showed up there and the rest of them went to other talks.
The math department didn't have any lecture series which was of broad interest to the department.
Physics brought in senior people who had something interesting to say at the colloquium, but more than half of the people the CS department brought in for their colloquium (which I attended a lot later when I had a software dev job at the library) were job talks and often the people did not know what they were talking about and it could get embarrassingly bad. I was lucky to see Geoff Hinton speak before he got famous but there were times I would team up with "the meanest physicist in the world" (for real, not my evil twin) to ask questions at the end that would call out the weakness of the speaker because the audience was just not holding therm accountable.
I never made this connection, but it makes so much sense.
The truth is that almost all papers published across almost all fields cannot be replicated, full stop. For those that can, the results rarely match up, indicating that the tests are chosen to boost the performance of the method.
If you want to boost citations legitimately, purposefully leak your own paper to get past the publisher pay wall, make it colourful and accessible (if it's too dense they are unlikely to understand it), give examples on how to replicate your method, the abstract is likely the only part that will be read so make it count.
This is not true for Mathematics.
The fact is that these pure mathematicians could almost surely write the more applied papers but don't because it would feel to repetitive. You don't need to read Euler's work in order to indirectly benefit from it. It's the same for a lot of pure math.
(That said, performance measurements in most papers are complete junk. Most code written by academics is missing trivial, obvious optimisations which could completely change any comparative benchmarking results.)
There was that time I got a C program from a famous ML researcher, tried to run it and it segfaulted, I pointed gdb at it and realized it was crashing before it got into main(). Understanding why was an educational experience (don't allocate a 8 GB array you never use on a 32-bit machine... It worked for the researcher because he had a 64-bit machine) but it did not increase my faith in the code quality of academic projects.
Some CS researchers are brilliant programmers but the fact is that ordinary devs make a living writing code and CS researchers make a living writing papers.
It's a sign of poor quality code.
He probably was using the array for something at one point, quit using it, and never bothered to delete it.
And of course it is C so there are no unit tests despite it being the kind of code that is eminently unit testable (if it wasn't written in C, not like you couldn't write unit tests for C but who does?)
I agree with this, but I would pose it the opposite way. Academics shouldn't be prematurely optimizing their code: they don't have the resources for it, nor will their results be representative across all hardware. I routinely see (and publish) papers with novel cryptographic protocols where the reviewers will say things like "compare the running time of your algorithm to a state-of-the-art production system that Apple spent $50m developing and optimizing for their silicon, using hand-tuned assembly." We're not going to win in that benchmark, and we don't have the resources to even compete. Moreover, the resulting code wouldn't make for good reference code. But that doesn't mean we shouldn't publish our work, so that e.g., Apple (or other resources) can begin the process of optimizing it for production.
While I agree with that in principle, the question of whether or not a given algorithm is fast enough to be practically usable is very useful information for the reader of your paper. Do I need to read your paper, understand it, implement it myself, and only then figure out that its 3 orders of magnitude slower than existing algorithms? I know its a lot of work, but as the reader I don't want to do that work either.
I've spent the last few years working on a novel collaborative editing algorithm for text editing. If the algorithm was orders of magnitude slower than existing CRDTs, it would never get adopted. And I want my work to be used. I want to use it, but again - only if its fast enough to be practically usable. I ended up writing a very highly optimised implementation of the algorithm - which took a massive amount of time - to answer that question. For the paper, I spent another couple of days writing a much slower reference implementation that people can read to understand how it works. (The optimised version is thousands of lines, and the reference code is a few hundred lines).
I understand how much work and expertise is needed to do things like this, but I still want someone to do this work. Even if its "We wrote a straightforward implementation of <competing algorithm> and <our algorithm> in plain C code. They perform at roughly similar levels of performance <see chart>. We suspect highly optimised versions of both approaches would also perform at roughly similar levels of performance."
For that matter, who's going to replicate a condensed matter experiment for which a grad student took three years to build the apparatus at a cost of $250,000?
Peer review is an absolute joke. In principle researchers could publish their data sets and reviewers could see if they could replicate the analysis but they don't do it because they know if they tried they'd get different answers, see
In the case of the LHC, which I assume is what you're talking about, this is why we have 4 pretty much independent detectors around the LHC looking at different but at least partially overlapping stuff. The Higgs boson was detected by the ATLAS detector, but wasn't considered to be "discovered" until it had also been detected at the CMS detector.
> Peer review is an absolute joke
That strongly depends on the field and the journal in question.
When I wrote my first paper somebody else had done a very similar experiment at another university and we wound method up peer reviewing each other’s papers. Turned out we were using the same bogus data analysis that was being used in 1000s of other papers. I was vaguely aware of this when I wrote the paper but did not feel psychologically safe at all in an environment where the American Physical Society claimed a PhD had a 3% chance of having a permanent career in the field.
Someone who was later to become a titan in the field was at our lab and was crying some nights because he had no idea where he was going to get his next postdoc wrote a paper about this problem more than a decade later after he had gotten tenure in a statistics journal that physicists probably won’t even read.
For a relatively recent example - in my field of quantum information someone found in the course of reviewing some newer work that there was a mistake in one of the big foundational papers that this newer work was based on. They shared it with some of their colleagues and together they did some work finding out exactly where the error was and what the impact is on work that is based on it.
The response from the community was they got a talk slot at a pretty prestigious conference (QIP 2023, you can watch the video here: https://www.youtube.com/watch?v=2Xyodvh6DSY) and the work they did has been published here https://quantum-journal.org/papers/q-2023-09-07-1103/ and in a followup paper here https://www.nature.com/articles/s41567-023-02289-9.
My point is essentially that it isn't a big shock that this error wasn't found by peer review when the foundational paper was published in 2008. Even when it is working as well as possible peer review can only detect most errors, not all errors. However when your field is functioning "normally" and everyone does their jobs properly this doesn't matter because the errors that get past peer review get detected by other people building on the work later. On the other hand if your field is not functioning properly and people are knowingly doing stuff that is wrong then peer review can't help you. If the status of the field is so broken that people who know stuff is wrong are still pushing for it to be published then you aren't in a place where peer review can save you.
My opinion is you are incorrectly putting the blame on the (highly imperfect) process of peer review here where it doesn't really belong. Peer review as we perform it today sucks for many reasons, but it isn't to blame for the problems you saw.
In medicine see
https://www.cochranelibrary.com/
where there is a systematic process to search the literature and select papers with usable results. I've never seen more than 50% of papers get accepted and the other day I saw one where they found 80 invalid papers and 2 valid papers. It seems to me that peer review isn't accomplishing very much if it is passing through papers that can't be built on. For that matter, why is this work even being funded?
This isn't an indictment of peer review as it is practised in every field.
Look at hep-th. If in 100 years people conclude there is no evidence for supersymmetry and extra dimensions something like 90% of that field will be “not even wrong”. There is so much of the drunk looking for his lost keys under a lamp. I mean, there is a huge industry in 2-d gravity where it is possible to calculate things but gravity doesn’t attract in 2-d so the line between that and Newton’s apple is unclear. I’ve talked to astro-ph refugees who think that current theories about accretion disks are “not even wrong”.
Then there are all the folks who tell me firewalls make no sense because there is no such thing as the event horizon (locally) but from the viewpoint of people who fall in it is a tiny difference getting killed at the apparent horizon meanwhile the nonsensical “black hole information paradox” (sorry, no unitarity = no quantum mechanics) still gets breathless and insufferable articles written about it just because somebody famous for their disability made a bet. If young people were able to get established in the field you wouldn’t see people so fixated on bad ideas.
That was my thesis. To make it worse, the laser I used became unavailable as I was finishing up. My work was adopted by 2 labs but required extensive adaptation.
If the results are interesting to anyone and there is more science to do, people will eventually replicate (or fail to do so, like with Majoranas). If not, then why would we care about replication?
It's extremely true in physics. I've published in (good) physics journals.
Computational papers: Authors leave out details because they don't want to give up the goose that lays the golden eggs. They broadly describe the technique, ("solve this PDE computationally"), and anyone who wants to replicate has to craft his/her own solution. If they fail to replicate, the authors just say "your simulation was poor and probably numerically unstable".
This was the norm.
Experimental papers: Same thing. Experimentalists are inventors. They don't buy off the shelf equipment - they build their own. They don't provide details of how they built it in their papers.
At conferences, PIs would openly discuss whether they "believed" a highly cited paper. When I was a new grad student, this troubled me. Why should belief play a role? Just replicate it! A year later: "Oh, of course these studies can't be replicated."
It's one reason I left academia. I was doing theoretical/computational physics, but the whole game reminded me of English literature: The value of your work was always seen with subjective eyes. Your paper could be rejected simply because the reviewer doesn't "believe" it.
If you are talking about theory or numerical simulation papers, just replicating the calculations in the paper are not enough to believe the conclusions. There are many assumptions that go into any calculations, in terms of how much of spherical cow you treat reality as. Changing those assumptions will change the conclusions. The assumptions are not always clearly stated, and even the authors might not realize all their assumptions. It can sometimes take many years or decades for people in the field to figure out how a paper was wrong.
So yes, experts do have to depend on their subject intuition to determine how much they believe some surprising result. Not to mention, if you go through historical papers in your field you will inevitably come across decades long fights where one group is coming to one conclusion in their paper and other is coming to another, before the situation is resolved one way or the other.
The guy built/fabricated a device, and reported interesting measurements. Want to know if it's true? Build it yourself! Except when you do and get differing results, there will be quibbling over whether you built it properly. There are always details left out of the original paper that can be appealed to. The original author is king of the field, and people will simply ignore your contrarian findings.
For theoretical/numerical results, it is for sure valid to criticize assumptions. But in the longer time frame, the question shouldn't be "Are these assumptions valid?" but "Is there experimental evidence to support these calculations?" With a lot of the papers published (including mine), I can assure you that no one will ever be able to construct an experiment to (in)validate my results. And that was true for probably over half of the theoretical/numerical results in my field. We were all publishing things that most of us believed could never be connected meaningfully to the physical world. We're not talking string theory or high energy stuff - more like material properties.
The goal was to publish, and convince others. Not to understand reality. Hence, more like literature.
I've discovered in the intervening years since graduation that he was wrong, you should start assuming by the bottom of the first page.
To be sure, reproduction is a bit easier in mathematics because you normally don't need exotic equipment, just time. However, that doesn't mean that people are necessarily taking the time to double check everything with sufficient rigor.
https://news.ycombinator.com/item?id=17430577
A number of papers on formal verification reported that restructuring their specifications for proof revealed problems they missed in the specification. Other works found through empirical testing that the specs didn’t match reality. Many theories in physics changed that way.
So, we should assume they’re all false by default until the specs are reviewed, the proofs are checked, and there’s empirical validation. If theoretical, its component theories must be built on observed truths instead of imaginary constructs. Then, we might trust the paper (i.e. mathematical claims).
It's true that it's hard to ignore all of the citations a reviewer suggests, but that's part of the role of an editor, and the response to reviewers (while being a frustrating exercise) can make it clear what is and isn't an appropriate citation, which serves as communication to the editor. Yes, this process is frustratingly low bandwidth
You are severely misrepresenting how things work here. A reviewer doesn't accept or reject a paper. A reviewer gives their response to an editor and the editor makes a decision. The editor knows who the reviewer is, and has access to the reviews and the author's response to them.
If the editor sees the reviewer has suggested adding bunch of references to their own work, and has suggested rejecting the paper after the authors refuse this then the editor is going to know what is going on. They aren't completely stupid.
> purposefully leak your own paper to get past the publisher pay wall
Many papers in maths, physics and computer science appear on arXiv before being submitted for publication. In the fields I have worked in you can replace the "most" with "pretty much all".
Isn't it ironic that we can't replicate your assessment?...
I personally have no idea how much papers can be replicated or not, but replicating an experiment would not guarantee that the idea/conclusion in the paper is sound (ex: paper extracting some data, observing a correlation and concluding it is an implication).
I do agree that it should be made easier to replicate for the fields where this is possible, but there are many other problems.
If anything, I'd think replication has gotten easier with the advent of open source and the expectation that top papers will include git repos which make it easy to reproduce. The advent of docker, locked-version-dependencies etc have all contributed as well. 10-20 years ago, we didn't have this - but I also didn't see * widespread * problems even then: if someone published a result saying that technique <x> led to <y> performance gain, I thought it usually did?
(that said, computer science research doesn't have the same money on the line as medicine and the "perish" part of the publish-or-perish equation isn't the same when the fallback for failed academics is still six figure salaries at tech companies)
And yet we still have advances in technology. I wonder how this academic nihilism matches reality?
It would be good to see what the criteria is for deciding if a journal is "to be taken seriously". I imagine for example that Chinese or Arabic language journals wouls be published and citsd in journals of those languages. That doesn't necessarily mean that they arent to be raken seriously in the field, it's just that they aren't Western publications.
Some journals now require an "Author Contributions" section to at least partially address this issue.
Prestige is not a failsafe guarantor of quality, but it kind of is, in the sense that somebody with a reputation to protect is going to be more careful about coming across as "unserious", and doing stuff like this is a good way to get a bad reputation quickly.
In general, State Flagships and Ivy Leagues+Ivy Tier Privates are largely comparable research output wise.
The GP is right about prestige minimizing bad practices. Look at how diluted Northeastern's brand has become by trying to artificially inflating their prestige [1]
[1] - https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
[1]*
I'm sure they understand it. But their goal in doing this isn't really research productivity (though it would be a happy byproduct) it's to obtain prestige, or its illusion. And/or be able to tell the people of your country how great you're doing. "Our country is a rising powerhouse in field X, in the top 3 in the world and still rising!"
It's internal propaganda. And externally targeted towards populations of other countries unfamiliar with the academic publishing system, which is most people. So for example the vast majority of people in the US that hear these claims by China, which are then echoed in click-bait headlines or simply taken at face value after a journalist checks something like the various HCR lists.
It can even be propaganda, not from China, but from other countries. One of the studies was funded by the US State Department to an Australian group, thus obscuring a primary influence on the results of the study.
Then even a single newspaper article can spark a dozen others citing it, propagating the message. All of these directly or indirectly rely on citations to support their headlines [2]
https://www.science.org/content/article/china-rises-first-pl...
https://www.wsj.com/articles/american-universities-continue-...
https://money.usnews.com/investing/news/articles/2023-03-02/...
https://www.reuters.com/article/idUSTRE72R6FQ/
[1] The quality of this comment will be measured by metrics that I made up myself.
[2] https://www.reuters.com/technology/china-leads-us-global-com...
*This comment was funded by a grant from its author.
Or the goal is research productivity and citation gaming is just the unhappy byproduct. Yes some motivated actors to playup PRC capabilities (ASPI/US think tanks) exist, but unless you think every western index on science and innovation, controlled for quality of citations (like Nature) are incentivized/coordinated to carry water for PRC's massive increase in research productivity in the last few years, then the parsimonious answer is PRC research productivity has gotten really good and world leading in some domains.
Which should not be surprising because what OP fails to understand incentivizing citation gaming is a smart way to incentivize output, FAST, and at PRC scale output quantity has quality of it's own. If system spams research, and start off with only 2/10 good research, but leader is 3/5 good research, then PRC has nearly caught up. Refine to 3/10 and there's parity. Refine system to 4/10 and PRC leads. Emphasis being fast because PRC started focusing on seriously improving tertiary and R&D/S&T ~10 years ago and the goal was to develop capabilites fast by customary overproduction. You don't get that by slow careful growth, you do that by incentivizing easy KPIs everyone in sector can coordinate around and the product of said KPI is ENOUGH good research after byproduct of citation gaiming. Having a bunch of chaff leftover to get wheat is the point.
There are research problems to solve in industry, with more prestige, more pay, and so on.
I'm certain his contemporaries said the same about Pythagoras.
You're not wrong about how much more complex things have gotten, certainly.
But you don't know what you don't know. Another Ramanujan or von Neumann could be right around the corner.
Just look at all the wasted potential in our education system - it's impossible to quantify just how much effort is profoundly wasted.
This article points directly to that wastage. We've tolerated an academic system that's hated by just about everyone except publishers, in an era where publishing is as close to free as it could be.
Now, start extending that limit by integrating computers? The options are literally unimaginable.
But that all kinda misses the point, which is, there's a lot of low lying fruit that we're leaving unpicked.
> I'm certain his contemporaries said the same about Pythagoras.
Pythagoras is best known as the center of a mystery cult. What did he bring a century forward?
https://en.m.wikipedia.org/wiki/Pythagoras#In_mathematics:
“Although Pythagoras is most famous today for his alleged mathematical discoveries, classical historians dispute whether he himself ever actually made any significant contributions to the field.
[…]
The Pythagorean theorem was known and used by the Babylonians and Indians centuries before Pythagoras, but he may have been the first to introduce it to the Greeks. Some historians of mathematics have even suggested that he—or his students—may have constructed the first proof. Burkert rejects this suggestion as implausible, noting that Pythagoras was never credited with having proved any theorem in antiquity.”
Labs and researchers are given funds based on "impact".
As we all know, when a metric becomes the goal, the metric gets gamified.
This is very hard to fix.
Gonna give you an example. As soon as you move in any direction in science you're entering a niche. Pretty much everything is its own niche where there's a limited number of people really able to review your paper.
No, there's not thousands of experts dedicating their life to helicoidal peptides interactions with metal layers. It's an extremely small number of people. They all know each other and are gonna review each other's papers regularly.
Solar-powered water splitting to produce hydrogen and oxygen? Again, extremely small club.
Perovskite? Graetzel solar cells? Bigger club, but still, at the end of the day the relevant luminaries are a handful again.
You think it's much different for quaternions or advanced complex numbers analysis?
And those small clubs set the rules and standards, and there is really not much way to have oversight on those small clubs.
Plenty of terrible science is published every day, I can confidently say 90 to 95% of experiments in Physics or Chemistry are impossible to reproduce (numbers people get by taking outlier results or even more often by tweaking the data).
When it comes to softer sciences like psychology it's even worse. It's crap.
It's sad, but I think the world desperately needs a *free* alternative to the biggest publishers out there like Nature or ACM or all these things. That free alternative has to be funded by universities and governments globally. This entity should only allow papers that present clear experiments that are reproduced elsewhere or under supervision.
This would greatly enhance the quality and reliability of papers.
It feels like it should be. Maybe the following is just the naivety of a novice looking at experts they don't understand.
Learning numbers, integers, rationals, and real numbers, introduced so much power with each step, to the point that every child is expected to know the basics about real numbers. Maybe not what they are theoretically, but the basic ability to do something like .5 * pi or root(2)^4. They likely never dig any deeper, most don't really get the idea of never ending never repeating decimals, but they are able to work with them on a simple level. For most fields, an introduction only requires this level of math.
When one does move to the complex numbers, it opens up far more possibilities. Fourier transform is everywhere, to the extent many use it without having the math to understand how it does what it does. More complex problems, and I mean outside of the field of math, only have general solutions if you allow for complex numbers to be used. These are difficult, mapping from C to C requires 4D to represent and thus are much harder to visualize, but people still struggle through it.
When I realized that complex numbers weren't the end, but the second step in a tower of infinite height, I wondered what otherwise unsolvable problems needed higher levels from that, just like how many problems needed complex numbers. The difficulty working with them grows, though somewhat given C to C is already 4D, we have already reached the limit of 3D viewing power.
Yet they are rarely used, and higher levels used even less. Maybe there is something fundamental that makes them less useful, far weaker than complex numbers. But from a viewpoint of a novice, I find that surprising.
The amount of additional power you get from real numbers is zero, and in fact it isn't possible to work with them because specifying an irrational number takes an infinite amount of time and space.
They give you additional power in the sense that they let you give names to the exact answers to certain equations, but not in the sense that you can manipulate those equations any better than otherwise.
> When I realized that complex numbers weren't the end
But... they are the end. That's the point. We call that the Fundamental Theorem of Algebra: every solution to every equation takes the form of a complex number.
Quaternions do not generalize from complex numbers in the sense of the progression of the numeric tower from integers to rationals to reals. That is known to stop at the complex numbers. They generalize from complex numbers in the sense of functions that scale and rotate two-dimensional images. Quaternions are a way of expressing functions that scale and rotate three-dimensional images. They are familiar to most people who have taken calculus under the name "vector cross product".
All I said is that as soon as you release a paper you fall in a niche of some topic (unless the paper is super generic) where the number of people with the expertise allowing them to review it is super small.
Quaternions or complex analysis where just examples where, even if not strictly experimental and more theoretical, if you're pushing the boundaries of knowledge the number of people able to review what you wrote and concluded isn't that big.
The reality is murkier. Huge names like Lorentz and Poincare had worked on relativity before Einstein. Gregory and Barrow had proved the fundamental theorem of calculus before Newton, not to mention Leibniz's work.
If you mythologize the past it's easier to look around and wonder why we don't have immortals like Zeus or incredible warriors like Achilles anymore. But the truth is science always proceeds by steps that look small at the time and it's often only in retrospect that things seem amazing and unprecedented.
Semi-relatedly, I used to attend a seminar with a well-known Russian mathematician who would often chime in with Russian/Soviet priority over historical results mentioned by visiting speakers. The cold war created two mathematical cultures that had limited contact. So famous European and American mathematical results from, say the 40s to the 90s often had Soviet versions worked out independently in journals nobody here had ever heard of and written in a language they can't read.
So this is all just a way of saying that empirically, the big kahuna theory of mathematical development seems more fiction than reality. And it should be treated with skepticism when you hear things framed in those terms.
[0] Not that their achievements aren't great. They are all incredible. I just mean they seem more incredible than they are if we forget all the other incredible achievements. Shoulders of giants etc etc.
But math keeps advancing at a good clip, there are tons of brilliant young people. Physics IIUC is largely gated on waiting for new experimental results. Einstein, Poincare, Newton and many others arrived at a time when the experimental physics data was perplexing and theoretical advances were needed to clear up the picture.
The reverse is true now if I understand. People are just waiting for more funding to build larger accelerators or whatever big expensive astronomy things they need to test the many predictions we have from the various mathematical attempts to unify quantum mechanics and relativity.
Biology seems to be moving pretty briskly. I can't speak to chemistry or any of the other sciences.
EDIT: I should add that when I left academia about 8 years ago, the mood was that academia was increasingly hostile to smart people who just wanted to do research. The move toward running universities like businesses has pushed out a lot of people who would be candidates for the next Einstein or Poincare. Poincare happened to be independently wealthy, so wealth may again be more of a factor than it was during the last century.
Exactly these people are more likely to be spending their time speed running Mario.
Which doesn't seem at odds with the grandparent post, given that we shouldn't expect throwing more people at the problem to accelerate discoveries.
I think it's really an empirical question how it works. My own take is that it takes both. You need the ideas floating around and you need an obsessive synthesizer.
A music example might be Bob Dylan. Folk music could have been a passing local fad, but Dylan studied all of it, distilled it down, and produced a version that seems different in kind than what came before. Without Dylan, we could look back on that era as being like the swing revival of the 90s. Just a period where a kind of music was popular again but with no real lasting change.
Similarly, there could be areas of math that are ripe for a Newton figure to come in and clarify everything. But that field could wait 100 years in the backwaters of math journals before anyone took notice. So you can't directly jump from the ideas floating around to the synthesis, you do need some sort of figure who will read and understand everything and put all the pieces together into a coherent piece of writing that people can read.
... this and that paper also worked on this research area and contain this and that stuff you can citate ...
The alternative is several independent communities of researchers that work separately on the same problem, but do not acknowledge the existence of each other. Now, that would put you off!
I think this is mostly just a kind of fun way of complaining about reviewers. Having been a reviewer for many computer science conference, I've often had the names of other reviewers visible to me. Most requests for citations are not for the reviewer's own work, and truly irrelevant citation requests seem to be pretty rare.
I do perceive journal/lab-centric fields to be worse about this, though.
Of course, I've also seen EDITORS email me saying they would like me to cite some papers from their journal after I submitted my paper. That was definitely a turn off.
For a level of an individual researcher this is easy, you just learn what is true and what is false. For a level of the whole society the battle is already lost, your (highly valuable and competent) solution to make a gigachad search just will be either lost or gamed or sued/prohibited.
Someone could write a paper on this.
> There were people that published in journals that no serious mathematician reads, whose work was cited by articles that no serious mathematicians would read, coming from institutions that nobody knows in mathematics
is really bad. It suggests one can be a 'serious' mathematician only if you went to XYZ universities and published in ABC journals. This elitism is the main reason why people try to game the system in the first place, because they know that they can't advance in their careers unless they build that perception of prestige. The single minded focus on prestige is then what smothers the real science.
Money in general is a bad incentive if honesty and rigor are wanted.
That was my impression whenever paid review was mentioned. Nonetheless, it is also possible that more who dislike the idea were not voicing themselves.
A prominent paper with a false hypothesis can get many citations because people reference it when they refute it.
A related but much larger problem is that citations are a measure of the number of researchers working on a particular problem and nothing else. There was one topic in particular that had many researchers making small contributions, but all of those papers cited all the others, simply because they were written on that topic. Maybe the methods weren't the best or the conclusions weren't robust, but the only reason to cite a paper is because it is prior literature. It's not like you can refuse to cite papers because you believe they are of low quality.
Administrators needed some objective metric that allow them to make decisions themselves without relying on the personal opinions of researchers, whom they don't trust. This is magnified by rankings that like these metrics for the same reason, publishers trying to promote journals, and database access sellers.
Unfortunately, this has now circled back to the researchers, because the administrators are making hiring / promoting decisions based on these metrics, and people started to game the system. However, I think in many circles, there is a healthy culture of rejecting this nonsense.
I never got this obsession with detailed rankings. Sure "above average" and "under average" makes sense, but position x or x + 1 out of 10k does not make much sense anymore (and by similarity if you are 1 or in the first 1k out of 10k could be similar)
- bust citation rings
- catch plagiarism
- flag statistical malpractice/mistakes
- attempt replicate published studies/experiments
Could give jobs to loads of young scientists and be a huge net positive for society.
Most R&D is driven by Government Procurement or Commercialization anyhow, which can validate if shit works or not.
A lot of the subfields with a replication crisis are those that aren't exactly "tangible", and as such don't have a quantitative way or vested interest in validating that stuff works as expected.
If I'm researching a compound as a MatSE on a grant from the DoD or Applied Materials, it's on the assumption that tangible results will happen, and any attempt at fraud will be detected once commercialization or productionizing kicks in.
At the end of the day, a field like Pure Math or Anthropology just doesn't have a commercial incentive that forces reproducibility because there isn't a tangible impact.
Research that gets federal grantmaking attached to it essentially gets validated at the production stage.
Fields dealing with a severe replication crisis are also those which aren't validating or productionizing research.