Academic urban legends (2014)
journals.sagepub.com
journals.sagepub.com
> The problem in this case is that I omit a piece of information: the fact that Larsson’s statement is based on an entirely different source, namely Hamblin (1981). In other words, I am referring to an article that I very well know is a secondary source, and thus hide from my readers the fact that Larsson actually just passed on information published by Hamblin 14 years earlier. A good reason for avoiding the use of secondary sources in academia is that messages that pass through several links have the unfortunate tendency to become modified or altered along the way, as in the whisper game.
I have always practiced and encouraged the opposite. To cite the original source, even in the unfortunate cases where it can't be read (there's an example from mine in another thread in this post).
It's a matter of giving credit where credit is due. It wouldn't make sense, e.g., for a seminal paper to not be cited because a later survey paper by different authors explained the findings better.
Yes, as I clarified in my answer to the older sibling of your comment >> If you can't, you cite the secondary source as citing the primary source. (Think "Secundus writes that "Primus writes that ..."", or "Secundus cites Primus on ...".) <<
The source is that this is what I was taught at university. Also, to me at least, this makes sense if you think it through. Just think about the trouble that not doing so has caused in the story reported in the gp comment.
Edit: A quick search indicates that the American Psychological Association, for example, agrees with identifying both the direct and indirect source [1].
In the reference list, provide an entry for the secondary source that you used. In the text, identify the primary source and write “as cited in” the secondary source that you used.
[1] https://apastyle.apa.org/style-grammar-guidelines/citations/...
Maybe this is just a biased view... I'm from a country where they obsessively count metrics to evaluate researchers (not that I agree with that, but it's what they do), so being cited or not can be a big deal.
I think the best solution might be to use the original citation but mark it somehow as inaccessible, that way it would be explicit that one is just citing for credit and did not really read the document. (I don't think I have seen this ever being done, though).
The alternative is to trust your memory that you’re quoting the correct information on a source you haven’t read. That’s going to quickly mutate into something unrecognizable.
The scientists I worked with seemed to know about the situation, but personally, I lost the very strong belief in natural science results. Turns out, there was no rigorous fact and results checking culture, as in mathematics.
It seems to me, it is mostly the lay people who treat published scientific results as some proven ground truth, the scientists themselves are more relaxed about it. Science may be in some ways about telling interesting stories in a convincing way.
Hence the meme "I think castrating children is wrong" "hurr durr- hAvE yOu GoT a SoUrCe FoR ThAt?"
This may reflect the feeling that for experimental sciences, the most reliable results checking may be to re-do the experiment, or perform a related experiment that is expected to produce similar results. In math (as I understand it), a proof is a proof -- the result is complete. In experimental science, an experimental result is often reproducible, but does not in fact guarantee that the hypothesis is correct.
While reproducibility and correct results are obviously important; most experimental scientists look for supporting experiments and a mechanistic framework before changing their beliefs.
Bioinformatics, especially genomics, is particularly prone to medical hype- the implication that these discoveries will rapidly lead to health improvements. Scientists are incentivized to claim the largest gains with the least amount of supporting data.
I spoke to several academics who had mentioned it, none had read the original - they all relied on someone else's referencing of it.
See https://shkspr.mobi/blog/2021/06/where-is-the-original-overv... for more details.
You find a lot of citations of Metropolis, Metropolis, Rosenbluth, and Teller’s paper on MCMC but I bet 1 in 10 have read it.
Ulf Grenander’s massive book on Pattern Theory is another, or Kolmogorov’s 1933 monograph on probability. Often cited but come on, did you really read it?
Another I’m aware of is references to the notion of VC dimension in learning theory. Many papers mention a hard paper by Shelah as an original reference to the concept…I saw the same reference given in a book by mathematician David Pollard with a remark along the lines of, “people say this is the same as VC dimension but I confess this paper is impenetrable to me so I really don’t know.” I appreciated the honesty.
I actually read that. Its definition of probability is pretty nice: it allows non-impossible events to have zero probability even in discrete case, e.g. a coin landing on its edge.
For example, when I started working in NLP, I did some things with the CYK algorithm (https://en.wikipedia.org/wiki/CYK_algorithm), which was defined in three papers by three separate authors. While I learned the algorithm from secondary sources, I of course cited the original papers (because one has to acknowledge the original author of the algorithms one uses). At that point, I wanted to read the original papers, even if just for curiosity, but they were impossible to find (I think one of them was available, but behind an outrageous paywall).
Now those papers are easier to find, but anyway I don't think I'd encourage a grad student to read them except for curiosity, the algorithm has been explained much better and in more accessible ways in more modern papers. Which doesn't mean that we shouldn't acknowledge the discoverers.
It effectively opened the door to Purdue's marketing department.
"In conclusion, we found that a five-sentence letter published in the Journal in 1980 was heavily and uncritically cited as evidence that addiction was rare with long-term opioid therapy.
"We believe that this citation pattern contributed to the North American opioid crisis by helping to shape a narrative that allayed prescribers’ concerns about the risk of addiction associated with long-term opioid therapy."
Allegedly.
* Message A: X is true
* Message B: Message A is false
Based only on that information (which often is all we have), there is no reason to believe message B rather than message A, but people seem to love a 'debunking' (it seems to make them feel smart).
I seem to remember that it matches a cognitive bias, such as believing the more recent message. Or, maybe we just fall for whoever acts more confident - a literal con game: The person communicating B is claiming to know more than the person communicating A, and for some reason we believe them.
If we assume general benevolence of humans, Message A had no evil when it was established, it was a genuine mistake, so Message B is true. Now if we assume some part of evil, Message B was crafter by evil actors who, even in the presence of more information than Message A, still entered additional false information in this world.
I started to read it but it was so bad that I could not go on. I told her to get someone from her field because I am too traumatized to read that and if she takes my input into account she may well need to rewrite the whole thing.
I was at her defense and some people asked questions about the statistical part, but the level was "how do you calculate the average".
This worries me a bit because as far as the topic is concerned, statistical mistakes about Middle Ages in France are sad but that's it - but mistake sin nutrition or pharma can be worrisome. In her case that was quite esoteric so no risk for anyone but generally speaking I now do not trust any numbers I cannot analyze myself.
I think this persists well into the 2000s.
> The belief that spinach is a good source of iron, although falsified 30 years ago by Hamblin in a British Medical Journal article, is still widespread among my colleagues, all of whom have, at minimum, a master’s degree in health sciences.
1981 + 30 = 2011.
I meant #1.
Certainly myth #2 still exists in the 21st century. DDG easily finds recent examples, like https://hekint.org/2022/07/18/spinach-the-great-myth/ .
> How should I refer to my source? If I want to include this sentence in an academic publication, what should I place after my sentence?
Why would you include it in an academic publication? Just cite a correct source for the correct data. And maybe, at most, cite the original incorrect source and state that it was incorrect.
I still think much of this can be eliminated with a "just the facts" approach, less editorializing, and briefer introductions. At least in scientific journal articles. History of science is another discipline entirely, and the article obviously stands.
Well, it does say in its conclusion: "The basic thesis of this article is that organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations." [0]
I actually cited Conway's law to convince a senior decision maker that his organisation was badly structured. Briefly, a civil service organisation acquired systems for various users in a large customer organisation. The systems were rarely technically compatible although were often doing very similar things. This complicated technical interoperability and increased through life costs (e.g. because they had separate support contracts that duplicated basic functions).
The incompatibilities often resulted because the systems had separately written user requirements (e.g. using different terms for the same things, describing common processes in different ways). The requirements were incompatible because they were written by independent acquisition teams. The acquisition teams were independent because they reported to and were 'owned' by different parts of the overall customer organisation. Recognising this fact allowed the senior guy to request that the various customer teams established consistent terminology, processes and support contracts. In other words, going for coherence by design rather than (expensively) retrofitting it.
[0] https://web.archive.org/web/20190919111512/http://melconway....
The authors came up in their widely cited paper with a proper solution to spread the random hash seed into the inner loop, vastly enhancing its security by avoiding trivial hash collision attacks. But a secure, slow hash function can never prevent from normal hash seed attacks, when the random seed is known somehow. esp. with dynamic languages it's trivial to get the seed externally.
Other trivial countermeasures must be used then, which also don't make hash tables 10x slower, keeping them practical.
Go on, ruin my childhood.
Spinach is also a contender (along with rhubarb) for being one of the most concentrated sources of oxelate, which those of us who have had kidney stones should try to avoid, or consume only in moderation, or consume only with a good source of dietary calcium (advice varies)
It's ubiquity as a salad leaf, and it's general tastiness are a source of frustration, but sadly not very much iron
I dont know if Harvard Libraries has warehoused most their physical books like many other college libraries. One has to order up old books then.
Primary case for the prosecution is a paper like Flaxman et al, "Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe" [1]. It claims that lockdowns saved ~3 million lives. Amongst other problems this paper has: it is built on circular logic [2] and an over-fitted model [3], has numerous other statistical problems [4], required an enormous country-specific fudge factor and hiding data points to cover up the way Sweden's results disproved their claims, and their model always claimed that the last NPI was the most effective regardless of what it was.
The cherry on top is that it disclaims its own results, literally admitting in the text that their counterfactual model is "illustrative only" and that "in reality, even in the absence of government interventions we would expect Rt to decrease and therefore would overestimate deaths in the no-intervention model" i.e. they already knew their conclusions were wrong when they wrote it. (This rather important caveat didn't appear in their press release about the paper, of course [5]).
Yet according to Google Scholar this paper has been cited nearly 3000 times in the ~three years since its publication. It gets cited several times per day. I've watched the citation count go up with morbid fascination. What are people citing it for?
If we do a reverse citation search and check some papers, we see immediately that it's being cited for a wide variety of almost random statements, none of which it actually supports:
1. "Health-care workers, seniors and those with underlying health conditions are at particularly high risk" [6] [8]. This claim appears with identical wording in two different papers, but Flaxman et al don't present any data on this or even reference risk stratification by job as far as I can tell. It's certainly not the focus of the paper.
2. "However, because most countries have implemented multiple infection control measures, it is difficult to determine the relative benefit of each" [7]. Flaxman et al claim it's easy to determine the benefit of each and that they did so.
3. "health agencies have long relied on predictive models to estimate future trends and to assess the potential effectiveness of various disease control methods" [9]. The paper doesn't show anything about the history of health agency decision making.
etc. Even when citations characterize its claims correctly, they are just taking its assertions at face value without realizing that the paper's methodology is circular and even the authors don't believe their own numbers. Of what use are citations in this environment? These aren't cherry picked examples, they're literally just randomly selected by when I happened to search Scholar. Even so, most of the citations are wrong.
[1] https://www.nature.com/articles/s41586-020-2405-7
[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7674856/
[3] https://www.nature.com/articles/s41586-020-3025-y
[4] https://nicholaslewis.org/did-lockdowns-really-save-3-millio...
[5] https://www.imperial.ac.uk/news/198074/lockdown-school-closu...
[6] https://www.nature.com/articles/s41577-020-00434-6
[7] https://jamanetwork.com/journals/jama/article-abstract/27683...
[8] https://www.sciencedirect.com/science/article/pii/S147330992...
[9] https://journals.sagepub.com/doi/full/10.1177/00375497231152...