Does history have a replication crisis? | Hacker News Reader

Does history have a replication crisis?

ageofinvention.xyz

43 pointssalonium_21 comments

In order to become a history teacher, I had to get a master’s degree in education. I was required to take a minimum of five graduate-level history classes in the history department as part of it. I was transitioning from being a lawyer. Professors would talk about checking citations (although not systematically) but it was more for the purpose of finding useful material you want to use, not to check how they are being used by authors. Ok, whatever. I was used to reading cited sources for the purpose of evaluating the argument they were purportedly supporting. But every single time I raised in a seminar that some citation did not bear the weight placed on it, or even simply failed to logically support what it purported to support, it was explained to me that I was missing the point.

After the second seminar of this happening, the chairperson of the history department called me into his office. He said he understood that I used to be a lawyer and thus was likely to be more drawn to or interested in arguing about things. He understood I was going to become a teacher, not a professional historian. But my approach to sources and citations was being complained about by history graduate students and the professors agreed. I was told that when reading secondary sources in seminars I should spend more time thinking about what is valuable about them and how they might be used in further research than scrutinizing the evidence and logic underpinning them. Anyway, it was a little insulting for me to think that I could legitimately critique such things in a useful way when I had not spent the years of work in the archives that the authors of those works did. He had called this meeting with me in order to be frank with me about the problems that the history department was having with me and to give me a “second chance” before talking to the education department where my degree would come from.

Suffice to say, I got the message: I obediently complied and shut my mouth in the remaining seminars. I got my degree and was certified and am now a history teacher at an urban public high school. I’m happy with my life. And I’ll never think about professional history/historians with the same fundamental deference to expertise as I did before I got a glimpse of it up close.

As a degree mathematician who has done a little contract work for both historians and lawyers and may have occasionally regarded law as the study of first order logic and theatrics (fun sledge, amusement rather than disrespect intended) I have to ask whether the history crowd had a valid point wrt to strength of reference in supporting an argument.

If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives.

It's fair to say that such claims formed the zeitgeist of our recent times for a substantial portion of the population affected and that any citation failed to logically support what it purported to support.

This is the "History is what happened" argument not the "History makes sense" position.

I freely acknowledge I've slid past the arguments made in this thread source, that current history isn't being rigourous with open access to sources and annoted commentary of primary material - but I note that the crafting of alternative histories can be coincident with primary events.

The election is something that distance makes clearer, although 50 years might not be enough time. Think of the history of the roman senate. Truth is barely talked about, the narrative revolves around factions, power, and outcomes.

Call it post-truth, call it machiavellian, but being on top is a social game, not a practical one.

> If five decades onwards one were to write a History of Our Times (the Central North American Edition) it would be correct to, say, point out that claims were made in the media and courts regarding a stolen election and to then cite court filings, articles in prominent newspapers, and perhaps videos from both network and youtube archives. It's fair to say that such claims formed the zeitgeist of our recent times for a substantial portion of the population affected and that any citation failed to logically support what it purported to support.

If you argued claims were made in the media regarding a stolen election and cited a variety of newspapers and blogs where people talk about how DT thought he won the election because of 10,000 dead people voting in GA or whatever, so he should be President not Biden, how would those citations fail to logically support your argument that claims were made in the media regarding a stolen election?

We're doomed to be in agreement, I'm afraid I have a hard time avoiding passive indirect citing of "factoids" ..

it's absolutely correct to state that "On <some date> the New York Post reported that aliens landed in Washington [1]"

it's deceptive (whether with delibrate intent to mislead others or just self deceptive) to state that "Aliens landed in Washington on <some date> [1]"

which is one of the root causes of the proliferation of "alternative facts".

It's not just that something is cited, it also matters how it is cited.

[1] link to New York Post article of <some date> reporting the landing of aliens.

> It's not just that something is cited, it also matters how it is cited.

You called it: I couldn’t agree more!

> Suffice to say, I got the message

And this is at least part of why I don't do well in academia (and, tbf, many other settings); I refuse to get the message.

I had a different objective: get my MAT (Master of Arts in Teaching), get certified in my state and get a job at a secondary school. They were threatening to throw a monkey-wrench at me. It wasn’t worth the trouble.

Had I been a history graduate student, on the other hand, I very well have fought that fight, because it represents a significant problem in the field of academic history.

About five years ago I asked myself whether I wanted to pursue a career change out of tech. I’m always excited by history and I’m sad that more people don’t feel the same way. I considered becoming a history professor or teacher and it was the credentials required that stopped me.

I’ve never had the patience for school even though I love to learn. I make a good living in my field despite not having a CS degree like many of my peers. I commend you for taking it upon yourself to do what I did not. I hope you have a very successful teaching career, by which I mean that I hope you have at least one student discover the joy in learning history.

This is kind of an open problem, especially in the Internet age. Any retraction that’s published will have far less reach than the original incendiary claim. And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis. Not sure that there’s a solution that’s actually implementable.

> And it’s simply not viable to follow that hundred-chain length sourcing tree for everything used in an analysis.

Maybe not for the passing reader, but I would imagine it's totally viable for those especially interested. So why not make it available anyways?

Retractions should defacto be capable to create chain reactions?

But then you have to link everything. Also, not all retractions will invalidate dependencies, so each cascading dependency (that will greatly increase in number) will need to check if the retraction affects their claim. Maybe they modify it. Now a dependency on that has to check if the modified claim still warrants invalidation. This is where the “implementable” problem comes in.

Life's too short to check or even verify the many factoids you encounter in daily life. With:

Check = check cited sources yourself. And

Verify = re-do a study yourself. Outside of hard science like physics (and relatively simple experiments), in most cases that's not even possible. Especially history: original artifacts disappear, witnesses die & the places where things happened are changed.

Written records are just that - what was written down. Accurate or not.

This is true for ordinary people as well as scientists. Though I'd expect scientists to make a bit more effort (+ using field-specific knowledge to sort things out).

But news reports tend to be notoriously unreliable imho. Especially popular media or (yuk) social media.

If some science study is reported on, it's often enlightening to read that study yourself. And find it says very different things than the news reports. Or does not even say what's reported elsewhere. Bias, sensationalism & clickbait are a thing.

Yeah, this has been a pet peeve of mine in online conversations where people call for sources. It's pointless, and maybe harmful. You can't just go looking to attach a citation to prove your point, without critical thought.

I've seen countless examples of "citations" that literally contradict their point. I can recall and exchange on reddit were someone smuggly claimed "they use science" when trying to make this crazy claim about how intermittent fasting is good for managing blood sugar and you should skip breakfast.

The citation said that people with blood sugar issues should always eat breakfast, and have small meals throughout the day. How did any of this help us get to truth?

The problem he mentions happens in science too. It's interesting to see it manifest in history, but it's well-documented that retracted scientific papers continue to be cited as veridical at high rates long after they've been retracted. I've personally noticed papers being cited for things other than what they actually say as well (someone posted an article to HN not too long ago about a particularly amusing example of this but I can't recall it very well).

It doesn't really undermine what the author is saying, but I do think it might be one of these cases where errors in citations reveal something more fundamental about citation process in general. For example, I can also think of shifts in who gets cited on a point, as if something is a new discovery, when it's really more fads in who is prominent in a field, and people "discovering" something by virtue of reading it for the first time by a specific author. The common denominator is convenience or ignorance (not paying attention per the author), which has implications for many things, not just factual errors per se.

The article focuses on the fact that nobody checks citations, but that isn't a problem specific to history. It's more like a general issue with academia. Science papers have the exact same problem. I found during COVID many papers being widely cited in the press as the consensus of experts which had bogus citations in them.

But there are other issues that are actually specific to history. Some years ago I had fun writing a series of essays about these, through the framing of conspiracy theories about history.

Part 1 looks at claims that key historical sources might have been forged during the Renaissance.

https://blog.plan99.net/meta-historical-conspiracies-part-1-...

Part 2 looks at problems with astronomical dating methodologies.

https://blog.plan99.net/meta-historical-conspiracies-part-2-...

Part 3 looks at methods like radiocarbon dating and tree ring counting.

https://blog.plan99.net/meta-historical-conspiracies-part-3-...

The goal here was to describe other people's claims about historical "replication problems" for entertainment purposes rather than try and discern the truth, so I wouldn't argue they're all real. But the trend towards presenting history as scientific doesn't gel well with the culture in the academic humanities. As dmvdoug writes, historians don't really care about even basic things like whether citations are accurate, let alone more debatable things like the accuracy of radiocarbon dating or whether sources have plausible chains of custody. And as the story of Henry Cort demonstrates, historians will happily make up narratives from whole cloth if it advances their ideological agenda, yet nobody within the academy seems to care.

> This lack of effective institutions or incentives was really brought home to me recently by the publication of a paper in the prestigious journal History & Technology by Jenny Bulstrode of UCL, in which she claimed that the inventor Henry Cort had stolen his famous 1783 iron-rolling process from Reeder’s iron mill in Jamaica, where it had been developed by 76 black metallurgists by passing iron through grooved sugar rollers. It was a widely-publicised paper, receiving 22,756 views — eleven times as many views as the journal’s next most most read paper, and frankly unheard of for most academic papers — along with a huge amount of press coverage.

It is pretty obvious that this became so widely spread because it agrees with a certain political viewpoint.

Those who do not study history are doomed to replicate it.

Those who do study history are doomed to not replicate it.

Problem is that no one wants to pay you to replicate a study and if you do replicate a study out of your own pocket it is a kin to setting that money on fire career-wise since no one will care.

The whole peer review science scene is all about making money and getting famous. At worse reviewers are trying to nitpick and block their peers long enough to beat them to publication so they can have the claim to fame and thus get more grant money for their next project.

HistoryNarrativeCheckerGPT maybe?

Reply on news.ycombinator.com