81 karma · joined February 25, 2021
The short version is that Gödel claimed in a paper that he had an algorithm to determine satisfiability for a certain logic, but the algorithm actually worked for a slightly different logic instead. People basically took Gödel's word for it for fifty years until a logician called Goldfarb found the mistake.
EDIT: As a disclaimer, I don't have any association with or endorsement of the website I've linked to. I found it just now because I needed a source for the story that I've known for many years (I think I first heard it in person, but it's definitely contained in the book `The Classical Decision Problem' by Börger, Grädel and Gurevich). I skimmed the post and it seemed to cover the relevant details, but that's the extent of my knowledge of the site and its contents.
Furthermore, I could believe that the above issues and, as you also mention, reproducibility, are worse in pure mathematics or theoretical CS than in other fields because so little is at stake. Which is not to say I think most maths papers are false, but just that they're not all _entirely_ true either. Their audience is so small and the readers/reviewers "know what they mean" anyway, most of the time. And others are just false and no-one noticed because they couldn't be bothered to check the details, because the result was inconsequential to their own work and the author was a nice enough chap...
Obviously, you have to bear in mind that high-profile mathematicians do get contacted by cranks and they have to weigh up what's worth spending their time on, but I have no doubt that if I did chance upon a genuine proof of the Riemann Hypothesis, I'd be able to find a decently respected mathematician to look at it and help me to publish it in a form acceptable to the academic community.
On the other hand, I agree with your comment about formal proof systems, and I'd love to use them in my (pure mathematical) research, but I've found the usability isn't there for me yet.