Hacker News Reader

> lately it feels like _Science_ and _Nature_ are in fact especially likely to collect results that are very exciting and also wrong, because they feel like they're the best journals so their results should be surprising.

I believe this is accurate on all counts.

Slate Star Codex put the issue fairly pithily: there's only one way to get surprising-if-true results that isn't surprising.

Can you explain the joke?

The only way to get a surprising-if-true result that is not surprising is if the result is not true. Hence the joke is that most of these surprising findings are probably B.S.

Here's how I explain it: Science and Nature play a role different from other journals. Their role is to stimulate new thoughts amongst the most skilled in the field. Any really good scientist in a field can read a paper published by another really good scientist and hone away the bullshit, but be inspired by the ideas. People who are not skilled in the art of reading a competitive paper are going to be disappointed by the sloppy lab work, sloppy writing, sloppy thinking, because it appears that the overall benefit of disseminating new ideas widely for experts to cogitate on may exceed the value from having "perfectly correct" papers.

Case in point: the original DNA structure paper (W&C, 1953) actually had a wrong detailed structure for the DNA (but the right conclusion about how the structure provided a hypothesis for a templating mechanism) because the alcohol concentration used to form the crystal was too high. The value of getting the paper out the community ASAP greatly exceeded the small details that were wrong (which eventually got resolved), because it stimulated the thought of the community around the templating mechanism.

Sure, but this particular study came out of the lab of the Nobel-prize-winning scientist who literally discovered prions. I think it's at least worthy of discussion.

Prusiner has been on the path to this study for quite a while, at least since a talk of his I attended in 2011.

He was on that path in 1999 when I skipped a day of high school to attend a lecture he gave. (I also later became a prion researcher, specifically in a lab that worked with Alzheimer's).

You know what they say, when you’ve got a hammer everything looks like a nail. I’m an armchair physician at best, and I’d say that this doesn’t disprove the viral infection thesis at all. That amyloid and tau act as prions isn’t really relevant if they’re in fact produced initially in response to an infectious agent in an attempt to stop its spread. I’d suggest this is particularly likely to be true as attempts to clear out or suppress the amyloid and tau plaques in fact clear them out with no impact on cognition. This is mentioned in the article itself.

plaques are proxies of disease, not disease-causing or necessarily harmful in themselves.

Pardon me if I misread, it seemed that the thesis here was Amyloid and Tau 'prion' self-propagation was being identified as a cause of Alzheimers.

The worst part is that while articles about a disease like this are a mere annoyance to us, for people with loved ones with the disease or suffering from the disease itself it becomes a cruel joke.

These are people who scour the internet looking for any information they can find and carefully read articles like this looking for a bit of hope that there may be a solution. But it’s bullshit. Don’t share these articles, for their sake.

Totally agree.

I'm the OP.

I do not have, but I am personally affected, by this disease.

I appreciate your point, but this is one of the more interesting alternative hypotheses I've seen in the past ~8 years as an unwillingly-interested layperson.

senior author of the article was the person who established that prions are a legitimate disease caused by protein folding he had to fight an uphill battle for over 20 years to get his ideas established so I'm going to actually give him a ton of credit and the ability to publish what he wants to because he's already demonstrated he's responsible. He also won the Nobel prize for that work.

You could modify your belief in his authority based on the fact that this result was presumably rejected by top-tier journals, and probably for non-trivial reasons. I don't know anything about the field so I can't say whether that's appropriate here. But if you're willing to grant credence based on a prior Nobel, you should be willing to consider taking some away based on the publishing journal's reputation. Why wouldn't a top-tier journal want to publish a Nobel winner's Alzheimer's breakthrough?

Where did you get the idea his work was rejected? Also, his work was rejected by top journals for decades until people finally realized he was correct.

If it could have been published in Science, then it wouldn't have been submitted to Science Translational Medicine, a completely different journal with much less visibility (and a much lower impact factor). For some of these Nobel guys you could argue they want to avoid supporting the flashy magazines and don't care about getting into the high impact journals because they are already established, but in this case he actually submitted to a Science-family journal, so he's very unlikely to be making some principled stand.

You claim is entirely speculative. There may be good reasons why it didn't show in Science (page space is extremely limited)

Yes, it's speculative. But so is the idea that this work is correct because he won a Nobel in the past. That's the point: you're making guesses based on limited evidence, and we're pointing out that you didn't use all available limited evidence to influence your guess, just some evidence that confirmed what you already felt. This is a kind of bias.

> There may be good reasons why it didn't show in Science (page space is extremely limited)

Um, exactly? Science is the most visible journal after Nature, so it's the most competitive. "Insufficient page space" is exactly the same as "not impressive enough to be accepted".

one of the authors discovered prions, so I am not so quick to discount their work.

that's an argument from authority rather than one of science though.

we believe things only once we've exhausted our attempts to disprove it

Parent is right this is an appeal to authority. It doesn’t make it more or less right whether a Nobel prize winner comes up with the idea or my mom does, we need to evaluate it on its merits alone. The corollary as I mentioned in a sibling post is that when you have a hammer everything starts to look like a nail.

We rely on a million appeals to authority every day of our lives. Try some new food, trusting that it won’t kill you because somebody says it won’t, and nobody bats an eye. Say you give more weight to a study performed by an expert, when that has no immediate bearing on your life, and everybody loses their mind.

Well said. Myself specifically, I'd like to think science is supposed to be better than that -- neutral, dispassionate. It probably isn't because it's done be humans, but I want it to be.

The people studying this stuff definitely need to be better than that. They can maybe take hints from authorities but they need to check facts for themselves.

But for the rest of us, we have no choice but to trust authorities in some way when deciding how to interpret some new result.

I have to strenously disagree on this point. In this case, the author won the Nobel Prize after fighting the establishment for 20 years over a basic idea- that a disease could be caused by misfolded proteins and that the proteins can themselves cause more proteins to misfold in a cascade- ultimately showing that his theories were correct.

When a person fights the establishment and shows their data and ultimately gets accepted and defines the new paradigm, they've earned a seat at the table to speculate on how things work. They proved their worthiness as an authority.

That doesn't mean we shouldn't evaluate on the merits, but the priors are very heavily in his favor.

> that's an argument from authority rather than one of science though.

No. An appeal to authority would be "this must be true because someone authoritative said it". Saying "I am willing to treat this claim more credibly than other claims because the source is credible" is not a logical fallacy.

It's a Hacker News favorite to point out logical fallacies in comments as if that automatically refutes them.

>"I am willing to treat this claim more credibly"

That's a personal heuristic - a very reasonable one, but it's still an argument from authority.

A heuristic is a bias, not a fallacy. "All else equal, I'll try this path first".

A fallacy assumes a cogent argument is being made. This is just a preference.

Assigning any credibility to a statement based on the identity of its source is indeed the logical fallacy known as "appeal to authority".

Whether you say "it must be true", or "it is likely to be true" does not matter. Credibility must only come from the statement itself and the evidence to support it.

However, while it is a logical fallacy, it is reasonable to temporarily hold beliefs for or against the truthfulness of a statement while no or insufficient evidence is present. As long as belief is not mistaken for proof, nothing is wrong about having such gut-feeling.

There’s nothing fallacious about assigning greater probability to a claim made by a relevant authority. Such claims are actually more likely to be true.

> Assigning any credibility to a statement based on the identity of its source is indeed the logical fallacy known as "appeal to authority"

I think you are confusing deductive and inductive reasoning.

This fallacy is usually taught as appeal to irrelevant authority. It concerns relying on those who purport to be experts but lack any actual expertise.

"The ad verecundiam fallacy concerns appeals to authority or expertise. Fundamentally, the fallacy involves accepting as evidence for a proposition the pronouncement of someone who is taken to be an authority but is not really an authority."

Stanford Encyclopedia of Philosophy (9): https://plato.stanford.edu/entries/fallacies/

Here we have a researcher who is an expert in the thing he's talking about. There are still absolutely reasons to be guarded-- the matter is not fully decided, it's just one study. Maybe this is an example of an ongoing controversy, and we're selectively ignoring other experts?

But it is different to extend a cautious benefit of the doubt here than it would be to take, say, Paul Rudd's word on the topic as final.

The important context here is that the author spent 20 years fighting The Establishment over the facts of prions and ultimately was proved completely correct. that makes him the ultimate authority in this case because he had to actually fight to overturn a long-standing scientific belief with heroic data collection and a hostile environment

Prusiner actually proved his case well over a decade before the establishment was ready to accept his findings. They literally denied his results were true and accurate. Accepting that proteins can catalyze misfolding in a cascade was not an easy thing for the establishment to accept (however, by the time I attended UCSF, prusiner's work was already being suggested to be valid, and people were already working on mechanism).

If the author discovered prions, wouldn't this make him strongly biased toward thinking that prions are the key to solving important problems?

Obligatory: http://languagelog.ldc.upenn.edu/myl/ScienceNewsCycle.gif

> Y'all

Instant throwback to 1997's Redneck Rampage game about US hillbillys..

Can you please not post unsubstantive comments to Hacker News?