Hacker News Reader

Comment by sva_ | Hacker News Reader

> Hopefully this can help address the replication crisis[0] in (social) science.

I think it isn't just p-hacking.

I've participated in a bunch of psychology studies (questionaires) for university and I've frequently had situations where my answer to some question didn't fit into the possible answer choices at all. So I'd sometimes just choose whatever seems the least wrong answer out of frustration.

It often felt like the study author's own beliefs and biases strongly influence how studies are designed and that might be the bigger issue. It made me feel pretty disillusioned with that field, I frankly find it weird they call it a science. Although that is of course just based on the few studies I've seen.

View on HN

> the study author's own beliefs and biases strongly influence how studies are designed

While studies should try to be as "objective" as possible, it isn't clear how this can be avoided. How can the design of a study not depend on the author's beliefs? After all, the study is usually designed to test some hypothesis (that the author has based on their prior knowledge) or measure some effect (that the author thinks exists).

There is a difference between a belief and an idea. I might have an idea about what causes some bug in my code, but it isn't a belief. I'm not trying to defend it, but to research it. Though I have met people who do hold beliefs about why code is broken. They refuse to consider the larger body of evidence and will cherry pick what we know about an incident to back their own view conclusions.

Can we recognize the beliefs we have that bias our work and then take action to eliminate those biases? I think that is possible when we aren't studying humans, but beliefs we have about humans are on a much deeper level and psychology largely doesn't have the rigor to account for them.

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants.

If you can't do science, don't call it science.

Which is a great idea if we ignore all other issues in academia, e.g. pressure to publish etc. Taking such a hard-line stance I fear will just yield much less science being done.

> much less science being done

This isn't obviously a bad thing, in the context of a belief that most results are misleading or wrong.

Let's do a less science then, but rigorous and throrough. Or find more funding.

But surely let's have a "hard-line stance" on not drowning in BS?

And where will the money come from for this second study? What about a third? Fourth?

We live in a money-dependent world. We cannot go without it.

Psychology is IMO in the state alchemy was before chemistry. And there's no guarantee it will evolve beyond that. Not unless we can fully simulate the mind.

Reply on news.ycombinator.com