Psychology’s Replication Crisis Is Running Out of Excuses

Serious question: how do these experiments come into existence? E.g. the Macbeth effect or the marshmallow test? It always feels like they show very specific effects and are extrapolated to explain broad parts of the human psyche. But since there isn’t really any objective basis for creating hypotheses, I have to imagine people just sit around, think of some idea that sounds cool, and then come up with an experiment that is cheap to execute, kind of zany and memorable, and easy to swing one way or the other.

I do research in engineering, not psychology, but if I had to guess: probably the same way a lot of engineering research comes about. You read the literature, see interesting results somewhere, and questions start to pop into your head based on the reading ("hey, that's cool! I wonder what would happen if...").

The tricky part becomes actually answering that question: designing an experiment to test a hypothesis is deceptively hard. In my area (computer engineering), often times the hardware or software to test your hypothesis does not exist, so you're left searching for analogues and work-arounds that emulate the desired behavior. I could easily see some of these experiments operating under a similar idea.

> it seems that one of the most reliable findings in psychology is that only half of psychological studies can be successfully repeated

Oof. That's a good line.

Is there a summary of which results did hold up?