Meta-analysis finds little evidence for the Macbeth effect
digest.bps.org.uk
digest.bps.org.uk
Seems like the big secret is small initial sample sizes. For example, the original three Macbeth studies: 60, 27, 32. Later tried with N=153, 148, 266, 210, etc. and surprise surprise...
Another example, the "Power posing will make you act bolder" study, originally with only N=42. Later tried with N=200 by researchers at the Univirsity of Zurich, and the effect disappeared.
And the list goes on [2]
[1] https://en.wikipedia.org/wiki/Replication_crisis [2] https://digest.bps.org.uk/2016/09/16/ten-famous-psychology-f...
I wish there were something approachable like that for other fields.
Only rarely will you find direct replications in the case of medical and social research though, so as a heuristic you can just assume the evidence for everything is weak.
Also, this reminds me of that paper where they "discovered" that if you spray fart smell in a room that people are more disgusted by the thought of a guy rubbing "his bare genitals along the kitten’s body". https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2562923/