I wonder how often this happens in Psychology studies.

Psychology, currently, isn't in a good position regarding repeatability for most of its studies.