Let's face it: Physics is easy! It's objects are often cheap to study, and usually doesn't react to the behavior of the observer. Additonally, the observed behavior is often regular and easy to model with maths. There are millions of research grants, and job opportunites because its findings are often economically useful. Also, people in general don't think they 'know' its findings in advance, so it's much easier to overcome your own biases, and opinions. In most cases, there is often no politics involved about the results.
In contrast, the social sciences are really hard. Lots of causes make research harder: Objects often react to the researcher, or the setup. They want to get payed for their time. Sometimes, they react to the experimental setup due a political stance. There are ethical constraints for experiments. The shere variability of human behaviour makes it hard to use mathemathical models to describe theories. Its findings are not as useful economically, because they cannot be impletemented on a large scale. Potential users of the research often think, they 'know' how people react, anyway (folk psychology). For some findings, many people are also motivated to question the results because it conflicts with their political opinion.
Overall, the barriers are much higher for the social sciences to make progress. I think these fields deserver the label 'hard', for finding the truth is really hard here.