There are definitely problems in the field, in some subareas more than others. I am pleased to see how much discussion in the field there has been regarding issues with replication, sample sizes, etc., and I do see improvements there. Quite honestly, the discussion about these things has instigated the same questions in other fields. The progress is slow, and it frustrates me that I don't feel like I can fully trust a lot of the older research in the field, but...there is progress.
But I think you're right that it's a difficult field in general. Human minds are complicated -- possibly one of the most complex things in the universe to study. Human social interactions, relationships, group dynamics, identities, and start with those complex machines and now layer on top additional interdependent cause-effect relationships. The area of study is just, fundamentally, hard.
The field has developed techniques to try to address this -- that's why psych surveys tend to ask the same question in multiple different ways. By triangulating in to a fuzzy concept from multiple angles, it at least gets us to the point where (hopefully) there's some systematic variance we can talk about and study. We're never going to get to the point where we can measure "agreeableness" or "prejudice" or "empathy" with as much precision as physicists can measure changes in an atom. But studying the human condition in as systematic a way as we can still feels incredibly important to me, to do what we can to improve human flourishing.