It gets concepts wrong, and can’t resolve interweaving narratives which a human can follow without issue. The advice it gives is generic and impersonal, and if you’ve ever had real therapy, you immediately sense of it’s short comings.
I’m sure a lot of that gets less noticeable as training and models get better, but it seems like we’re plateauing in the returns we get from more training.
Human knowledge is not the same thing as human experience. Create an AI that experiences the world autonomously, experiences trauma and come back to me.
I say this as someone who uses AI and doesn't completely dismiss it like many on HN these days. So far, I don't see it replacing the human being for connection and understanding. Replacing coders? That's a whole other question.