I used ChatGPT to give me a list of test areas for a certain type of scenario. And it actually pointed out one that I had inadvertently missed. Now if it said something off the rails I would've known that too.
I've probably had 100 conversations with ChatGPT and I can only recall one blatent lie that it told, which I think was P != NP. Which maybe wasn't a lie, but it didn't have sufficient evidence to make that claim.
I've actually so far gotten more accurate information from ChatGPT than I did in Wikipedia's earlier days -- where I discovered that Matt Damon dropped out of Harvard because he was too stupid to make it there.
The fact that you don't, or have managed to pass that as anthropomorphizing is curious. I'll have to think about that, but on its face seems strange alongside your comment about if someone can't find a legitimate use case for AI that they shouldn't be a person who summarizes it.
They can certainly be wrong, though.
So the flaws of chat AI currently are exactly what make art great. And art came before science, so perhaps scientific thinking is much harder on an evolutionarily timescale than we think.
But sadly, that may be evidence that there's a market here. If Amazon can cut support costs by 80% and only moderately worsen their already bad quality, some execs would consider it a success, not a failure.