I think a lot of the misconceptions about these LLMs (which I actually think are pretty cool and useful) stem from not really understanding what they are doing and how they arrive at their answers. This is pretty well illustrated by the fact that ChatpGTP only does well on tests where the content is well-represented in its training dataset, and terrible on tests on otherwise. But you have to keep in mind that the training dataset is enormous, and likely includes every random forum or reddit post ever made, so you shouldn't be surprised that it can formulate an answer like this.