There are specific things it knows that it doesn't know, like explicitly current information. But you can ask it for information that doesn't exist at all and it will happily generate nonsense without a hint that the question might be somehow wrong:

"please give me a history of the best teams in the American Soccer League from 1850-1859 in the form of a short epic poem"

> In 1850, a league was born, The best of soccer to adorn. From '50 to '59, teams did play, with strength and skill, each match a display.

(more verses follow...)

The American Soccer League is a real thing, but it didn't exist until the 1900s, and the snipped poem goes on to describe teams that never existed competing during a time period where the league didn't exist.

I asked it just now: "In POSIX, how do I create a new file descriptor which refers to the same file as another file descriptor but with its own seek offset?"

It answered:

> In POSIX, you can use the dup or dup2 system call to create a new file descriptor that refers to the same file as an existing file descriptor. The new file descriptor will have its own seek offset, which can be manipulated independently of the original file descriptor.

The actual answer is that there is no way to do what I asked. The dup or dup2 system calls give you a new file descriptor which refers to the same file description, so the two FDs will use the same seek offset; seeking one FD will seek the other FD too. But ChatGPT just confidently insists that dup creates a new FD which has a separate seek offset and can be manipulated independently.

This isn't the first time I've seen it invent plausible-sounding but wrong answers to tricky questions. The file descriptor thing is just something I decided to try just now because I encountered the problem a couple of days ago and it felt like the kind of thing ChatGPT would bullshit about. I was right.

I think you meant to type, and certainly ChatGPT could set us straight in the matter, "it has better spelling and grammer than I"

:)

...And FWIW, as an English major it pained me to reproduce the "er" typo in grammar.

The funny thing is, it often answers depending on what it "thinks" it should know based on conversational context, not what it actually knows.

Are you talking about when it objects to answering because it's a large language model blah blah blah?

I feel like that doesn't count because most of the time that triggers it's something it does know or can do just fine.