Out of curiosity, I today asked it to program the ELIZA in MAD-SLIP, the original (and now completely dead and obscure) language that ELIZA was originally written in. My expectation was that it would have no idea what MAD-SLIP is, or maybe would barely have an idea, and in any case would have no chance of doing what I asked and that it would, very reasonably, balk.
Instead, to my shock it made up a completely bogus meaning for the acronym: "Matched Activated Decomposition by Syntactic and Lexical Interchange Program", and actually wrote an ELIZA in this non-existing language, which was in reality just Python code using the rexexp (re) package, and was perfectly fine as far as it went.
Even scarier than confabulating a non-exisistent programming language, it confabulated a citation for it, claiming that it is described in Weizenbaum's Computer Power and Human Reason, which is utterly false!
Sigh. I fear that ChatGPT is merely bringing Weizenbaum's worst fears into reality! A seemingly authoritative AI that simply makes s#!t up! I'm feeling a bit like taking up arms at the moment.