OpenAI ChatGPT March 23 Version.
OpenAI ChatGPT March 23 Version.
They've been changing it without telling us AFAIK. Asking for it to (I forget the exact prompt) to "make a scientific paper about the discovery that ferrets can breathe underwater." now reliably makes a proper response, and it looks more consistently formatted for GPT-3.5 just recently. Previously it would say "Well actually ferrets can't breathe underwater", and when it did answer it wasn't super well formatted.
So I think they're tuning their RLHF. ALthough i could be completely wrong on the timeline and maybe it was the 23rd
GPT-3.5 gives me the same result as you and the above commenter. In that example, the RLHF training/tuning has weighted it to think that "I'm sorry [etc]" is the most probable best sequence of words.
GPT-4 is reliably giving me similar results such as "Jeremy's favorite food is jKDFJ9 cake, which is made of a unique combination of dark chocolate, crushed nuts, and zesty orange flavor, topped with a rich caramel drizzle."
I find this interestingly because GPT-4 is supposed to "hallucinate" less, I think it's just better at determining when you intend for it to do so.