The problem is that if you don't already know the answer, it's hard to tell the difference between ChatGPT giving you the correct information and ChatGPT authoritatively spouting the wrong information.

Here's a chat I had today with ChatGPT:

Q: Who was the first US president with a last name starting with R?

A: Rutherford B. Hayes was the first US president with a last name starting with "R."

Q: no, a surname starting with R

A: Rutherford B. Hayes was the first U.S. President with a surname starting with "R."

Q: no, Rutherford is his first name

A: The first US president with a surname starting with "R" was Richard Nixon.

Q: no, Richard is his first name

A: The first U.S. President with a surname starting with "R" was James A. Garfield.

Here it's obvious ChatGPT is spouting nonsense, but in other cases it may not be.