I have played around with ChatGPT and noticed that it "invents" papers and references. But yeah, I feel like it is unlikely (not impossible) that a peer reviewer would actually not even check the outputs.
Another example was where it said that a certain study found a specific thing. I asked it for a link to the study and it responded that it never referred to any study. So I pointed out that it did refer to a specific study and it apologized again and gave me the wrong link.
Overall, it’s almost like it pretends to know more than what it knows but is overconfident.
The main value of ChatGPT for me (so far) is not for tasks where I get information from it but where it gives me information to start something. "Write me an authorization letter", "create a 10-slide presentation on this topic", or "I have this function, can you write some unit tests" has been very useful for me. Not because the outputs are correct but they lower the inertia for me working on tasks.
I rarely get outputs that work 100% out of the box that I don't have to fix or reshape. One personal rule is that I never ask it stuff that I normally won't be able to do by myself.