Oh yeah this is very much the case. Every time I ask ChatGPT something simple (thinking it'd be a perfect fit for an LLM, not for a google search) and it starts searching, I already know the "answer" is going to be garbage.
Just takes some prompt tweaking, redos, and followups.
It's like having a really smart human skim the first page of Google and give me its take, and then I can ask it to do more searches to corroborate what it said.