Yes, who cares if the answer is hallucinated, as long as it's easy to read and stated in clear, confident-sounding language.
Or they work for a selection of cases but not others. Or they seem to work in a limited number of tests but aren’t truly doing the right thing. Or they do work but have a security flaw that makes your service vulnerable.
Coding solutions have tradeoffs and subtleties which go beyond “it did the thing right a couple of times when I ran it”. If that’s all it took to verify code is good, there wouldn’t be any bugs; you can test code a person writes too.
But contrast this with other types of information. If I ask ChatGPT "how hot is the surface of the sun?" and it answers "15 million degrees Fahrenheit", I have zero ability to verify that information. As a result, I don't find ChatGPT useful for most informational searches, except the ones related to coding.
(The example answer is wrong according to https://www.space.com/17137-how-hot-is-the-sun.html, albeit in a way that I could see potentially confusing an LLM.)
But the nice thing about ChatGPT is that you can give it more context than it is possible in a Google query, and you can ask follow-up questions.
They are still useful, you just have to make sure you understand what you are doing and read the relevant docs before you paste the code.
Things like "How do I delete a value from a dict" probably either works or it doesn't.
Then again, for that level of simple stuff, one can probably search a PDF cheat sheet just with their eyes faster than they can type the question for ChatGPT
On the surface it makes no sense to do this since it already gave you an answer. But that's not how LLMs work - they can simulate a variety of different "humans" depending on the prompt and presenting a different prompt persona (security researcher, code reviewer that focuses on edge cases, etc) yields a different quality of response.
And in general I find it’s more accurate and far more concise than anything that’s going to show up in a Google search.
I mean, there’s no way I’m taking what it says at face value or copy / pasting code from its response, but that goes for stack overflow, or any other source too. (maybe sans official docs)