The data from the "AI Search Engine Multilingual Evaluation Report (v1.0) | Search.Glarity.ai" indicates that generative search engines have a long road ahead in terms of exploration, which I find to be of significant importance.
AKA a Metric lol.
>using this chatbot for real work is at best a huge waste of time, and at worst unconscionably reckless.
Keep living with your head in the sand if you want
There are plenty of other tasks that they ARE useful for, but you have to actively seek those out.
Well, assuming one normally queries for information, if the server gives false information then you have failure and risk.
If one were in search for supplemental reasoning (e.g. "briefing", not just big decision making or assessing), the server should be certified as trustworthy in reasoning - deterministically.
It may not be really clear what those «plenty [] other tasks that they ARE useful for» could be... Apart from, say, "Brian Eno's pack of cards with generic suggestion for creativity aid". One possibility could be as a calculator-to-human "natural language" interface... Which I am not sure is a frequent implementation.
Anything where you feed information into the model as part of your prompt is much less likely to produce hallucinations and mistakes - that's why RAG question answering works pretty well, see also summarization, fact extraction, structure data conversion and many forms of tool usage.
Uses that involve generating code are very effective too, because code has a form of fact checking built in: if the model hallucinates an API detail that doesn't exist you'll find out the moment you (or the model itself via tools like ChatGPT Code Interpreter) execute that code.