I've been trying to use LLMs for things and it makes mistakes all the time. Just this week i had multiple instances of various LLMs basically saying "just run the software with --flag-that-fixes-your-problem" or "edit the config and add solve-your-issue=true" hallucinating non-existant options. Even if i manually link the relevant documentation pages it will still just make basic mistakes. and if im having to read the documentation myself anyway to fix the AI's mistakes, why is the AI even in the loop.
its infecting search too, because blogspam/slop articles are managing to make their way into search results by just making up untrue information, claiming software can do things it cant, or has options that don't exist.