Except that we currently have no ability at all to make sure these models tell the truth? You could ask a LLM about a historical event and get weird BS as an answer. I think Facebook had a serious problem with its science language model a few months ago? Great teachers don’t scale is true, but LLM’s are not even minimally competent teachers.