Amazing how much the mainstream media are still obsessed with LLMs and factual accuracy. I prefer to see them as the worlds greatest innovation as lateral thinking machines ie pattern A applied over pattern B to give a credible pattern C.
The appearance of credibility of the output makes this much worse, since inevitably decisions will be deferred to LLMs that they are not sufficiently accurate enough for.
LLMs are best treated as the AI equivalent of a human assistant who is knowledgeable and fast, but also inexperienced, and thus, prone to making mistakes. You won't throw out the work of such an assistant-it'll still save you hours of effort. However, you won't take the work at face value either.
It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION".
I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answer...
For eg, a judge in India used ChatGPT for a bail hearing. https://www.livemint.com/news/india/this-indian-court-has-us...
One of the main differences between people who see the astonishing value of LLMs right now and people who are some combo of skeptical, dismissive, and indignant, is the expectation that for something to be a valuable source of information, it has to be factually accurate every time.
The entirety of civilization has been built on the back of inaccurate sources of information.
That will never change and it absolutely can't, because (1) factual accuracy is not something that can be determined by consensus in a variety of significant cases, and (2) factual accuracy as a concept itself does not have a consensus definition, operationally or in the abstract.
Absolutely frustrating to see these topics addressed as if thousands of years of intense thinking around truth and factual accuracy has not taken place.
The results of those inquiries do not support the basic assumptions of these conversations (i.e. that factual accuracy is amenable to exhaustive algorithmic verification).
Actually some of banks biggest failures were when only a few and in some cases only one person was in charge.
https://en.wikipedia.org/wiki/List_of_corporate_collapses_an...
How is that different from LLMs?
Of course they will, if the other benefits are large enough. Checking factual accuracy of a large corpus can often be considerably simpler than generating the large corpus to begin with.
just because a small percentage technically oriented people know the limitations of LLMs doesn't mean that the rest do.
people have a tendency to anthropomorphize things, so they naturally think that LLMs think in the same way other humans they know think.
edit: first link I found about it, there was also some posts about it here in HN. https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt...
Perfection isn't necessary, it only needs to be better than the average human. In a year or two I expect those kinks will be worked out for countless job tasks.
If I'm making a tool for doctors, I don't need to surface the exact scientific fact the LLM recalled from memory, I can design an interface that surfaces verbatim text from sources based on the LLM's understanding of the situation.
No serious product should be surfacing a ChatGPT style chat window to the user, it's a poor UX anyways with awful discoverability.
Also, at the moment you should view e.g. ChatGPT as your autistic well-read friend who really do know quite a lot of things, but only approximately, and is prone to make things up instead of being found out for not knowing. It's OK to ask your autistic friend how electricity works, but don't expect him to correctly cite the titles and authors of relevant research papers. Just common sense stuff if you really think of the LLM as a brain of compressed information and not a search engine.