LLMs can't do anything reliable AFIK
they are too much probabilistic to be fully "reliable"
at least if we take the definition of reliable from well written programs or even proof based programs
if we take the definition of reliable from idk. how reliable a underpayed and not well treated employee is then ... it might be a different thing
I think the core issue is perhaps that they don't have any personal experience, nor know the trustworthiness or core competencies of their sources. Their training material is just an undifferentiated mass of text - some might be the writings of Richard Feynman, some the writing of someone on 4chan.
Us humans know the value of personal experience and expertise. If we have seen or done something ourselves, applying our own learned skepticism and knowledge, then we treat that as much more trustworthy than something posted on 4chan. If we read something in a book by a well known expert, or hear it from an acquaintance we have learned to trust, then we also treat that as likely to be true, although perhaps still not as much as if we've seen/tried it for ourselves (for which there is little substitute, since only then can we experimentally test any surrounding assumptions for the transmitted knowledge to put it in context).
Personal experience would require architectural changes, not least online learning. No need to anthropomorphize - our brain is also just a machine, and we can use it as inspiration for what is missing to build more intelligent systems.
"Here's some information on that" (links to WikiPedia page)
Are you speaking just out of anecdotal experiences? Try something simple like +1 to an integer. Or swapping an ascii character. Neural nets are layers of function approximations, and plenty of those weights were trained beyond the precision needed to map the entire set of syntactically correct inputs to the correct output, for a given well-written traditional function. Remember to set the temperature to 0 (deterministic). Of course, as you ask more complex things you quickly dip below the threshold of perfection. But to say they can't do anything reliably isn't true.
-641114711832495315269166494840 + 1 =
-641114711832495315269166494840 + 1 =
-641114711832495315269166494840 + 1 = -641114711832495315269166494840 + 1
The result of this calculation is:
-641114711832495315269166494840 + 1 = 641114711832495315269166494841
This is more "project this result onto a context -specific human like space of answers" than "compute it all in LLM"
A failure in this particular space in an observability product we use leads people wildly astray. While the data is correct, the summary is definitely not because it does not have enough understanding of the context.
I said to him "Would you really take a report into a meeting without knowing 100% what it was going to say on it beforehand? Even in the summary, do you want to risk presenting some wildly inaccurate LLM output to your boss and peers?"
Essentially: Accuracy is only one thing we want out of software. Replicability is another one. LLMs are terrible at the latter and only "good enough" at the former
Better use cases for LLMs are either where:
a) There is a non-judging human in the loop who is using the LLM as a tool, and accepts it's errors/limitations. It's a bit like asking a co-worker for advice - they might have something useful on occasion, but you're not just going to take what they say unchecked and put it on the company web site or send it to your boss as accepted truth.
b) You can verify the output of the LLM in someway - maybe easier if it's a structured output of some sort (SQL query perhaps) rather than free-form text.
b) if it was that deterministic, then you could generate it without an LLM.
Other than just playing with them, the only place I've personally found any utility in LLMs is essentially as a kind of search to get a toehold in a subject matter I'm not familiar with.
For the summary thing, I think we eventually settled on a workflow that generates the summary using LLM, then we store the output in our dataset for the report. This also allows the human building the report to modify it if necessary
This is different from say code generation where, to first approximation at least, automated tools (compilers, unit tests) can do this cheaply.
I don't see an equivalent approach that can be applied to LLMs.