We hold AI to a pretty high standard of correctness, as we should, but humans are not that reliable on matters of fact, let alone on rigor of reasoning.
Grep works just fine, despite implementing non-deterministic finite state machines. Monte Carlo simulations are behind nuclear weapons (where they were invented), weather forecasts, financial trading, etc. Las Vegas algorithms like randomised quicksort also diminish no one's responsibility.
In principle, you can run training and inference on neural networks completely deterministically. But I don't think that makes any difference to responsibility. (To make them deterministic, you obviously have to use pseudo-random number generators with fixed seeds, but less obviously you also have to make sure that when you merge the results of parallel runs, the results 'merge' deterministically. Deterministic parallelism is an extremely interesting field of study! Or, since we are only talking about principles, not what's practical, you could just run everything in series.)
The problem with LLMs is that they are complicated and their actions are hard for humans to predict or reason through. Complexity is the bane of responsibility: if you have a complicated enough system (and a complicated enough management structure involved in producing that system), that's where responsibility goes to die, unless you specifically work to establish it.
In this case, employing LLMs is no worse than employing humans. If upper management gives bad instructions and incentives for lower level employees, we tend to pin the responsibility on upper management.
Disagree that it's easy to pin on "how they were brought up". It seems very likely that we may learn that the flaws are part of what makes our intelligence "work" and be adaptive to changing environments. It may be favourable in terms of cultural evolution for parents to indoctrinate flawed logic, not unlike how replication errors are part of how evolution can and must work.
In other words: I'm not sure these "failures" of the models are actual failures (in the sense of being non-adaptive and important to the evolutionary processes of intelligence), and further, it is perhaps us humans that are "failing" by over-indexing on "reason" as explanation for how we arrived here and continue to persist in time ;)
Indeed. That might play a role, but another less politically charged aspect to look at is just: how much effort is the human currently putting in?
Humans are often on autopilot, perhaps even most of the time. Autopilot means taking lazy intellectual shortcuts. And to echo your argument: in familiar environments those shortcuts are often a good idea!
If you just do whatever worked last time you were in a similar situation, or whatever your peers are doing, chances are you'll have an easier time than reasoning everything out from scratch. Especially in any situations involving other humans cooperating with you, predictability itself is an asset.
Only to continue to reaffirm the original post, this was some of the basis for my dissertation. Lower-level practice, or exposure to tons of interactive worked examples, allowed students to train the "mental muscle memory" for coding syntax to learn the more general CS concept (like loops instead of for(int i = 0...). The shortcut in this case is learning what the syntax for a loop looks like so that it can BECOME a shortcut. Once its automatic, then it can be compartmentalized as "loop" instead of getting anxious over where the semicolons go.
I wonder what responsiveness/results someone would get running an LLM with just ~20 watts for processing and memory, especially if it was getting trained at the same time.
That said, we do have a hardware advantage, what with the enormous swarm of nano-bots using technology and techniques literally beyond our best science. :p
Humans and human language have co-evolved to be compatible. Language makes no such allowance for the needs and quirks of LLMs. (However to a certain extent we design our LLMs to be able to deal with human language.)
I'd say most humans most of the time. Individual humans can do a lot better (or worse) depending on how much effort they put in, and whether they slept well, had their morning coffee, etc.
> If anything, the current limits of these morels show the limits of human cognition which is spread throughout the internet - because this is literally what they learned from.
I wouldn't go quite so far. Especially because some tasks require smarts, even though there's no smarts in the training data.
The classic example is perhaps programming: the Python interpreter is not intelligent by any stretch of the imagination, but an LLM (or a human) needs smarts to predict what's going to do, especially if you are trying to get it to do something specific.
That example might skirt to close to the MuZero paper that you already mentioned as an exception / extension.
So let's go with a purer example: even the least smart human is a complicated system with a lot of hidden state, parts of that state shine through when that human produces text. Predicting the next token of text just from the previous text is a lot harder and requires a lot more smarts than if you had access to the internal state directly.
It's sort-of like an 'inverse problem'. https://en.wikipedia.org/wiki/Inverse_problem
The ability to circumvent these limitations, is encoded in company procedures, architecture of hierarchies/gremiums within companies and states. Could AI be "upgraded" beyond human reasoning, by referencing these "meta-organisms" and their reasoning processes that can produce things that are larger then the sum of its parts?
Could AI become smarter by rewarding this meta-reasoning and prompting for it?
"Chat GPT for your next task, you are going to model a company reasoning process internally to produce a better outcome"
This should also allow to circumvent human reasoning bugs - like tribal thinking (which is the reason why we have black and white thinking. You goto agree with the tribes-group-think, else there be civil war risking all members of the tribe. Which is why there always can only be ONE answer, one idea, one plan, one leader - and multiple simultaneous explorations at once as in capitalism cause deep unease)
I never understood this line of reasoning.
1. Humans can't run faster than 30 mph.
2. Therefore we can't complain if cars/trains/transport always go slower than 30 mph.
These comparisons also hide that we are comparing best of AI (massive LLMs) with median/average human reasoning.
At the same time I see such negative sentiment around the capabilities at their current limits.
We are reaching an era of commodified intelligence, which will be disruptive and surprising. Even the current limited models change the economics dramatically.
Yes, though at the moment they hype is still a lot bigger than the impact.
But I am fairly confident that even without any new technical ideas for the networks themselves, we will see a lot more economic impact over the next few years, as people work out how to use these new tools.
(Of course, the networks will also evolve still.)
But there's a lot of stuff they can't really do (in their current form), or can't do reliable, yet.
> Every non-technical person I know that's still working has used ChatGPT for work at some point, and quite a few of them are using it regularly. And I'm nowhere near Silicon Valley or any serious tech hub.
Yes, that makes me optimistic for their future, too.