If you were home and a family member asked you that question, you'd probably criticise the question rather than answering. LLM are RLHF'd into being milk-toast helpers that just try to answer questions like that with no criticism.
This is all beside the fact that the world of AI has changed pretty dramatically in the last few months.
It’s nonsense to test if a product that is marketed and sold as being able to provide generalised intelligence on demand, does what it says on the tin?
Check yourself
Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.
The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.
Incidents like hugging face are partly rooted in the lack of common sense. It still functions like a supercharged toddler.
I'd love to overcome this because it'd mean I spend less time guiding the the LLM to produce usable outputs.
And we've had difficulty as humans to childproof our sandboxes and infrastructure. Things that are otherwise innocuous spots to coordinate between like minded toddlers can become problematic.
consider me optimist now, but just few months ago, even frontier models were dumb, doing stupid mistakes all the time, all of them were so dumb I'd never expect anything to change in just few months.
This is always the issues in the discussions.
There’s the outcomes camp (objectivists?), which points at the things LLMs can do.
Then there’s the process methods camp, which talks about what is actually going on.
If you only care about the outcome, then the process does t matter.
If you are talking about what is happening, what the underlying mechanics and science of it is, then the process matters.
These models aren’t thinking. They simulate cognition well enough to do useful work in several fields and domains.
Both are true.
They are for any definition of the word that makes any kind of sense. I'm sure you have a contorted definition that magically only includes humans though...
The models are simulating thinking, if the fidelity is good enough for you - great!
The normal definition of the word "thinking" definitely includes what LLMs do. Hell people used to say computers were thinking even before AI. It's super weird to get all uppity about the semantics of the word now.
If a model isn’t able to count to 100, but is able to use python to do math, it isn’t thinking.
If you are only an outcomes person, then this difference doesn’t matter to you.
But when people are talking about AGI, anthropomorphizing LLMs, and making assumptions on its behavior based on that, then people are being loose or misled.
Language changes over time anyway, so even if you really believe what LLMs are doing isn't the "thinking" of 2024, it probably will be the "thinking" of 2027, because most people are using it that way.
And yes, we do need precision. Language is a vehicle to convey ideas, at its best it creates clarity. At its worst it creates deception and confusion.
What the labs are doing is designed to mislead. Being clear about the terms and avoiding needless anthropomorphization, is a basic decency we owe to the many others who are being riled up or terrified by the conclusions and implications that come from using “thinking” when it’s something else.
The difference between being satisfied by the outcomes and being interested in the process is sufficient to indicate the exactness required in your terminology, and does not harm any position.
These aren’t thinking machines, but if what they simulate is good enough, and your work ends with the output, it doesn’t quite matter.
People can be happy with a meal from McDonalds, or they can want something prepared by a chef, or Soylent.
All are acceptable and bring utility to their customers.
For "thinking" here we mean "assessing a representation of an object". That, or equivalent, is required to be reliable. So it is fundamental and critical.
But the outcomes group "ignores" the fundamental limitations of models which are purely text based.
E.g, a baseball players trains to catch high-speed balls and they dont do it by: "ball velocity 50mph, vector:[1,2,3], run move hand command now"
That's absurd.
No, there is an embodied network which is "trained" on visual, tactile input, and control as direct output.
LLMs are fundamentally not the right tool for that.
That is a NN that learns a skill.
But that is not an Analyst. If it were ballistics, then the answer to "how to parametrize the launch to reliably hit the target" excludes getting the result through natural skill.
The problem lies in the need to get "AI" facing "LLMs": the latter create a need for reliability, for "AI".
Speech is an endowment of both those who give educated guesses via developed skills and of those who return answers like Analysts, who check and compute. LLMs create a confusion between the two, and they will remain a problem until an ability to act as Analysts - strictly - will be implemented.
Dynamical systems will never be solved in a semantic domain.
IMO, they can be helpful the in robotics stack, from planning level up, but that's it.
What exactly is not clear? I will rephrase.
The internal process of the blackbox oracle determine the reliability of the output.
Two abilities are very different: learning trajectories through empirical training ("increasingly catching thousands of thrown balls"), and determining trajectories through computation (a rational thinker at work). The former is a finetuned parametrized engine (a «NN that learns a skill»), the latter is an Analyst. The former is fuzzy, the second deterministic.
In front of fuzzy LLMs, which use potentially misleading outputs - text ("has it guessed or has it thought?") - the urgency of warranties of reliable output gets evident.
So, that they «simulate cognition [only] well enough» (Intended wrote), and that there are «fundamental limitations of models ... purely text based» (Kooi wrote) raises the urgency to overcome the "fuzzy" and achieve the "deterministic" - it is not that we can stall on a «fundamentally not the right tool for that».
Inventing an oracle calls for urgent striving to overcome the original weakness.
Uh, no? So much of what we learn and take for granted as common sense is not learned via language, and not even expressible in it.