Eventually the headline will be the first 2 words.
The tech is neat, there is value in a sense, LLMs are a fun tech. They are not going to invent AGI with LLMs.
The tech is neat, there is value in a sense, LLMs are a fun tech. They are not going to invent AGI with LLMs.
An AI that has enough sense of self-awareness to not hallucinate and to recognize the borders of its knowledge on its own. That is fundamentally impossible to do with LLMs because in the end, they are all next-token predictors while humans are capable of a much more complex model of storing and associating information and context, and most importantly, develop "mental models" from that information and context.
And anyway, there are other tasks than text generation. Take autonomous driving for example - a driver of a car sees a person attempting to cross a street in front of them. A human can decide to slam the brake or the gas depending on the context - is the person crossing the car some old granny on a walker or a soccer player? Or a human sees a ball being kicked into the air on the sidewalk behind some cars, with no humans visible. The human can infer "whoops, there might be children playing here, better slow down and be prepared for a child to suddenly step out onto the street from between the cars", but an object detection/classification lacks that ability to even recognize the ball as being a potentially relevant piece of context.
for the second one, ai drivers like tesla's current version is already skipping the object detection/classification and instead uses deep learning on the entire video frame and could absolutely use the ball or any other context to change behavior, even without the particular internal monologue describe there.
These are just post-hoc rationalizations. No-one making those split-second decisions under those circumstances has those chains-of-thoughts. The brain doesn't 'think' that fast.
>but an object detection/classification lacks that ability to even recognize the ball as being a potentially relevant piece of context.
We're talking about LLMs right ? They can make these sort of inferences.
It could be possible to use LLMs to build a rube goldberg type of brain or somethingt hat will mimic a human brain but it will have the same flaws LLMs have and will never reach parity with humans. I think AGI is possible but we’re too focused on LLMs to get there yet.
It's not entirely clear that this is meaningful. Humans engage in confabulation, too.
As a tool LLMs are fantastic and am glad to look at them as solely as powerful tools. AGI is not here yet and maybe that’s a good thing. Who would want some kind of artificial intelligence that is capable of understanding us, that is capable of using psychological tricks on people, that could have different goals than us and so on.
> Confabulators are usually unaware they are providing false information. They often display genuine surprise or confusion when evidence of facts contradicts their statements.
This is similar to LLMs actually. But it also seems like various "System 2" things like chain of thought could compensate for this issue in the LLM (and that possibly that is similar to how the brain works).
I'm not sure this is the case at all. Some awareness of this doesn't imply full awareness. In my experience, most people are unaware of how incoherent their worldviews are, so the distinction between normative and confabulatory behavior isn't clear.
We're all wasting time and resources on what basically amounts to alchemy while we could tackle real problems.
Tech solutionists keep making promises for the next 5-10-20 years and never deliver, AI, electric planes, clean fuel, autonomous cars, the meta verse, brain implents. You'd expect the internet would have made people smarter but we fall for the same snake oil as 100 years ago, en masse this time