Hacker News Reader

> These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs?

The input of this was a collection of data produced by humans. The by default behavior (if advanced) would presume to be human-like, and yet everyone seems to want to set the bar to be an all knowing oracle.

It's such a fundamental mistake people keep making.

There are a whole lot of humans out there. Not to mention there are 5 year-old humans, 3 year old humans. We seem unable to imagine for example a human child before developing theory of mind, but with an adult's vocabulary is still extremely human like. LLMs may not develop in conjunction with how humans develop, they may get some skills earlier some later. They may never get it from LLMs. But we need to get better at judging them for what they are capable, and not simply dismissing them because they aren't a perfect oracle nor a Phd grad.

Because you can't debate the AI. It's not on an open forum. People expect it to be an oracle because we interface with it as if it were.

> I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs

Axiomatically, Brains can't depended in their function on something they invented. People were hunting and skinning animals before learning to speak.

> in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high

well, firstly, if you as a business are offering "here is an API to our AI, it has the intelligence of a drunk conspiracy nut" maybe that is not a viable business.

Second, there is surprising ability of human mind to go to work, do complicated tasks, drive a car, and still believe in dumb shit.

My point is that a lot of debate surrounding LLMs is based around the idea that the brain appears to have structures that are innate to language, but that LLMs don't (or at least, not to the same degree; ISTM that tokenisation is one such trick). And to me, this means that there may exist further mathematical optimisations that the brain is using that LLM's don't. Making it likely that we will some day be able to encode LLMs in less space/energy.

At which time I suppose we'll be told that LLMs can't be intelligent for some other reason.

It definitely doesn’t have a comprehensive real world model like a human, but there’s probably something extra besides just statistics:

https://thegradient.pub/othello/

> They suggest language models can develop world models for very simple concepts in their internal representations (layer-wise activations), such as color [9], direction [10], or track boolean states during synthetic tasks [11]

But I don’t think there have been any probes like this for ChatGPT yet

This is better evidence than the Twitter example but I bet it still wouldn’t be well received in that thread

As a side note I just got GPT to write me some Go code and it came up with this:

    templates, err := embedFS.Sub("templates")

interestingly, this is not valid Go code (Sub is a function in the "fs" package, not a method) - but it would be quite typical in say Java.

So this mistake makes me think that GPT certainly does have a "mental model" for Go. And it's made the same kind of logical mistake I made when I first learned Go.

I wonder if it does better if you instruct it to 'pose' as a Go programmer. Or even one who comments to his younger Java-model self?

Also, you can have high general intelligence and still be confidently wrong.