Playing games with AIs: The limits of GPT-3 and similar large language models

Large language models might be excellent candidates for evolutionary methods and RL. They need to learn from solving language problems on a massive scale. But problem solving could be the medicine that cures GPT-3's fuzziness, a bit of symbolic exactness injected into the connectionist system.

For example: "Evolution through Large Models" https://arxiv.org/abs/2206.08896

They need learning from validation to complement learning from imitation.

I think the difference is what "learned" means. This paper basically says that any finite amount of knowledge compression is learning, whereas the other camp defines learning as some kind of infinite information compression like being able to add any two numbers no matter how large, which is something no language model will ever be able to do.

Personally, I think both sides are right, GPT3 has compressed language patterns across the internet, but its undeniable that GPT3 makes up stuff frequently. Overall, the point still stands that these language models are valuable in some specific contexts, but its not clear how far they can go.

I think text prediction only gets you so far. But I guess you could use the same principles to predict the next symbol in a binary string. If this binary data represents something like videos of physical phenomena, you might get the AI to profound, novel insights about the Universe just with next-bit prediction.

Hmmm, maybe even I could code something like that.

Paper contributes to debate about abilities of large language models like GPT-3

Evaluates how well GPT performs on the Turing Test

Examines limits of such models, including tendency to generate falsehoods

Considers social consequences of problems with truth-telling in these models

Proposes formalization of "reversible questions" as a probabilistic measure

Argues against claims that GPT-3 lacks semantic ability

Offers theory on limits of large language models based on compression, priming, distributional semantics, and semantic webs

Suggests that GPT and similar models prioritize plausibility over truth in order to maximize their objective function Warns that widespread adoption of language generators as writing tools could result in permanent pollution of informational ecosystem with plausible but untrue texts.

The "Hill" example in the text is easily understandable by the author's own presented concept of "text-based, word placement, associative semantics" vs "semantics by definition" we obviously use semantics in the latter sense, hence it's definition, the AI doesn't.

Semantic word relationships identified by GPT3 are based on the frequency of words prior to and following a position in a sentence/text – as presented to the AI in the programmed/learned dataset. Another easier example is when information is known to be untrue. If I include 1,000 written examples of the words “Jack and Jill ran down the volcano” than my AI will be incorrectly answering prompts to finish the nursery rhyme. How many instances of providing users the wrong answer or number of analyzed writings with the correct “ran down the HILL” text before my 1,000 false volcano statements ceases to be the most probable and likely accepted answer.

So, like the article example, if asked "Where was John Smith born?" the AI sees that it has to answer definitively bc its been asked question, so it's going to make a statement it concludes to be the most probable acceptable answer to the prompt - it doesn't see the prompt as a question, the words are not defined as ideas of themselves, nor do the sum of the words present an idea. Word definitions are not really part of the answer process. The AI checks it's dataset and knows all the related word examples it's previously identified through its handy token system – that self-controlled tokenization of memory for storage & retrieval, further removes this from a human brain-like function/process we can empathize with.

Anyways, the AI knows that most statements following words arranged like the question in the prompt include words/textual identifiers not used similarly used in other places of texts – Names! Imagine how it figured out names without understanding definitions conceptually by using frequency of word composition and grammar structure alone. Even though it “knows” the definition of the word “name” that definition is just more words – it has no meaning without context. Prompts provide context not definitions.

There are 3 names in the example prompt question, first/last name and the name/word required as an answer – the location! None of these words the AI see as a person's name or the name of a place, it sees our question as "word that requires my reply to be a definitive statement, primary “name word”, secondary “name word” + "born" --- it knows a different type of name word (a place) is the most plausible next word in sequence because it has a whole token dedicated just to birthdates, with limitless examples. Upon searching its dataset for John Smith, identifies "Hill" as the name word most often associated with the words “where” “was” “john” “smith” “born” used consecutively. The incorrect city, Hill, makes sense given his academic career obviously generating more digital information than his birth announcement/obituary in the hometown paper.

Regarding the wrong date – the AI was never actually answering a question and never made any statements with intent to be truthful. The incorrect birthdate is simply the most probable date given the incorrect "John Smith born in Hill" statement. It couldn't present the correct date following the word “Hill” bc no such examples existed with a higher probability of being acceptable than the incorrect date given. In fact, given the incorrect semantic links made early in the answer, an incorrect date was most probable.

None of that is compression loss. It's just an AI being an AI, doing exactly what it does. I think it's obvious, based solely on what the authors presented themselves, that it's in fact recalling everything - only arriving at a failed answer due to differing expectations of what the answer was. The AI delivered the most probable reply to the prompt, given the contextual data available to it – the same way it delivers answers we expect and are factually correct. It didn’t draw incorrect conclusions bc it chunked everything it learned up and consequentially “lost” some its “memories” in the process.

Programming AI with facts, or only factual information, doesn't solve the problem at all - operating with only factual data would help it regurgitate a "born in this town on this day" type answer more correctly but only bc the token words identified as correlated, in a factual text, do in fact have actual correlation. That only increases the probability of an AI arriving at a "correct" reply/answer while still using the flawed “logic” that allowed for these errors to occur. An AI that speaks only true statements will still have no actual concept of truth.

“Prediction leads to compression, compression leads to generalization, generalization leads to computer intelligence.” - quote from the article.

I know when ppl do this, memory chunking, we do lose stuff – why do we assume that is true for an AI also? What exactly are they compressing? Our memories are filled with lots and lots of data beyond the reason a memory is a memory. The background clutter, noises of a crowd, cars driving by or what lunch was that day, are not necessary to recall the memory of your first kiss, for example – unless you were in a crowded cafeteria at a racetrack, that might be all you recall then. A kiss, loud crowd, race cars – from an entire day of activity, those highlights will be all that remain in time. We need to do that - even with that feature we still forget important things.

What background noise is an AI having to “chunk” away? The parameters for it too broadly set? - narrow them. If it sees too much, we tell what not to see. If it's capacity to effectively store and utilize information encountered as it exists, than we have failed to create an effective AI.

If an AI “reads” a 500 page paper and tokenizes the data – what makes you so sure it cannot recall exactly all 500 words from that token alone?

AI compression loss, in a tokenized type system, would have to derive from the further compression of the tokens themselves or failure with the token system.

Just my quick 1,000+ words

Yeah... sry for the book.

Tl;dr -

I find the idea of AI being wrong due to “compression loss” to be a silly concept.

We should avoid humanizing AI and AI learning – all similarity lies on the surface.

Thanks for reading my rant – have a great day! - Jakksen

GPT can (only) mimic speech it is trained on. It can sound or read like real world human speakers it is mimicking. But it can not REASON about whether what it is saying is "true" or logically consistent.

It can not reason. Intelligence requires the ability to reason, logically, right? Therefore I posit GPT is not intelligent. Therefore it can not be AI.

I haven't used GPT but I wonder what happens if you ask it to explain its reasoning to you? What happens if you ask it whether it thinks what it says is true and why it thinks so?