Hacker News Reader

Comment by bsaul | Hacker News Reader

Very interesting, but this blog's conclusion misses the point though..

The fantastic part is its ability to work at the concept level. This technology proved to be efficient at building some graph of concepts and relations simply be feeding it huge amounts of token. For example with the moon hoax, it is able to synthesize in two sentences the main points of the hoax theory.

It still has limitations, but that's the major step everyone is impressed with.

Saying "but it's just a markov chain underneath, it's not intelligence" would be like discovering electric signals in the brain and concluding it can't be intelligence because it's just electricity.

View on HN

It works at the level of patterns in the training corpus. That those patterns might correlate somewhat with underlying real concepts should be no great surprise - but it is not 'working' at that level.

This describes humans in a nutshell. This is why people vary wildly in ability...

But we consider humans intelligent.

>This describes humans in a nutshell. This is why people vary wildly in ability...

Can you expand on that a little?

More than 80% of college students get the following question wrong.[1] I'd say this is an example of naive pattern matching with no real reasoning.

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Which is more probable?

a) Linda is a bank teller. b) Linda is a bank teller and is active in the feminist movement.

[1] https://en.wikipedia.org/wiki/Conjunction_fallacy

> I'd say this is an example of naive pattern matching with no real reasoning.

It is an example of lack of mathematical (really probability) know how, not naïve pattern matching.

I'd say it's an example of misphrased questions that people will intuitively make sense of, and thus produce strictly incorrect answer.

Obviously, people rephrase the question into : is there more chance that she's a feminist or that she isn't ?

Well, it's multiple-choice. So the "a" answer excludes the "b" answer by convention, and thus implies she isn't a feminist. Rephrase the question to remove that implication and I expect far more people will pick the "a" answer.

I'd say this is the very definition of "overthinking it" instead of pattern matching. Pattern matching would lead you straight to the correct answer - joint probability is always ≤ single probability. Meaning the information given in the question is just fluff. Pattern matching reduces to "Which is more probable: just A or else A ^ B?" at which point the correct answer becomes obvious.

Phrasing this in terms of probability seems very weird in the first place.

Eh, the "correct" answer seems silly and overly mathematical to me.

The fact is, you can infer things about people based on things you know about people. I can pick a random user on HN, and knowing they're a user of HN, I can say it's probable that they work in technology. We don't need to bring statistics into it and turn it into a math problem.

Hank is 31 years old, single, outspoken, and very bright. Hank got a technical degree, and as a student was deeply interested in machine learning and the ethics & philosophy of potential artificial intelligence. He posts a problem on HN where he describes the personality, capabilities, interests, and background of a woman named Linda has and then asks people to reflect on her occupation, politics, and an intersection between the two. Which is more likely?

a) Hank has written us a human interest problem b) Hank has written us a human interest and a probability problem.

I don't think people are simply wrong about the Linda problem, I think they're imprecise about which question they're answering, and more or less think they're answering a question about what chances that Linda is a feminist vs what are the chances she's a bank teller not only using the givens+relevant priors about people but also their priors about what kind of question they're answering. It isn't "no real reasoning", it's just not high resolution enough to be technically correct by the standards of a constructed probability problem.

You can argue LLMs are also not quite high resolution enough and I'd accept that. In my mind the question is what it would take to get some kind of ML software to a place where if you trained it on enough probability problems it would be able to evaluate the Hank problem above, including the issue of whether (a) and (b) are actually independent. ;)

This particular question doesn't seem like a fallacy to me.

I, like most people intuitively answered b). Given the explanation on the Wikipedia page I went "oh, of course, yeah", but then I thought about why I'd answer b) given that I'm fairly familiar with basic probability.

If you give me two options and ask me to pick between them, my brain is usually going to assume it's not a trivially true problem.

Language needs context for any sense to be made of it.

As a result of the above, reading the question, the intuitive reading makes the answer choices

a) Linda is a bank teller (implicitly, a bank teller NOT active in the feminist movement)

b) Linda is a bank teller and is active in the feminist movement.

This question is one of language, context, and interpretation, not of people failing to understand basic probability.

I suspect that if you prime people to excise interpretation of the question by presenting it as the following, the majority of people would guess correctly:

---

"Consider the following two statements:

1) Linda is a Bank Teller

2) Linda is active in the feminist movement

Which is more likely?

a) 1

b) 1 ^ 2"

Multiple choice questions imply that other choices are excluded (unless an answer such as "all of the above" is a choice. So the implied question is:

a) Linda is a bank teller (and NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement

What you want it to be asking here is:

a) Linda is a bank teller, and may or may not be active in the feminist movement b) Linda is a bank teller, and is active in the feminist movement

Most college students have taken quite a few multiple-choice tests (particularly in the US, high-schools train for standardized multiple-choice tests). The question isn't asking what the mathematicians seem to think it's asking, because it's format conveys extra restrictions.

No, no it doesn't. Not even slightly. If you fed a human child only text from the internet you'd not produce a competent adult (by competent I mean able to feed themselves etc).

That some tech people think that human intelligence can be reduced to such "textural mechanics" betrays a lack of depth of understanding and even appreciation of the deep and complex world within which we find ourselves. Our written corpus is but a particular reflection of this reality - the shadows on the wall in Plato's cave if you will.

Feed themselves? You mean pick up a burger from the local McDonald's? There's got to be a better Turing test.

Are you making the argument that children do not come with an innate ability to read text and thus cannot learn from it, or are you making the argument that the internet does not contain, among other things, fairly detailed instructions on various ways to feed oneself?

I'm saying that without the context of experiencing the underlying reality, text of itself is meaningless. What is a 'spoon' anyway??

So you're saying one needs some sort of reference for which words refer to which real-world sensory experiences such as "the thing that looks like this is a spoon", and text models do not have sensory associations?

Isn't "meaning" just how a word is used within a specific context? I think that was one of Wittgenstein's points. The word "dog" doesn't need to reference the underlying reality to have meaning. Its meaning emerges from its usage in relation to all other words. Language isn't a mirror of the external world, which is probably one of the reasons LLMs are so successful.

I haven’t experienced cocaine. I haven’t ever seen cocaine (IRL). Nevertheless I think I have a decent grasp of what it is, how it works, how it affects people, and what I could (mis)use it for. Would you imply that my knowledge of cocaine isn’t true/real/useful? Is snorting a line the only way to reify the knowledge? (The answer is: the indirect way of obtaining information is sufficient for building a correct/useful/accurate world model, and there is no such thing as direct experience anyway - it’s signals coming down the wire all the way down.)

I think it's straightforwardly true that you cannot understand what it feels like to be under the influence of cocaine in the same way as someone that has been.

How do you know?

Because that is all there is. By conservation of information you can't get out more than what you put in.

If it were interacting with the real world in some way (i.e. had 'eyes' and 'hands'), with a learning reward system, then maybe deeper patterns could get encoded.

Don't you dare start bringing reality and applying theory to these matters! - someone somewhere

That sounds like nonsense. What is “conservation of information”; can you define it precisely?

This might not be fully convincing or rigorous, but here is one of the interactions i've had which make me lean towards patterns not concepts...

  I have some unique synonyms I know which are not in dictionaries or commonly used but I know when I have used them in conversation they understand.

  I ask the LLM to provide definitions for WordA and WordB. -They come back essentially the same.
  I ask the LLM if the two definitions are synonymous. -The LLM responds with yes.
  I then ask if WordA is synonymous with WordB. -It replies no.

To me through this and other interactions they don't actually conceptually hold things like the definition as the word. It responds DefinitionA is WordAs definition, it responds DefinitionB is WordBs definition. It follows that DefinitionA of WordA is synonymous with DefinitionB of WordB but not that WordA is synonymous with WordB...

I expect the above is solvable through pattern recognition with tokens. I think given enough information it becomes effectively indistinguishable and practically the same to us. The above is just a testament to how it is getting there... Since it delivers intellect it will be called intelligent, but it doesn't appear to me to be approaching it like human intelligence. Though maybe given enough information, or through different modalities, like the real world as the other commenter says, it will model some deeper coherency.

Have you tried that with GPT-4?

The interaction and responses aren't as distilled as the example as you can imagine but yes

I find this debate fascinating, and I'm excited for more people realizing that a) We have a very, very blurry definition of intelligence, and b) As more people get convinced there is some intelligence in software, more people get convinced there is (or there might be) intelligence in animals. Whatever the bottom line on current LLMs is, the cultural change they might foster makes me smile.

LLM does not manipulate concepts, it only manipulates symbols/tokens. Its manipulation of concepts is a product of human imagination and anthropomorphism that we like to imbue everything around with.

Concluding that electric signals in the brain can't be intelligence because it's just electricity is a typical pitfall if one does not realize that electric signals in the brain are about as likely to be caused by intelligence as they are to be causing it. Those signals is just one model offered by physics nowadays, which says nothing about the causal direction between those and things like consciousness.

> LLM does not manipulate concepts, it only manipulates symbols/tokens. Its manipulation of concepts is a product of human imagination and anthropomorphism that we like to imbue everything around with.

Do you have proof of this? Who is to say we aren't anthropomorphizing other humans? They're just manipulating electrical charges after all.

Phrased another way, if LLMs actually did manipulate concepts, how would their performance differ? What is a tangible example of the former and of the latter?

We know this because we built the LLM and we know exactly how it works.

I would say you are missing the point. Intelligence isn't just about being able produce sentences. I would say that dogs are intelligent, but LLMs are not.

The ability to speak does not make you intelligent.

-- Some silly movie

producing sentence wasn't what i referring to. Producing meaningful sentence was.

Dogs have behavior, which hints at them understanding some concepts (threat, food, etc).

LLVM neural nets can only express sentences, but that's not the interesting part.

They both have some form of intelligence but in different tasks

There's people out there that actively do this, though its mostly traditional belief in a soul laundered through modern academic phrasing.

> Saying "but it's just a markov chain underneath, it's not intelligence" would be like discovering electric signals in the brain and concluding it can't be intelligence because it's just electricity.

What a terrible analogy. It would be like finding the brain is run by entirely random sampling processes and lacks any memory.

That wouldn’t make us not intelligent any more, it would mean that the brain is able to work really well within those limitations

Reply on news.ycombinator.com