Hacker News Reader

Comment by rossdavidh | Hacker News Reader

So, if you dealt with a person who knew all the vocabulary related to a field, and could make well-constructed sentences about that field, and sounded confident, it would almost always mean they had spent a lot of time studying that field. That tends to mean that, although they may occasionally make a mistake, they will usually be correct. People apply the same intuition to LLMs, and because it's not a person (and it's not intelligent), this intuition is way off.

There is, additionally, the fact that there is no easy (or even medium difficult) way to fix this aspect of LLM's, and it means that the choices are either: 1) ship it now anyway and hope people pay for it regardless 2) admit that this is a niche product, useful in certain situations but not for most

Option 1 means you get a lot of money (at least for a little while). Option 2 doesn't.

View on HN

Yep. I think this is right on. The anthropomorphization in their behavior and problem descriptions is flawed.

It's precisely that analogy we learned early in our study of neural networks: the layers analyze the curves, straight segments, edges, size, shape, etc. But, when we look at the activation patterns, we see they are not doing anything remotely like that. They look like stochastic correlations, and the activation pattern was almost entirely random.

The same thing is happening here, but at incomprehensible scales and with fortunes being sunk into hope.

Even with a human speaker, that's not a totally safe assumption, and a certain type of fraudster relies heavily on people making this type of assumption (the likes of L Ron Hubbard in particular liked to use the language of expertise for various fields in a nonsensical way, and this is extremely convincing to a certain sort of person). But LLMs might almost have been designed to exploit this particular cognitive bias; there's really significant danger here.

Agreed. The optimistic scenario is that being exposed to so many "hallucinating" LLM's, people will become better at spotting the same thing in humans. But, I admit freely that is just the optimistic scenario.

I find that "intelligence" was a shaky concept to begin with, but LLMs have completely thrown the idea in the trash. When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

> When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

So you have a good definition for "intelligent", and it applies to LLM? Please tell us! And explain how that definition is so infallible that you know that everyone who says LLM aren't intelligent are wrong?

> So you have a good definition for "intelligent", and it applies to LLM?

No. I feel like that was my whole point.

I'm sure there's plenty I haven't realized, but the reason it's worth pointing out that LLM's are not intelligent, is that their boosters routinely refer to them as "AI", and the "I" in there stands for "intelligence", so pointing out that the label applied is not accurate, is important.

In that case, what AI system do you feel that the label can be applied to?

I don't think there is one. Many researchers had tried to switch the field to using "ML" instead, since it's a lot more accurate of a label, but it doesn't hype as well, and that appears to have been the decisive factor.

When I did my masters fifteen years ago, it was a masters in "machine learning". AI was already just what laypeople called it.

If your argument is that "this isn't AGI", I don't think anyone at all disagrees, but then that's a bit of a tautology.

Reply on news.ycombinator.com