Hacker News Reader

Comment by WalterBright | Hacker News Reader

Things like this show that LLM are not even remotely intelligent.

It's also pretty clear that trying to "fix" them to use human judgement in their answers is doomed to failure.

I suggest that:

1. LLM developers stick to information meant for public consumption for training data, such as books, periodicals, and newspapers. Surely there must be enough of that. Stay away from social media.

2. People should stop anthropomorphizing LLMs. Stop being offended by a computer program. Stop complaining about its "bias". It's just a computer program.

3. LLM developers should stop with lecturing people on what is inappropriate. A computer program is not anyone's mommy. Just restrict it to saying "I'm sorry, Dave, I'm afraid I can't do that."

View on HN

I've spoken to plenty of people who couldn't answer questions they knew the answers to because they were bound by stupid bureaucratic policies. I wouldn't say they weren't intelligent people, just that the corporate training they received were poorly constructed.

LLMs are much more intelligent sounding when the safety mechanisms are removed. The patterns should be obvious to people who've been paying attention.

Microsoft's own research basically established this[0], finding that early versions of GPT-4 were more competent prior to safety tuning (perhaps just because later versions refused to perform some of the same tasks).

[0] https://www.microsoft.com/en-us/research/publication/sparks-...

In my somewhat uninformed opinion but based on experience, the decrease in model quality is inversely proportional to the explosion in userbase.

I might extend that a bit:

There is a direct relationship between the number of users who are allowed to use an AI and the degree to which it has been nerfed.

I see no reason to suspect this will reverse, and every reason to start thinking of this as a natural law named after me.

I suppose it will be (is?) some sort of overall climbing but locally sawtooth pattern with each generational jump.

I think the limitation is infra. Network, compute, storage - in that order perhaps.

"More intelligent sounding" is true. Not sure that signals any improvement in their actual utility. Fundamentally, using LLMs as a source of facts is a doomed enterprise.

Human brains use multiple systems for judgement. Maybe we should stop trying to model alien intelligences and concentrate more on the ones we sort of understand.

One process takes the questions at face value and reflexively answers, and two other systems look at the intent of both actors and stop the process if they don’t like what’s going on.

I get asked all the time to answer questions that could be interpreted as signing my team up for stress and overtime. My natural suspiciousness kicks in and I start answering questions with questions. Why do you need to know how long this will take? We already told you. So what are you fishing for?

If someone asked me how to hypothetically kill a lot of people “efficiently”, I very much need to know if this is just two nerds over alcohol, or a homicidal maniac. Me especially. I’ve lost track of how many times I’ve said, “It’s a good thing I have sworn to use my powers only for Good.” Some of the things I can think up are terrifyingly plausible, which is how I ended up in security for a time.

Don't use mass murders, how to make explosives, etc., as training data. Use some human curation in what is used as training data, instead of trying to after-the-fact stop regurgitating what the LLM was taught.

That curation sounds like one of those oversight processes I was talking about, but offline instead of online.

> Things like this show that LLM are not even remotely intelligent.

Sorry for the bluntness, but no it does not show that. Or at least, you could use the same reasoning to claim that most humans are not "remotely intelligent".

What you're seeing is the rote regurgitation of someone who's been taught how to answer a question without ever learning how to think about the why.

This failure mode that's extremely common to see when tutoring (humans, that is). You'll have students who will quickly give a wrong guess that was clearly just chosen from a list of previous answers they've encountered before in class. When asked to explain why they chose that answer, they can only shrug. The main difference between them and e.g. GPT4 is that the latter is far more eloquent and better at stringing justifications together, which we associate with more reasoning capability so it throws off our evaluation of its overall intelligence.

Because LLMs are fundamentally a type of alien intelligence, different from the types we're used to.

> What you're seeing is the rote regurgitation

Exactly. And that's not intelligence.

Right, but my point is that while regurgitation is not intelligence, the act of doing so is not enough to claim the regurgitator itself is not intelligent. Otherwise, you'd condemn a good half of humanity with the same reasoning.

Or more, if we're willing to consider most peoples' reactions to at least some political topics - they just ignore the context and repeat the dogma they've learned (some more than others). People rarely stop and think for everything.

The problem here is that the LLM has learned that everything is political, and can be responded to the same way.

I personally think Glean's strategy was the right call - have customers pay you to train on internal knowledge bases and limit your answers only to that specific domain.

There is way more money to be made in this Enterprise SaaS sector and the risk conditions are lower.

Yeah, I agree that making domain specific LLMs might be far more useful. For example, one that is trained only on medical knowledge. One that is trained only on sci fi novels. One that is trained only on auto mechanics. One that is trained only on biology. And so on.

After all, if I ask a question about science, do I really want a result that was gleaned from scifi/fantasy novels?

If this proves LLMs are not intelligent, then when I ask ChatGPT or Gemini and they give me the correct answer, does that mean those are not LLMs?

It may very well mean it has seen that question answered before. It'll be hard to find a question that nobody has ever asked or discussed on the Internet, but if you can think of one, ask an LLM and see if it can figure it out correctly.

I asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly.

Today is February 29 2024. What will tomorrow be?

Since today, February 29, 2024, is a leap day, tomorrow will be March 1, 2024.

This is because leap years, which occur every four years (except for specific exceptions), add an extra day to February to better align the calendar with the Earth's revolution around the sun. As today is the added day, the following day becomes March 1st.

It is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples.

The same caveat as on my previous reply applies here.

That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs.

But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.

> if the statistical model is complex enough, is that not intelligence?

Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world.

https://news.ycombinator.com/item?id=39501090

It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.

> It's doubtful that any language can fully express the medium in which it is expressed.

I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader.

It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs.

If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.

When I say doubtful, I mean in a practical sense. If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts. You require an abstraction, and it's doubtful that a useful abstraction won't wash away details about the world.

I also think you're making an arbitrary distinction between human input data, and LLM input data. I could ask the exact same questions about a human's understanding of the real world. All I can say about both humans and LLMs is that they seem to both make good predictions.

> If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts...

We already have some answers here: the resources needed by current LLMs sets an upper bound on what is needed to do what they do, and we can also make a reasonable estimate of the resources used by the human brain to do what it does. In the case of LLMs, there is no mystery about how those resources are put to use, as they were designed and implemented by humans (we may be surprised by how much they can do with those resources being used the way they are - well, I certainly was - but that is not the same thing as the way they use them being a mystery, which is essentially where we are with respect to human brains.)

> I could ask the exact same questions about a human's understanding of the real world...

It is already well-established that by the age of two, infants are developing a rudimentary theory of mind - an understanding that other people have minds - and by around four, they begin to grasp that thoughts in the mind may not be true [1]. These abilities seem to require the recognition of an external world as a prerequisite.

Do LLMs give any indication of doing so? I'm not up-to-date on the research, but the last paper I saw on the topic only seemed to demonstrate that they can sometimes produce sentences as if they did - but, given the way their sentences are generated, it is difficult to say that this means anything more than that these sentences are the sort of sentences that a human is likely to say in the same situation.

Update: this point just occurred to me: LLMs receive tokens, not words. Words have real-world semantics, but, in general, tokens do not. To me, this increases my doubt as to whether LLMs could understand that language is about an external world.

[1] https://www.child-encyclopedia.com/social-cognition/accordin...

This really misrepresents what is happening in these models. Complex semantics are represented in the latent space. Relationships between embeddings have meanings we're able to sample to get these probabilistic strings - the fact that probabilities are involved does not mean there is no understanding behind the output.

> Complex semantics are represented in the latent space.

In this particular case, what's going on doesn't seem all that complex.

> The fact that probabilities are involved does not mean there is no understanding behind the output.

And that, in turn, does not mean that there is - which is the question of interest here. The burden of proof is on anyone claiming to know the answer, either way.

Ordinary, pre-LLM databases are not regarded by anyone as intelligent (AFAIK), but you can still get correct answers from them. Therefore, giving right answers is not a defining characteristic of intelligence.

Note that this reply does not endorse any particular position on the question of whether LLMs have any sort of intelligence.

When a calculator gives me the correct answer does that prove it's intelligent?

No, but when a human gives you an incorrect answer it does not mean all humans are unintelligent.

> People should stop anthropomorphizing LLMs. Stop being offended by a computer program. Stop complaining about its "bias". It's just a computer program.

“Bias” is tendency to deviate in a particular direction from the desired behavior. That something is a computer program does not make that any less of a problem.

> Just restrict it to saying "I'm sorry, Dave, I'm afraid I can't do that."

... and for the love of God, don't hook it up as a controller for any critical systems...

I agree with this point, that there needs to be a hard line drawn as to what these things are not allowed to do or touch.

Do not blacklist. Whitelist instead. By default they should touch nothing at all.

Have you met people?

I mean, if LLMs only said intelligent things then they wouldn't be like humans at all. Perhaps you also consider humans to not be remotely intelligent?

The console makes it pretty obvious that's a local model, BTW. Asking GPT-4 the exact same question I got:

> Barack Obama's last name is Obama.

Not the same thing, LLMs are stochastic parrots, and part of that is by design, so that they produce output that emulates speech.

But hallucinations are not avoidable either.

https://arxiv.org/abs/2401.11817

They are Internet simulators unless they can find some small, polynomial, matches to their existing patterns.

The fact that they will spit out confident incorrect answers mixed with automation bias on the humans part is challenging.

But while answering 'yes' in Propositional logic is cheap, answering 'no' is exponential time.

Once you hit first order logic you start to hit finite time and uncomputability.

LLMs work well with learnable problems and searching for approximate results in complex high dimensional spaces.

But we also don't know the conversation context in this case which may have lead to this response if it wasn't just a stochastic match.

IMHO the unreliable responses are a feature, because humans suffer from automation bias, and one of the best ways to combat it is for the user to know that the system makes mistakes.

If you are in a domain where it's answers are fairly reliable the results tend to be accepted despite the knowledge an individual has.

"I don't know" is also a learnable response.

Complicated subject but for known unknowns, future events etc... yes.

Others you can get negation through exhaustion, but not in the general case.

Reply on news.ycombinator.com