Hacker News Reader

Comment by layer8 | Hacker News Reader

It’s a double-edged sword. From the article:

GPT-3 doesn’t actually know anything about anything at all. It’s a huge pattern generator. You can’t trust anything it says, because all it does is group words together into convincing looking shapes based on text that it’s seen before.

Once again, I’m reminded that tools like GPT-3 should be classified in the “bicycles for the mind” category. You still have to know how to pedal!

They’re fantastic tools for thinking, but to actually use their output effectively requires VERY deep knowledge—both of the subject matter in question, and of the way that the AI tools themselves work.

View on HN

The types of error made by the model in the article are exactly the sort of error that I (senior engineer / decade of experience) would make if asked to describe how something works and I didn't want to spend too much time on the details. The difference is, I can usually give a decent estimate of how sure I am of something, and know how to zero in on the truth when it's important.

Maybe a confidence level for a given explanation, along with some sort of "here's where you can go to learn more" would be useful? No idea if language models would be good at that kind of meta reasoning.

There’s already a per-token confidence score given by GPT-3 actually. As for citing actual sources, retrieval architectures like RETRO are becoming popular for this as they can tell you which sources they used for the current prediction and are actually faster thanks to including a fast KNN lookup over the dataset.

Dunning-Kreuger notwithstanding? ;)

And, in particular, GPT-3 will be very easily misled by plausible-but-incorrect inputs -- like programs written by amateur developers. I definitely wouldn't trust it to find subtle logical errors in a student's computer program, for example.

What is thinking if not pattern generating?

It's not enough to state that the human mind is a pattern matcher as pretty much everyone of the "scale is all you need" crowd does. You have to prove it too.

Otherwise it's just a variant of Russell's teapot.

Even if you prove it, it's not useful.

You aren't actually serious, are you?

We use far more variety of inputs than GPT-3.

GPT-3 has more textual training data than any human could read in a thousand lifetimes, but we base our judgment on much more than that.

I agree that in a sense it’s probably only a matter of degree. However, take for example its explanation of the xm modifiers, where it only explains the x, but gives the impression that it explains both modifiers. It doesn’t seem to have any awareness that it provides an incomplete and potentially misleading explanation. It would be interesting to see how it reacts to “What you explained is only the x, what about the m?”.

It's missing the part where you're not sure what the answer is, so you look it up, do an experiment, or ask someone.

That's a bit much to ask of a fiction generator, though. It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not.

> It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not.

Sounds very human to me.

Humans are sometimes curious enough to look for answers.

This model has already spent more time learning than most humans ever would...

Imagine if we gave it more power and funding. Maybe redirect all of Congress's salary to it?

Look into RETRO, which greatly reduces the model’s tendency to confabulate by teaching it to query a document database known to be truthful, and justify its answers with specific references: https://www.deepmind.com/publications/improving-language-mod...

That looks like "if confidence < %some number%, then try to expand the dataset using known related sources". Load a library of short book descriptions into such an AI, and it should reproduce this behavior. Though it also needs the capacity to learn. Otherwise you could put a snippet that automatically queries it for better materiel about the subject when its confidence is low, so it just tells you where it thinks you should look. That seems doable now.

I'm no machine learning expert, but agree that it seems doable. I think rather than an if statement (which seems like a bit of a hack), it would be more principled to somehow train it on when it's worthwhile to do a search, sort of like a multi-armed bandit problem.

Either way, it's not just scaling up, it's changing the algorithm.

Things we have that current AI models do not:

Intent: We can have a goal which directs our thinking. This changes what we come up with much more flexibly than experience, even if experience is often needed for good results.

Temperment: We can be angry, tired, excited, relaxed, etc. while thinking. This isn't always good, but it is a way we differ.

Self-awareness: A very difficult term to define, but we can (hopefully) take a step back and discard our current thought if we realize we are falling into one of our usual bad patterns of thinking.

Intent and self-awareness do come from experience.

The goals we want to achieve are tied to what has been going on throughout our lives thus far. We don't invent our own values out of a vacuum, we tend to adopt them from other people, and rank them depending on how our nervous system works.

Self-awareness is the ability to reason about the process of reasoning, plus, in your example, a recollection of past examples of incorrect reasoning that disrupted the process of achieving our goals.

So basically current models need a memory and the ability to change their own weights in response to new data to even start approximating humans, but that looks like a doable task, in principle.

Indeed the brain is pretty good at this, so much so that it is annoying when you are trying to generate a variety of textures by combining a small set of textures. This of course extends to audio patterns, temporal event patterns (e.g. post hoc, ergo propter hoc) etc.

That may be the cornerstone of "intelligence", but that's only one part of it. I think another one is making or finding relations between the parts of the whole.

Life is the continuous adjustment of internal relations to external relations [1]

Once you have internalized relations of the real world, you can start to hack them in your head, run simulations of the results and take action.

[1] https://www.thoughtco.com/famous-education-quotations-herber...

My take is that pattern-matching is thinking. But it's low-quality thinking. High-quality thinking is logic. And higher still is causality, which is to logic what calculus is to algebra. I.e. if logic studies the relationship between x and y, then causality studies the relationship between dx and dy. And causality is what we actually want, because causality is power. E.g. causality is what lands astronauts on the moon. When folks like Judea Pearl complain that current AI isn't truly thinking, they're complaining that current AI can't reason logically/causally.

Reply on news.ycombinator.com