Hacker News Reader

I recently got into stories generated by GPT-3. What I notice is that it seems to be missing an understanding of state that causes constant inconsistencies.

For example, a popular Youtube video has a battle between Link and Kirby in it: "He finally releases the Hylian Shield and lets Link be engulfed in a massive fireball. When Link is reduced to a pile of ashes, Kirby is victorious. Kirby wins the fight to the death. Link stands there, dazed by the attack."

Most of that actually sounds pretty darn good and even sounds written by a human. It's to the point where there is a sensible structure to the story because the AI is getting the relationships between words. Massive fireball -> pile of ashes -> victory -> wins the fight to the death. That all looks good. The problem is that "Link is reduced to a pile of ashes" should put Link into a "dead" state, and when in the "dead" state Link can't stand and be dazed.

The problem of course is that the computer can't understand all of this. It can understand that there is a probabilistic link between "pile of ashes" and "fight to the death" so after writing the first it is much more likely to write the second. But it still doesn't understand what "death" actually means. I've thought for a while that neural nets alone aren't going to solve machine generated speech and that the real solution will be some sort of hybrid between a neural net and some sort of finite state automata. The finite state automata could then put a character into a "dead" state and know that when in a "dead" state they can't "stand" or "be dazed."

Source: (Video by DougDoug where he manually sets up battles between characters with a few paragraphs and then lets the AI generate the text of the battle. Sometimes it makes sense and other times someone's face will turn into a button or their eyes will shoot lasers) https://www.youtube.com/watch?v=PwY-jVSM-f0&t=2835s

Have you ever read stories written by young children? Kids learning to write have similar issues albeit with a much smaller vocabulary.

I was thinking the same thing, but you can explain to a child what the problem is. They can learn, in just a few minutes, how not to make that mistake again and improve their model of the world. It’s not as clear to me how you’d do that with GPT3, could you construct a text that includes this information and have it ingest it?

Yes, you can put it into the prompt. The prompt can contain the task name, a task description, examples, and example rationales. Instruct GPT-3 can get the meaning of the task very fast, usually with just the task name.

> The problem is that "Link is reduced to a pile of ashes" should put Link into a "dead" state, and when in the "dead" state Link can't stand and be dazed.

For a cartoonish video game, that's not too far fetched...

I've seen more ridiculous things in animes and such.

That said, I do get your point, and I agree.

> some sort of finite state automata.

Why not go one step further and have build in game engines that will work like imaginations of future states based from interactions within and between engines.

Check out PaLM and chain-of-thought prompting for a marked improvement on reasoning.

https://ai.googleblog.com/2022/04/pathways-language-model-pa...

GPT-3 anecdotally can't pick up on chain-of-thought quite as well. https://www.lesswrong.com/posts/EHbJ69JDs4suovpLw/testing-pa...

How is the model forced to do chain-of-thought?

> the computer can't understand all of this

It could...

> the real solution will be some sort of hybrid

An engine that built ontologies even just through ANNs could maybe suffice. It's still a game of entities and relations ("state" is still a relation, and relations can be implemented in ANNs).

Meaning that the network has to define "battler", "instance", "Link", "Kirby", "engulf", "fireball", "victory", "death", and progressively "know" what those things are and what they imply. It has to build a world including the laws and the entities.

I've played a lot of AI Dungeon and this is one of my main issues. You need to constantly correct the AI or retry the latest action (that there are big easily accessible buttons for such actions is itself telling).

One of the best sessions I had was a fairly epic story about a god of shadows whose unruly shadow monsters were attacking all humans. The god himself wanted them stopped and sought my (a powerful mage) help. We needed to make our way to his lair where a powerful ritual could destroy him and banish all his minions.

After an epic tale the plan succeeds, the god is destroyed, his shadows disperse. As the dust settles the god congratulates me but reminds me that I must make haste for the god of shadows has sent his monsters to attack all humans and he must be stopped...

This is more or less the concept of time, correct? I think an AI can understand state in simple cases (i.e. it can likely answer correctly "I didn't water my plant in 2 months, is it dead?"), but the way the current models are designed its just request / response, they perhaps from the very root way of how they are implemented don't (or can't) have a sort of _narrative_ sense of state. This is also present when you have a conversation with them. They won't bring up topics from the start or earlier part of the conversation, because they don't really "know" they happened. They simply receive and reply, that doesn't actually change the state of the model itself. To me this is one of the main keys that still need to be unlocked in AI capabilities. You need a neural net to modify itself in real time and track those changes to be able to have this sense.

To provide an example, it's almost like you need a _time series_ of GPT-3s, not just a single GPT-3 neural network, and the model itself would need to be able to self-inspect those time series and say to itself "ah yes, this was my old foolish understanding, now I have this new, better, understanding". I have no idea how this would look in technical terms, these are just the musings of a somewhat more-than-casual AI observer.

It's not really consistent outside of temporal issues.

As mentioned there are buttons for redoing or changing the AI response. This is because it often spouts nonsense or loses the plot.

In my example above the god who should have been dead talked about himself as a third person.

The AI also often mixes up people's gender or roles. One issue that used to be very common but has improved lately is that it used to mix up who said what in dialogues.

> The problem is that "Link is reduced to a pile of ashes" should put Link into a "dead" state, and when in the "dead" state Link can't stand and be dazed.

Yes, this kind of problem is real. But recent papers show you can ask the model to do reasoning / chain of thought / rationales before coming up to the answer. They can do complex tasks in a series of small steps instead of trying to do it in one step and failing. I believe it's not a fundamental limitation, just a matter of "blurting out something stupid" vs "taking your time to think before you speak".

> "taking your time to think before you speak"

One of the foremost core requirements for actual general intelligence.

Do you have any specific papers in mind?

I saw that mentioned in OPT175 release. How is the model forced to give reasoning?

It’s not forced to, they just give a couple examples with reasoning and then the model figures out that that’s what it’s supposed to do

Regarding this:

> ... the real solution will be some sort of hybrid between a neural net and some sort of finite state automata.

From https://datascience.stackexchange.com/a/25819:

> There is research showing that a DNN can simulate any FSM. Since there are more frameworks for DNN and DNN can perform more tasks than FSM, it appears to be more useful just to forgo FSM altogether.

> More specifically, "Neural network for synthesizing deterministic finite automata" shows how a relatively simple neural network (NN) can quickly and automatically learn the correct deterministic finite automaton (DFA).

There's this belief that if we come up with a problem hard enough for computers that humans can already do, where there's no obvious search or brute force strategy, we'll be forced to make something like a generalized human, and then have it solve the problem.

Of course, the issue is that the way we actually solve these problems is by coming up with a clever search or brute force strategy (chess, self-driving, poker), and yet we're no really closer to a generalized AI. Part of this is philosophical -- we can't define things like consciousness or intelligence in concrete terms, and so we have no real goals to work towards. We keep hoping that stumbling around in one of these other areas will lead us in the right direction, but so far, it just hasn't. The goalposts aren't moving, we simply haven't found where they are.

In my opinion, we won't ever have generalized AI. Or at least, we might not ever agree that we have it. If 3,000 years of philosophers bickering hasn't given us any true insights yet, it's probably not going to suddenly show up now. But for now, weeeeee!!! Enjoy the funding before the next AI winter kicks in! Look at all the cool new progress happening! But to paraphrase someone who's name I forgot, "If your goal is to drive from San Francisco to the Moon, making it to Boston sure looks like you've made a lot of progress."

> define things like... intelligence in concrete terms

Who told you we cannot.

You kind of did, by failing to do so.

I do not think it dutiful to repeat myself in each post.

(And honestly, while the base attitude of course remains to be "helpful", for current lack of a better term, sometimes - fortunately very occasionally - to provide a proper "complete" reply ceases to be economically viable.)

I think there are potential concrete definitions, but people disagree on which (if any) are correct

> but people disagree

Oh, but there is no need for such """correctness""".

The way is also well defined negatively - the lacks express their presence clearly -, and this page provides already a few examples:

-- "A is taller than B, B plays basketball, who is taller?"

If the system does not reply "A", then it does not understand the question; by a general consideration, it may be evident that it was designed to "guess" instead of "know", and there lies very patently the structural fault.

-- "A just left the house and is now in the market, where is A?"

If the system does not understand "states", etc. If it cannot build a world representation, etc.

-- "A fell from the flying helicopter, how is A?"

If the system does not understand "falling from an height", etc. If it cannot "tell itself a story" (Prof. Patrick Winston), etc.

The latter tells you about the logical relations between those "definitions" (of conjunction, so that (¬Av¬B)→¬C ). Prof. Winston defines "intelligence" as "the ability to tell oneself stories, which will also reveal outcomes of experiences you have not concretely lived". The definition may leave gaps when taken alone (especially if by somebody who confuses indications and definitions), but the lack of what is intended with such idea has full impact (no """inner storytelling""" - actually, I would say, no "productive engine" -, no "intelligence").

I agree with most of your comment and it’s a point well made. It’s frustrating to see all the “moving the goalposts” comments all the time when it’s just a problem of not knowing the right goalposts.

> In my opinion, we won't ever have generalized AI. Or at least, we might not ever agree that we have it.

But I disagree here. Eventually we’ll have something that can do everything we can do, just better, and at that point we might still be unable to draw a line, but we’ll at least be able to agree that a superset of human intelligence counts as AGI.

I think it should be pretty clear at this point that we can at least achieve "generalized AI" to the extent that we as humans have it. There is no reason whatsoever to think we can't build a machine at least as intelligent as we are.

I don't think there is any scientific consensus around this.

All of the arguments against it are completely non-serious. I don't think there are many real AI researchers that disagree with the thesis that it is in principle possible to build a machine as intelligent as a human. There is broad disagreement on how close we are to it, or whether or not the current track that ML research is on is sufficient to get us there.

> There is broad disagreement on how close we are to it, or whether or not the current track that ML research is on is sufficient to get us there.

So we don't know, it just sounds plausible.

Exceedingly, overwhelmingly plausible.

I agree that the arguments for it being impossible are, rather unconvincing.

However, I do think there is an important difference between "in principle" and "in practice".

I agree that there does not seem to be any fundamental obstacle that would necessarily prevent it. But at the same time, I'm not convinced it will ever be achieved.

Ya, I think that's potentially a reasonable position to take. However, if you wanted to be confident in that view I think you'd need to posit some particular reason it ought to be difficult.

I at least can't come up with any convincing specific reasons. That doesn't mean it doesn't exist of course, but given the amount of progress we've made on tasks we previously believed to be equivalent to general intelligence, it's difficult for me to see any clear reason why it shouldn't be possible sometime in the next century or two.

We have a rough idea of the quantity of computation the brain is doing, so either the brain is using computational methods beyond Turing completeness, or the only piece we're missing is the algorithm (since we're within reach of the raw FLOPs). If it's "just" an algorithm that we have to search for, it seems extremely difficult to predict when that may or may not come.

Huh? We’ve made demonstrable progress towards GAI, massive leaps in the last five years. If you don’t think GPT, DALL-E, muZero are evident of that then you are lost in cynicism.

We don’t need to define consciousness or intelligence concretely to have things that most people would agree are intelligent.

There were a lot of people that thought ELIZA was intelligent. There's some in-roads in finding specialized models that work very well, up until they don't, and when they fail, they fail hard, showing us just how different their processing can be from our own mental models.

My own personal definitions of intelligence cover some amount of self-reinforced learning. That is, if I tell an AI how to do something, it will think about how to do it, Google for the answer, watch and understand a YouTube tutorial, and try to follow along with it, and if it doesn't understand a part it has the ability to rewatch and try again.

I've never seen a jaguar in person; I've maybe seen 10 minutes total of videos and photos of one, and yet I can imagine one running through a field, chasing down a predator, and eating its corpse. Current AI need to be trained on gigabytes of data manually labeled "jaguar" to identify a single one. Why can't an AI, after seeing something it doesn't recognize, ask a question like "what is that"?

Just a few years ago, there was that Minecraft AI challenge that failed spectacularly -- the task was to watch 50 hours of video of someone else playing the game, and then do it themselves. I could probably do that after ten minutes.

In twenty years, we're going to look at classifiers in the same way we look at pre-Google search engines. They are buggy as hell because the classification problem is incredibly ill posed.

DALL-E2 is a fantastic example of zero-shot learning working spectacularly well. It's able to combine various concepts interchangeably.

Modern models can do few shot learning, and MuZero proved the ability to learn a game quickly using self-play.

They aren't human level yet in all tasks but we are getting demonstrably closer. I would spend more time studying what exists today and less time critiquing

> it’s AI, we're not there yet.

At this point, we still barely even understand what human intelligence is :)

But one thing is for sure: we can definitely notice a _lack_ of intelligence, which is why I think the goal posts keep moving as AI improves (maybe in a way similar to the uncanny valley, that the closer you get to the real thing, the farther away you seem, I.e. all the things that make it NOT human get amplified on observation).

> At this point, we still barely even understand

If one believed that e.g. Reasoning can be spawned as an emergence, then Reasoning should be a specific attempted result.

I think they are moving because people are scared

Or simply, because specific problems get solved and "ticked on the clipboard".

Moving goalposts is fine, and I speak as the opposite of an AI skeptic, thinking as I do that it's 70% likely general AI is invented in the next 15 years. You move the goalposts when you discover that the goal you set didn't match what you were trying to achieve.

I think that it's now possible to give goals that you know you probably won't have to change, but those goals, by their nature, are worse metrics for current research. One example would be that the AI should learn in response to human instruction by text, video and audio, as well and as fast as a human being does with the same access to that instructor, across a variety of tasks. This would be something like "online multi-modal few-shot learning" in the current jargon, and you can see that most research isn't really working on it – probably because it's still really hard to get good results. But there's not much hope of solving the harder problems without solving the easier sub-problems first.

Per your last paragraph, moving goalposts aren't frustrating to real researchers and practicioners -- they are the only valid outcome of successful progress!

The obsession with reaching an AGI finish line is limited to charlatans and critics, who are off to the side enabling each other.

> moving the goalposts

Those goals are "research" in terms of "playing with the available tools to refine them".

On the strict technical sense, "Intelligence" in AI remains "to be able to provide some solution that could compete with that of a human solver".

On the large proper sense, "Intelligence" means "understanding".

"Understanding" is just as difficult to define as "intelligence". Any outward sign of understanding can and will be shown by a sufficiently complex artificial system sooner or later.

The human mind is not qualitatively different than sufficiently complex software that imitates it perfectly. Unless, that is, we return to the idea of body/mind dualism which is 200 years out of date.

People keep being disappointed by these results because magic tricks just aren't as impressive if you know how they work.

The human mind is likely infinitely more complex than software that imitates any aspect of it. Just like human thought process is orders of magnitude more complex than that of a parrot.

You would be surprised at the complexities of parrots, though...

The Bekenstein bound seems to suggest that the human brain is only finitely more complex.

(And, if the mind is more complex than the physical/material brain, then, unless whatever else the mind is has a non-material influence on the material brain, any additional complexity of the mind would not need to be modeled in order to imitate the outputs.)

“The human mind is not qualitatively different than sufficiently complex software”.

Just no. Nothing AI researchers are building has subjective experience.

I could be wrong but I think they might have been talking about a hypothetical, arbitrarily complex software. As a limiting case, if software were simulating a mind down to the quarks, it becomes unclear what the difference would be.

I agree with your point of course, what we have today is certainly not like a human mind

The problem with the hypothetical arbitrarily complex software is that there is no particular reason to believe it could exist, never mind that it will (at least not for meaningful definitions of "software"). A computer so powerful and a programmer so smart that they can represent the behaviour of the constituent parts of a human brain at the subatomic level as a state machine programmable to achieve different thought processes is at least as much of an imaginary construct as the metaphysical dualist mind it's supposed to be a counter argument to.

And you don't need to think that your brain is anything other than an immensely complex state machine to think that some of the core parts of what we consider to be self-awareness (emotions... or chemical responses to certain stimuli which have over billions of years helped the brain parts of biological organisms make more optimal eating and fighting and fucking decisions for the survival of the gene code) are an altogether different level of problem to train an AI on than solving maths problems or generating text. Not least because if you want AIs to write love letters to each other, you can get very pleasing results quickly with a Chinese room without the inconvenience of having to simulate all the intractable chemistry of desire.

> "Understanding" is just as difficult to define as "intelligence"

But in this field the latter is the generic term and the former an implementation detail. For clarity: there are disciplines about it.

> Any outward sign of understanding can and will be shown by a sufficiently complex artificial system sooner or later

And? That remains a mockery of understanding, instead of the thing itself.

> People keep being disappointed by these results because magic tricks just aren't as impressive if you know how they work

No. The issue here is that illusionism ("«magic tricks»") is not magic, and it sometimes pretends to be it on bewildering stances.

Such as, if you want the engine to compose like Beethoven, make it understand and thus reconstruct the implicit references in rhythm melody and structure, or without that, you may implement further tests while keeping it a mockery machine, but you start looking like putting makeup on a puppet.

It's useful to show to people who don't understand AI. There's a mental effect similar to "the computer is always right" where people will see AI doing something and assume it's some engineered piece of software running an algorithm. This kind of thing can help remind them that all AI can really do is generate convincing looking noise.

That is demonstrably false based on the tasks AI is proficient in today

It gets a lot right but not 100%. Like I said, it's convincing noise.

Humans don't get 100% right either, is that also convincing noise?

> "If your system can have a conversation with a human who believes they're talking to another human afterwards, it's AI, we're not there yet."

To be fair, that was pretty much the first goalpost proposed; it wasn't moved back so much as other people put goalposts up closer than it.

>"If your system can have a conversation with a human who believes they're talking to another human afterwards, it's AI, we're not there yet."

That happened well over 50 years ago! There's something fundamentally wrong with the goalpost, and abandoning it was far from arbitrary.

> "If your system can have a conversation with a human who believes they're talking to another human afterwards, it's AI, we're not there yet."

==============

[16:31:08] Judge: don't you thing the imitation game was more interesting before Turing got to it?

[16:32:03] Entity: I don't know. That was a long time ago.

[16:33:32] Judge: so you need to guess if I am male or female

[16:34:21] Entity: you have to be male or female

[16:34:34] Judge: or computer

==============

The Turing test you're referencing [1] (transcripts included) was a charade. It seems all participants and organizers were determined to create a scenario where the Turing test could be passed, regardless of whether it actually could or not. Turing's 'test' was never precisely described but he essentially said that after 5 minutes an "interrogator" would not be able to effectively determine whether an AI he was interrogating was a man or a machine.

I'll just list various details on the event, in no particular order:

- Turing specified "interrogators" who would be actively seeking to expose the AI as an AI. The test in question used judges who made no effort whatsoever to challenge the AI frequently asking questions like "How old are you?"

- The judges were not judging an AI but instead having a simultaneous interaction with two entities, and had to pick which was human. Some of the humans seemed to actively make an effort to appear non-human, which paired with judges making no effort to challenge the AI increased the chances of a random result.

  - The "AI" that won 'replicated' a 13 year old non-native speaking boy, probably as an excuse for its frequent incoherent responses. Other of its responses were effectively refusing to cooperate or going on complete nonsequiturs which could again be excused for being a 13 year old boy.

 - The interactions were limited to 5 minutes with two entities and seemingly slow typing from both the judge and the participants. Some interactions were limited to as few as 2 responses from which to make a decision. 

 - In the paper itself, the researchers were quite confused why some of the participants and judges behaved the way they did. Obviously they just wanted to be part of something "historic". 

 - The dialogue I quoted at the top was from a human, obviously trying to trick the judge into misclassifying him as a computer. And it worked, that was one of the 3 misclassifications required to hit their 30% benchmark for passing.

[1] - https://www.tandfonline.com/doi/full/10.1080/0952813X.2015.1...