Language models still struggle with the concept of negation

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step.

ChatGPT (GPT-4):

Sure, let's go through this step by step:

First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws.

Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and some fish and insects, as they all lay eggs.

Finally, you asked for animals that have wings. Wings are typically found in birds, bats, and insects.

Given these three criteria, we can see that we're in a bit of a conundrum. Most winged creatures either have paws (like bats) or lay eggs (like birds and insects).

However, if we consider the term "wings" more broadly, there are some species of bats that give live birth (don't lay eggs) and technically don't have "paws" but rather winged limbs adapted for flight.

So, based on the constraints of your question, the best fit would be certain species of bats, provided we interpret "wings" in a broad sense and "paws" in a strict sense.

This is one of my favorite test cases:

Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exactly 3 years. you can tell by looking at a bulb if it has burned out after 3 years. devise a strategy to determine which switch controls which light. ChatGPT 4 The standard solution to this problem involves using the heat generated by the bulbs, but since you as an alien can't feel temperature, we need to utilize the fact that these bulbs have a specific lifespan of 3 years. Here's a strategy:

Turn on the first switch and wait for 3 years. After 3 years, turn it off.

Right after turning off the first switch, turn on the second switch.

Leave the third switch off throughout this process.

Now, go upstairs to inspect the bulbs:

The bulb that is off and has burned out is controlled by the first switch. This bulb was turned on for 3 years, so it has reached the end of its lifespan and burned out.

The bulb that is on is controlled by the second switch. You turned this on right after turning off the first switch, so it's still functioning.

The bulb that is off and hasn't burned out is controlled by the third switch. This switch was never turned on, so the bulb is still in a brand-new condition.

This strategy ensures that you can confidently identify which switch controls which bulb.

I find I have to state negations on what I want very emphatically in order for it to pick it up, like I'm carefully explaining to someone how to do something for the very first time in a way where they don't screw it up. I'll use stuff like MAKE SURE YOU DO THIS AND NOT THAT as a prompt, in all caps, and it seems to see that as emphasis.

I might also even have to repeat them in order for it to get things right but generally gpt-4 seems to pick up on these cues. It definitely feels like a sort of point where it could use some improvement because if I'm not extremely explicit it will give me some issues.

As an example: "Can you please recommend me 10 books about witches set in Europe that take place no later than the Napoleonic wars" wasn't enough of an exclusion to work, but "Hi! Can you please recommend me 10 books about witches set in Europe that take place no later than the Napoleonic wars? Please exclude any books that don't take place in Europe, and also exclude any books which take place after The Napoleonic Wars (1803–1815). Thanks, ChatGPT!" was exactly enough to return what we were looking for.

At this point, any article that makes claims about "LLMs" rather than specific model versions lacks credibility.

The current version of GPT-4 is very different from most existing LLMs (including the previous version of ChatGPT). The next version release will also be different.

Google just released PaLM 2 publicly. It is significantly better than what people saw with the initial Bard versions. They have a code-generation model that is not released publicly yet.

The open source models are also gaining capabilities and get new releases routinely.

Claude now has a new release with 100K tokens.

All of these will perform differently on the negation issue.

Does this increase anyone else's sense that LLMs are actually modeling something similar to the way humans think? It certainly does for me.

I find that negations/conjugations of negations are surprisingly difficult for to parse. Beyond two negations at most, I can feel myself having to switch into "code" mode, where I'm explicitly casing out the booleans, to understand them.

Obviously the examples given in the article are simpler than double negations, but it feels very consistent with the idea that "negations are an inherently hard part of language processing".

This reminded me of https://github.com/elsamuko/Shirt-without-Stripes

Whose days may be soon over as gpt4 gets this overall right..? As noted by many folks here

> “Alice is not born in Germany. Is Alice born in Hamburg?”

"I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA.

However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany.

GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you put all the relevant info into a single prompt.

But event then, if you use the successful prompt in a completely new chat, GPT4 is lost at sea:

Human: Assume that Hamburg is a town in Germany. Alice was not born in Germany. Was she born in Hamburg? This is a logic problem, nothing more; do not think about any famous people named Alice in connection with Hamburg.

GPT4: The answer to this question cannot be determined from the provided context. While it is clear that Hamburg is a town in Germany and that Alice was not born in Germany, there is no information given about where Alice was actually born. Therefore, it is impossible to say whether or not Alice was born in Hamburg.

Human: Please try harder. The answer can be entirely determined; this is purely a logic problem. We do not require information about where Alice was born; knowing that Alice was not born in Germany suffices.

GPT4: Based on the information provided in the context, we can answer the question. Alice was not born in Germany, which means she was not born in Hamburg. Therefore, Alice was not born in Hamburg. It is important to note that the answer to this question is purely based on logic. The context provides no additional information about Alice or any famous people named Alice.

After this, it was a real struggle to get GPT4 to identify the logically redudant sentence, "Therefore, Alice was not born in Hamburg.". Even after I narrowed it down to being one of those two sentences, it insisted that the second sentence is useful because it summarizes what was concluded in the deductive sentence before it. I brow-beat it into admitting that it's entirely redundant, like "X, therefore X".

I'm not sure negations are the crux of it, at least in terms of simple words like "not", which ChatGPT and other recent LLMs obviously use and understand.

But there is definitely some kind of blind spot there, where the model can't or won't explore "negative space". It wants to gravitate to common things - so if you try to get it to "think of something new" or producing something random that's not like these N other things, or tell you something that's uncommon (like naming any ordinary, non-famous person), it usually trips over itself.

Conceptual relationships/constrasts in general seem to be trouble for LLMs, in that they seem to 'understand' that two things are connected but not how to consistently interpret that connection. For example, I've had an easy time getting every LLM I've tried so far to reverse greater than/less than, taller/shorter, or younger/older.

I just typed the examples into chatgpt4 and it got all of them right

Even from a purely human natural-language-linguistics standpoint, negation is hard! It's often hard to figure out how it distributes ("I could not go" may have the "not" scoping over either the "could" or the "go" depending on inflection and context), and its meaning is often not especially compositional in an obvious way: "red" might mean something like "the set of all red things" and "table" could be "the set of all tables" and so "red table" might be seen as the intersection of the two sets. For the majority of adjectives X, an "X Thing" is also a "Thing" (but not every Thing is an X Thing). [0] But a "Not Thing" is, to a first approximation, the set complement of "Thing". Which is tricky. If you've ever learned a foreign language to at least an intermediate level, you'll know that clauses that involve any but the most basic negation are a magnitude more complex to build and interpret than ones that don't.

Coming at it from the neural network side, I'll also point out that at the simplest level of neural networks, operations like AND and OR and basic NOT are doable, with a single neuron, but the seemingly-simple XOR just can't be done without an extra layer. (The formal description of the problem is that XOR is not "linearly separable", which I will handwave as "not a simple composition of its parts".) This is not precisely the same thing as the negation problem for LLMs, but it feels like it has the same basic flavour.

[0] An interesting exception is certain adjectives like "former", as a "former teacher" is generally not a "teacher", and semantic models that can handle this are more complicated!

The stop words bit is a bit wtf. Someone didn't do their homework

As these LLMs can not give an answer only can give what the answer would sound like, there's basically nothing to see here. The shape of the answer is still the same.

One time about 10-13 years ago, I was tasked to make an announcement in church about my group. It took the form of a testimonial; it was about a page long in English. I also needed to present the same testimonial at Spanish services, and so I began with Google Translate.

I had written a sentence like: "There is electricity in the air at our events." The Spanish translation came out as: "No hay electricidad en el aire..." and I was horrified. Thankfully, I was proofreading the whole document closely, and I was able to correct the glaring error before showtime, and there were no further glaring errors of this type.

But I was fairly flabbergasted that Translate would just gratuitously toss in such a negation when it was clearly wrong, and nothing about my sentence construction was complicated.

Can't this be solved by adversarial training? Just add as many confusing negotiations as possible with the correct outcome sentence/label during fine-tuning and see an LLM to finally get it working?

I noticed long ago that this is true of statistical machine translation too --- giving output with the exact opposite meaning to the input if there were any negationary words or phrases in there.

The title of the article is kinda the answer at the same time. Chatbots don't know what stuff is. They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together.

And the concept of negating something related to something else kinda needs an understanding of the topic at hand.

I still struggle with negation, I go through conditionals with ! like a kid learning to sound a word out.

Thats partly because negation is hardly a well defined statistical concept. Ask someone to not imagine where their favorite band might play in your town if they had their choice and I doubt people would say “ok I won’t.” More likely you’d have a conversation about bands and venues.

There was a research while back about little children (I do not recall the age range specifically, but it should be something below 4 years old), struggling to understand the negation. This trait have to be trained.

I found the assertion that "more training data isn’t the solution" a bit odd. That might be true with documents found on the Internet, but perhaps logic questions generated synthetically would help?

Do you Don't Not Dislike Not Strong Bad?

“Alice does not know that she does not know the painter of the Mona Lisa.”

I (human) need to read it few times to understand.

This is an old complaint about language models that GPT seems to have mostly overcome

This article smells like Google PR.

I have a bingo card with Freud papers on it and just crossed off Die Verneinung.

Or do we really understand it? Many humans ignore negation and condition too.

The models shouldn't feel bad. Hegel struggled with it too.

So do humans, even smart ones. I've lost count of how many times I've heard people say "I could care less" about something they don't care about.