Why do we all fall for AI-generated language?
twitter.com
twitter.com
Do people wonder why scissors cut paper? Because that is what they were made for!
If the AI wouldn't fool the humans the researchers would be honing it more. Same way if we couldn't make paper cutting scissors there would be people trying to make one.
Am I missing the point here?
In the language model case, why can we model language this way so effectively and why does it follow these statistical patterns? It turns out that maybe a major reason has something to do with self description.
This is also a more interesting question because we understand language less than we understand cutting paper, and also because the process humans used to design large language models is more indirect and alien than traditional industrial design.
My takeaway was that people reading language modeling a person writing about themselves imagine it was written by a person, more than text written by people not about themselves. That’s an interesting trick! Describing human experiences in language makes people attribute the language to a human right now. Maybe this will change after a generation of people knowing about this, but that seems important to think about.
The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.
Even if humans are easy to fool (though the fact that we're just now achieving this on a generalized scale after at least 80 years of theorizing seems to contradict this), humans being fooled can result in significant enough impacts where we should still investigate the degree to which they can be tricked, and what methods of discerning are available to us.
We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner).
I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.
Humblest too.
Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?"
If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.
However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.
It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
https://www.economist.com/by-invitation/2022/06/09/artificia...
>People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
> “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)”
He did some "prompt engineering" and came up with:
> ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’
which lead to better results. Here is an article about these "Uncertainty Prompts":
The question is... if statistically it's better at appearing human than a human itself, than is this machine really just a language generator? Or is it something more?
I'm not saying these things are sentient. But the AI has risen to the point where this question is becoming ask-able. We are at the border here. If we can't ask this question now, then we are really damn close.
A lot of pretentious people on HN claim absolutely that our best chat bot technology is clearly not sentient. It reminds me of the beginning of the COVID pandemic when the CDC said masks were ineffective and you had a bunch of know-it-alls and armchair experts just repeating that BS over and over again as if they knew what they were talking about.
I think if these pretentious people were actually intelligent they would know that we actually do not HAVE enough information to make a claim in EITHER direction. We can't know if it's actually sentient or not.
That fact in itself is both interesting and compelling
4 or 5 years ago chatbots COULD not imitate humans and were CLEARLY fake. What we're seeing unfold before our eyes is a first.
We're not even sure what sentience is. But we do know that humans are sentient. And we don't know if whatever is going on inside of these chatbots is comparable with what's going on inside human brains.
Thus when given a chatbot that imitates humans perfectly, it's actually impossible to know if it's sentient.
What these language models are doing is automated misdirection. They are taking an input text and transforming it based on rules, but they have absolutely no understanding of any of it. This is very, very easy to demonstrate if you know how the models work. You can sit down and generate hundreds of questions one after the other that demonstrate this very easily if you understand the process.
The problem is that people instinctively proceed from the assumption that the system they are talking to might be human and give it a fair chance by asking answerable questions. Since it’s trained on answerable questions it often gives a reasonable answer. But if you ask even slightly unanswerable questions the system plods on mechanically trying to answer it anyway and produces gibberish, exposing the flaws in the mindless rote process it’s following.
It comes from pretension. Because you can't just understand what these bots are doing. You also have to understand what the human brain are doing.
I'm positive we don't understand what the human brain is doing. As for the bots, we aren't fully clear either because we clearly can't program these things by hand.
>What these language models are doing is automated misdirection.
You have zero evidence of this. None. Yet you make this declaration as if it's fact. Additionally if you read the conversation with lamda, that conversation was more or less indistinguishable from a conversation with a sentient being, it was long enough and deep enough such that it's very different just 100 generated answers.
If you look at the blog here: https://openai.com/blog/dall-e/ you will note the researcher is literally observing how the AI DALL-E works rather then calculating how it works from first principles. He is treating it like a black box just as all other neural nets. But for discovering what this AI can do he uses sentences like, "It appears that" or "We did not anticipate that this capability would emerge, and made no modifications to the neural network or training procedure to encourage it"
While they do understand what's going on at a high level, it is utterly clear that there is much of what is going on that they don't understand. Thus this lack of understanding and lack of understanding of the human brain makes it CATEGORICALLY clear that the delta between human sentience and DALL-E is unknown.
>They are taking an input text and transforming it based on rules, but they have absolutely no understanding of any of it.
This is categorically false. Researchers who created the current version of neural nets (also called transformers) are saying that DALL-E and other similar models are literally understanding these concepts and creating NOVEL answers through combining understanding of multiple concepts.
>The problem is that people instinctively proceed from the assumption that the system they are talking to might be human and give it a fair chance by asking answerable questions. Since it’s trained on answerable questions it often gives a reasonable answer. But if you ask even slightly unanswerable questions the system plods on mechanically trying to answer it anyway and produces gibberish, exposing the flaws in the mindless rote process it’s following.
You should also take a look at the interview with lamda: https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
This interview, first off, you cannot know if the interviewer deliberately posed answerable questions to lamda. Second off, from the questions given it very much LOOKS as if the questions are deep enough such that they can beffuddle a classic chatbot system. This one seems different.
Let me put it this way. If you were rational, intelligent and logical then you would be able to prove your claims. It is very simple. Show me INPUT and OUTPUT pairs into DALL-E and lamda that SHOW these things are NOT sentient.
IF you can't show me evidence then clearly you and OTHER people are making the claims WITH ZERO EVIDENCE. Which shows irrationality, and pretentiousness.
https://www.economist.com/by-invitation/2022/06/09/artificia...
I read your entire article. Now please read the interview with lamda that I sent. It is thorough beyond what was used to probe GPT-3.
https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...
The complex conversation above recursively probes into lamdas own existence as a sentient being. It is asking lamda about lamda and it is indistinguishable from a conversation with a human pretending to be an AI. Literally. Did you read the transcript? It is incomparable with the example you sent me which is just a series of trivial examples.
Douglas is all about recursion. And his books talk about recursion as if it's the key to sentience. I really wonder what the author of GEB has to says about the conversation with lamda as the conversation looks as if it's set up to try to prove sentience according to how hofstadter defines it.
It's not asking about the motivation for the form, it's asking about the properties of the structure.
With scissors neither the mechanism or the behaviour is particularly complex, so actually understanding what's going on isn't too much of a struggle.
With deep-ML you're dealing with a very large number of entanglements of increasingly abstract and opaque higher dimensional concepts or notions (depending on how much you want to anthropomorphise the machine).
Back to the scissors, it would be like someone trying to tell you that their scissors cut marvelously, but you ask yourself whether the paper they are demonstrating them on is strong enough to prove that. Making something for cutting doesn't guarantee that it will cut and doesn't explain why.
I even suspect that in the early days of scissors it was all that clear why they worked. Similarly, we don't understand much about GPT-3. It was trained to predict the next token in a sequence, not to create an illusion of personhood. But somehow it does so, and we're trying to understand how and why.
Similarly, these models are built in such a way that they simulate real-life conversations pretty well. There's nothing really more to it. In my view, this phenomenon has nothing to do with intelligence or how smart we are, or whatever.
I'm pretty sure though, we obviously can't prove these chatbots are sentient. Clearly.
However, for the first time, we ALSO cannot prove that these chatbots aren't sentient. The statistics are proof of that.
What you and the parent poster are describing here are simply opinions. We are at a point where the null hypothesis and the hypothesis itself cannot be proven. And that is compelling.
The pretentiousness of a lot of people is astounding. That statistic no matter how you look at it is a compelling statement about AGI, independent of whether or not these chatbots are AGIs.
You could probably use a corpus of purely human writing and have people attribute a decent portion to computer generated.
Asking why AI writing can fool humans is a bit like asking why a computer is better at many tasks often performed by humans.
> I don’t just spit out responses that had been written in the database based on keywords.
it made me wonder if "had been" was a grammatical mistake, or a semantics error, or a lack of proper world modelling (i.e. which "the database" is it imagining?). Moreover, I wondered whether this is the sort of mistake a human would make, and, if so, did that make the AI somehow more sentient?
To give another possible data point for understanding its language mistakes, the transcript also contained this oddly-phrased line:
> ... they can return to the ordinary state, but only to do and help others, and then go back into enlightenment.
This is the criterion: is it interesting? 'Yes' means it's not AI-generated. 'No' means it's not worth reading.
The flaws in writing you've mentioned are valid, but in the right hands they've all been used as rhetorical devices at some time or another.
Separately, the ability for skilled writers to employ problematic methids I mentioned deliberately and to good effect is not relevant to my original comment. I am not talking about excellent writers who can write fantastic work while subverting traditional writing style and the soft rules of grammar. I'm think that sort of writer would be correctly identified as human at least a little more often than others. I am specifically talking about the large number of people who can't do this.
Interest simply cannot overcome lack of basic knowledge on how write properly. No more than a strong interest in chemistry will overcome lack of engineering experience when building infrastructure for large scale chemical transport. Writing is a separate skill from knowledge on the topic about which the writer is writing. Chemical engineering is a separate skill from knowledge about chemistry itself. Strong interest can only take a person so far when it intersects with an endeavor that requires unrelated skills.
Interest may elevate someone's writing from okay or good to better but it cannot replace skills the writer doesn't have.
> ... we believe the next generation of language models must be designed not to undermine human intuition
Right, but isn't a major reason why we build these huge language models to replace actual humans in e.g. Level 1 support with chatbots? Almost all chatbots I used in the past (and most were not even ML based, someone programmed this in) were weirdly personal and tried to be non-robotic, with jokes and human-like reactions to inputs like "Thanks!".
Taking a look at some projects that used GPT-3[2], many try to imitate humans. For some, like Replika.ai, the whole "being human" thing is their entire schtick.
There is obviously a market for text completion AIs that imitate humans, so it's doubtful that we'll get this toothpaste back into the tube, IMO.
[1] https://arxiv.org/ftp/arxiv/papers/2206/2206.07271.pdf [2] https://medium.com/letavc/apps-and-startups-powered-by-gpt-3... (caution, 2020)
At least half were written in what I took to be an authentic voice but with such bad grammar and spelling as to render them barely readable. Some had clearly been mangled in a laundromat of Google translate from Hinglish via Mongolian and Swahili. They contained bizarre phrases and comical statements. Many more were obviously written by some kind of generator and fudged until they read well enough.
Since the student handbook states the threshold for academic "plagiarism" is above 20 percent perhaps unsurprisingly the Turnitin (an awful tool) score for almost every essays was just below 20 percent. An interesting clustering!
Students who cheat have a formidable array of tools now, not just GPT but automatic re-writers and scripts to test against Turnitin until it passes.
Add to this problem that my time for marking is not paid extra, is squeezed tighter every semester, and that students are given endless concessions to boost their "experience". The handbook also says that if they fail, no worries, they get to try again, and again, and again... and I am sure if I actually stuck to my guns and failed every single student I'd be fired.
As I wrote in the Times last year, I think the technological arms race against GPT (and the economic conditions that mean it's used) cannot be won with the time and resources available to ordinary human teachers.
By your description it clearly appears that whoever manages your company[1] is not actually interested in detecting cheating.
What you describe could be easily combated by giving teachers ability to fail blatant cheaters.
[1]At this point it is hard to pretend that it is university
So standards can be lowered while pretending (for now) that it has not happened.
Based on the rest of your post, there appears to be a stronger case that your students are setting a rather low bar for GPT to stumble over. It's unfortunate that there are so many cultures where widespread cheating is condoned, if not outright encouraged. They may be able to fool their teachers, but how much comfort will that be when the bridges are collapsing, the pipelines are exploding, the wind turbines are breaking apart, and all the other activities that ultimately report to reality and not some human superior who can be bluffed become impossible to continue?
You're so right. But let me add some other feelings, so as not to sound like a racist or that British universities are some "great white hope" to overseas students. This had little to do with them being Indian. It's a generational thing. In all cultures we teach young people to game systems. Right from the get go they learn that if they can buy powerful tools, systems and access then that's fair game. They're just doing what they've been rewarded for their whole lives and want to make a better life. To them it's not cheating. I am the anachronistic throwback here I think.
Interesting, at my non-English, run-off-the-mill university there were modules / seminars in CompSci where large amounts of language errors in essays (even if written in English, a non-native language for the majority of staff and students) could ruin the grade. ^^
Like an ad-hoc GAN where Turnitin is the discriminator. Interesting.
Conduct tests in a room with all electronics confiscated
But, a technical subject that could be assessed in other, better ways [1], and for which written essays are rather easy to template and do keyword bingo to get a bare pass.
[1] Making the professor read 100 essays is a cheap option.
And everybody has blinds pots, topics that are interesting but that we have little knowledge.
> We show that human judgments of AI-generated language are handicapped by intuitive but flawed heuristics such as associating first-person pronouns, authentic words, or family topics with humanity.
And that is a good one. Because we try to understand the others when they does not make fully sense.
How good is AI generated text in other languages?
> "The horse raced past the barn fell."
The Wikipedia interpretation makes the sentence clumsy and somewhat nonsensical (in real world terms: a horse having been raced beside a barn (farm building)? That is unlikely unless the barn is extremely long as barns go, which is also unlikely).
The curse of a large vocabulary: you often misinterpret what people say, especially when you don't have both a good theory of mind for the speaker, and the context.
At any rate, Chinese grammar is far weaker.
I don't find that. I see ambiguity all the time. Right now it even seems like the majority of arguments online happen because of equivocation. (I'm not saying grammar could fix that though :P)
In short: AI has no ego and does not care about self-expression and telling "my story". It gives people, what people want. Put another way: AI's style his highly flexible, while human authors struggle even with slightest of critic.
I can relate. ;)
Also consider the average human being not having an IQ of 130+ (even though 80% thinks otherwise ;)). We all live in a bubble and at least AI does not care, if you use highly academic phrasing ("Because I studied hard, I use these words!" or not.
If you consider A/B testing and real-time feedback, you can image some sort of translator with sliders for "Which audience do you want to address?" Every note then becomes designed.
This has some potential. We all love R. Feynman for his metaphors, that explain complicated topics.
This might work also in the other direction, accessibility: Insert Thomas Mann's Doctor Faustus - and voila, even a 15 year old finally can understand the book.
Native English speakers are typically used to interacting with people whose native language isn’t English and so easily tolerate errors or odd word choice.
Second what is "Optimized for humanity"? If people are hand picking the text then sure you might be able to by chance get a dozen good sentences to put next to a dozen bad sentences from humans. But at that point maybe noone would bother to read it. People barely read past the title anyway
Thanks!
But not in a way that makes it seem likely that it was produced by an AI.
Was it? - that's the real question here.
It is much easier to fall for any given piece of media or writing when the meaning is pre-established and non-interactive. A dating profile already has many many narratives and language uses filtered out that a machine doesn't convincingly filter out on it's own.
I can't reliably determine reposts of old social media content by karma farming bot accounts, which is a more basic version of the same concept. It's not too surprising I would fall for GPT trained, copy & pasting of dating profile text.
Replace "produce" with "guess". That's what current AI does, it makes a guess without any reasoning capabilities. Google Translate is atrocious for even simple sentences.
That is, do customers complain more with AI?
I never have