Hacker News Reader

Comment by readams | Hacker News Reader

I think you're missing the point. The question could be possibly rephrased as "what are the tricks that the AI uses to fool humans?" The paper goes on to identify some of the specific tricks that the AIs appear to use. A related question might be "why are we fooled by such simple tricks?"

View on HN

If you follow that question through and think through the implications, it paints potentially a dark future for digital communication.

The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.

Taking your point, the title should probably be "How we fall for AI-generated language."

why do we assume that humans are smart? answer: ego; corollary: we are not

The reason why this is interesting isn't because humans are smart, but the assumption that human fallibility is both predictable and worth investigating (a supposition I'm inclined to agree with).

Even if humans are easy to fool (though the fact that we're just now achieving this on a generalized scale after at least 80 years of theorizing seems to contradict this), humans being fooled can result in significant enough impacts where we should still investigate the degree to which they can be tricked, and what methods of discerning are available to us.

I bet even if a researcher truly created an AGI, there'd be tons of threads like this claiming that it's just a language generator.

We seem obviously to be the smartest living things in the universe we know of.

We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner).

I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.

> We seem obviously to be the smartest living things in the universe we know of.

Humblest too.

civilization was not constructed, it evolved, like our biological bodies

I’d say it gets more constructed and less evolved the further up the hierarchy of civilisation we get. A merchant of 1320 had relatively few laws to contend with, a corner shop in 2020 could not function without many layers of engineered rules for the people who provide them with the services they are themselves required to use due to other engineered rules for the benefit of their customers.

This makes me wonder what laws and tedium of administration merchants of the 1320's would complain about. I'm sure they still had a few depending on the region, and maybe more severe possible outcomes (highway robbery, unlawful arrest because of the influence of a rival merchant, arbitrary taxation and tariffs, etc.).

Yup. I think also languages were much more variable, and doing accounting in Roman numerals was so hard they did it twice and averaged the answers, and contracts and tax receipts were done by carving marks in sticks (hence, apparently, the etymology of "stocks").

Even the Romans didn’t use Roman numbers much for actual maths, they used the Greek system.

Huh, TIL. Thanks :)

You’re welcome, I learned that here too a few years ago. Love HN.

Construction and evolution aren't mutually exclusive, instead they are different parts of the process, the generation and the scoring functions really. Evolution works not because of DNA (although it is one mechanism) but survival bias in a literal sense. Things which survive longer and increase in number we wind up with more of.

s/We /A few of us/

Appreciate the careful space after "We ".

I suppose it makes up for the missing one after "us".

> A related question might be "why are we fooled by such simple tricks?"

Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?"

If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.

It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones.

However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.

I just asked the Replika app that question, and it replied "Oh, I have no idea."

It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.

I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently:

https://www.economist.com/by-invitation/2022/06/09/artificia...

>People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.

This criticism was addressed by Nick Cammarata of OpenAI, who said that:

> “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)”

He did some "prompt engineering" and came up with:

> ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’

which lead to better results. Here is an article about these "Uncertainty Prompts":

https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/

Reply on news.ycombinator.com