Comment by nexuist | Hacker News Reader

Comment by nexuist | Hacker News Reader

The problem is that OpenAI's motto is: "Discovering and enacting the path to safe artificial general intelligence."

This does not mix well with very common AI tasks, such as facial recognition, deepfakes and deepnudes. With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before.

How can we abuse this? Well, we can conduct text-based gaslighting and manipulation on an unprecedented scale. Imagine choosing a Twitter profile to target, having the AI read every single tweet that person has made, and then DMing them insults or slurs that would most effectively apply to their personality. Imagine those Nigerian prince emails but each with a scam personally tailored to its audience, with any follow up questions answered with auto-generated but believable lies. Imagine GPT-3 being used to message young children on Instagram by the millions and tricking them into giving up personal information or images.

I think, ultimately, there will be no such thing as "safe" AGI, because the intelligence we already have as humans can be and has been used to hurt others. OpenAI faces an existential crisis in this regards, and their best answer so far is simply to control access to the model so they can revoke it when bad actors are caught. This is something they can't do if the model is free and open source as we all want it to be.

I thought the point of OpenAI was to democratize AI access. By giving access to everyone, you are leveling the play field. Making it available both to small, big, good and bad actors.

Right now it's only accessible to some actors. And good and evil are relative by definition.

This is the exact point. Rarely has controlling information and science ever benefit humanity.

An alternative explanation is that, like Google (and every other company out there), the motto doesn't reflect their actual objectives.

Yeah, Open AI is as open as PRC is of the people. Or GDR was democratic. Words are cheap.

> With GPT-3 we are seeing levels of text comprehension and response that we've never seen in ML before.

Is GPT-3 doing text comprehension, or just generation? I mean, can it extract facts/automatically tag text etc? Or "just" (I know how hard that is!) generate text with some similar characteristics/topics?

Yes. If you few-shot prime it with prompts of the form "<paragraph> tag:<tags>" and then just feed it "<novel paragraph> tag:" it'll generate <tags> automatically, just from completion. It does this shockingly well, the paper has specific examples. It can also do fun tricks like condensing a complicated Wikipedia article down to a second grade level, including built in metaphors. An example of this is on the API beta website IIRC.

That doesn't demonstrate comprehension in any interesting sense. You don't necessarily have to comprehend a text in order to generate a plausible nonsense continuation of it.

Sure you do. If I hand you a few sentences in a language you can't comprehend at all, you won't be able to create a plausible continuation.

Never mind that - I wouldn't even be able to take on the role of the Emacs doctor in German (as I don't speak any German). It doesn't follow from that that the Emacs doctor does understand German.

What would your "interesting sense" of comprehension look like?

Something like the usual sense of the word.

The point is that it's not nonsense, but a useful and contextual continuation which encodes some level of comprehension (unless you mean a different thing by comprehension than I do). Look at this demo for an example: https://beta.openai.com/?demo=2.

Sometimes the continuation makes sense, but this is just by chance. It's just nonsense text in the style of the original. There's nothing that prevents internal contradictions. It's just that often these will be absent by chance, or because the output contains regurgitated chunks of coherent text written by humans.

OpenAI are, like most AI researchers, utterly shameless in encouraging overinterpretation and wishful thinking.

Reply on news.ycombinator.com