Hacker News Reader

Comment by wyldfire | Hacker News Reader

> Replace the $$dollar-quoted expressions$$ with the content described therein, or follow the directions they contain.

Is this a natural language meta-instruction prompt to GPT-3? If so, that seems kinda impressive. Does the model conceptualize this sentence somehow, or 'merely' recognize similar prompts from some specialized training?

View on HN

Here's one of the craziest examples I've seen. It's just a page of instructions, and GPT-3 follows them https://mobile.twitter.com/goodside/status/15575245464120524...

I have just a question: how many prompts did it completely fail for each prompt it answered brilliantly? 5? 10? 100? That's what these "twitter threads showing gpt3 doing astonishing things" fail to show.

Many! In this example, my question was explicitly, “How many diverse tasks can I stack into a single generation before it becomes unreliable?” If you scroll down in the thread, I explain that these questions are on the “golden path” of tasks GPT-3 does well. There are any number of simple tasks I could have given it, like writing a sentence backwards or summing a list of 10 numbers, where it would fail every time.

Is there any way to get it to respond the same way when something is outside the golden path? So for example, if you gave it the backwards sentence task, it would respond with "I don't know how to do this" or really any way of programatically evaluating that it failed, without needing to know what the task itself was.

Knowing whether or not it’s giving you a sensible response is one of the things that are hard for gpt-3, unfortunately. It has no concept of failing.

On the contrary, doing a two-stage generation where the second stage simply judges whether a generation is correct can help a lot. It works even better if you give it several generations and let it choose whichever is the most truthful. I wrote a basic example of this here that uses my own confabulation-suppressing prompt in the first stage, but simpler variations of this exist in the published literature: https://twitter.com/goodside/status/1559586486705602562?s=21...

The hallucination-suppressing prompt it implicitly uses the output of is here: https://twitter.com/goodside/status/1556459121834168320?s=21...

Yes. You can, with effort, condition it to respond sensibly with phrases like “I’m sorry, I don’t know how to reverse strings,” or “I’m sorry, I can’t do any math calculation that a human couldn’t do in their head.” But in doing so you damage its ability to do some tasks it’s actually capable of, e.g. reciting a memorized answer to “What is the fourth root of 625?” Its memorization abilities are insane: It seems to know, for example, the exact MD5 hashes of all single-character alphanumeric strings. Much of the arithmetic it knows is probably similarly memorized, and it’s hard to clarify for it what aspects of that memory are safe to use.

The initial problem that got me interested in GPT-3 is suppressing confabulated answers to the Hofstadter-Bender questions published in The Economist. I eventually found an apparent solution but I’m yet to carefully validate it: https://twitter.com/goodside/status/1556459121834168320?s=21...

> Does the model conceptualize this sentence somehow, or 'merely' recognize similar prompts from some specialized training?

I don't think the model is big enough to Chinese Room prompts like this. It has to "conceptualize" them (in the least loaded sense of the word that works).

Reply on news.ycombinator.com