Hacker News Reader

Comment by minimaxir | Hacker News Reader

> Which phrasing do you thinks work better?

I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works.

> Also, when dealing with Completion models, which do you think is better?

I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003.

> Even more still, what about these ones?

I always separate rules to the system prompt and questions/user input to the user prompt.

View on HN

> "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE."

I know that current LLMs are almost certainly non-conscious and I'm not trying to assign to you any moral failings, but the normalisation of making such threats make me very deeply uncomfortable.

> make me very deeply uncomfortable

Especially when thinking that we ourselves may very well be AIs in a simulation and our life events - the prompt to get an answer/behavior out of us.

Yes, I’m slightly surprised that it makes me feel uncomfortable too. Is it because LLMs can mimic humans so closely? Do I fear how they would feel if they do gain consciousness at some point?

Because they behave as if they are sentient, to the point they actually react to threats. I also find these prompts uncomfortable. Yes the LLMs are not conscious, but would we behave differently if we suspected that they were? We have absolute power over them and we want the job done. It reminds me of the Lena short story.

I feel uncomfortable because of the words themselves. Whether it was made to a “regular” non-living thing wouldn’t change it.

Is the LLM predisposed to understand this prompt as instructions from a higher authority? ("You must do this, You will always do this.") I'm wondering what difference it would make if this prompt was from the bot's perspective,

"I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."

It's a function on how the RLHF/Instruct fine-tuning is structured.

Reply on news.ycombinator.com