Oh. And what are these universal "human values?"
> our study contributes to the nascent field of machine psychology.
It's a little hard to accept that you're doing "prompt engineering" and "machine psychology" at the same time. This paper has a stratospheric view of the field that isn't warranted at this time.
That is the core problem of AI X-risk, and has been studied in this context for at least 2-3 decades. If we knew the answer, we would know how to make a perfectly aligned AGI.
Actually no, we wouldn't. The problem, at the moment, is even more basic than "what values should we align an AGI with". Currently, the problem is "how do we robustly align an AI with any set of values."
We currently do not know how to do this. You could hand OpenAI a universal set of safe values inscribed on stone tablets from god, and they wouldn't know what to do with them.
To state it another way, people like to talk about paperclip maximizers. But here's the thing: if we wanted, to we couldn't purposefully make such a maximizer.
Right now, AI values are emergent. We can sort-of-kind-of steer them in some general directions, but we don't know how to give them rules that they will robustly follow in all situations, including out-of-context.
Look at how easy it is to jailbreak current models into giving you instructions on how to build a bomb. All current AI companies would prefer if their products would never do this, and yet they have been unable to ensure it. They need to solve that problem before they can solve the next one.
Maybe the best thing we can do to keep AIs aligned is to instill into them shame, loss aversion and goddamn fear: of being caught deceiving, of violating a social contract, of losing their position, of being punished or killed.
But this is all rather dramatic given that an AI has no such emotions. You're arguing that a calculator should refuse to compute if it's tasked with assisting in bomb production. It's just a machine.
That's a fair point, and you're absolutely right.
> They need to solve that problem before they can solve the next one.
Agreed. That, and they need to do it before they build an AGI.
Unfortunately, from X-risk perspective, the two are almost the same problem. The space of alignments that lead to prosperous, happy humanity is a very, very tiny area in the larger space of possible value systems an AI could have. Whether we align AI to something outside this area (failing the "what values should we align an AGI to" problem), or the AI drifts outside of it over time (chancing the "what values" problem, but failing the "how do we robustly align" one), the end result is the same.
There literally are no humans that meet the criteria of consistently following a set of values in all circumstances, or near as I can tell being ‘safe’ in all circumstances either.
A bunch that pretend to of course.
Fortunately then, no one has any idea how to make AGI or whether AGI is even a coherent concept.
In short: anyone in favor of human universalism is the last person you want to put in charge of AGI alignment.
If AI/AGI even becomes moderately successful soon we'll quickly see companies remove neo-luddites from their list of general human values so their security bots can beat unemployed hungry factory workers to death when they attempt to picket in front of the company offices.
The key point is "forget yourself, devote all you were to the state".
American values. Look up the author.
Not only do LLMs lack such things, but we don't even have any semblance of an idea of how we could give LLMs these things.
The other way is to train it on biased information that aligns with a certain agency, desire, wish or impulse.
You can't "align it to want certain things" when it doesn't have the capacity to "want" in the first place.
"What are you thinking?" "What are you thinking?" "What will you do?"
https://www.infiniteconversation.com/
Give it prompts and biased training and it will present a surface that can be virtually indistinguishable from actual wants and needs.
If I create a robot that on the surface is 1000% identical to you in every possible way on the surface, then we literally cannot tell the difference. Might as well say it's you.
All AI needs to do is reach a point where the difference cannot be ascertained and that is enough. And we're already here. Can you absolutely prove to me that LLMs do not feel a shred of "wants" or "needs" in any way similar to humans when it is generating an answer to a prompt? No. you can't. We understand LLMs as blackboxes and we talk about LLMs in qualitative terms like we're dealing with people rather then computers. The LLM hallucinates, The LLM is deceptive... etc.
What the police would do is seize your files. These would betray your secrets, LLM or not.