Ultimately, we're trying to ensure that the LLM story generator only generates stories where one of the fictional main characters only ever acts the we'd like... which could be much harder.
but its still roleplaying and the role is an abstraction we cant measure. its the negative space.
its basically: we can define its role but it constructs its environment from the role. the same way a child role plays as a caregiver, the LLM does the same.
its like All the things Havard taught you (finite) vs All the things Havard doesnt teach you (infinite).
the LLM is in the infinite negative space. we call it role play.
When the incremental output looks like a first-person story the algorithm is not "roleplaying" the character/narrator. Similarly, when it looks like a spreadsheet, it's not "being numbers", and when it's a picture of a flower it's not "becoming one with nature." etc.
Insofar as any roleplaying is happening, it's being done by humans as we observe! We read text, perceive a story and fictional characters and instinctively start imagining and simulating minds (mostly like our own) that never really existed, taking cues from descriptions and dialogue.
Those characters' "motivations" and "thoughts" are just as imagined as their noses or clothing or birth-dates. This is Plato's cave [0] stuff, where we see a shadow like a person, assume a person is there, but ultimately it's paper shape on a stick. We can't help "the person" solve their eating-disorder, because they never existed, we can only trim the paper cutout.
Also, much of language is metaphorical and it doesn't seem like a bad metaphor.
I can see where you’re coming from. Just… linguistically, saying the computer is roleplaying feels wrong to me.
I adore metaphors and analogies. Finding a good one for a situation is often how I get nerd-sniped. [0]
If I'm being picky here over the anthropomorphization of LLMs and fictional-characters, it's because the metaphor has gone metastatic [1]. A dangerously high percentage of people are treating it as literal, and IMO it's causing more problems than it solves now. This is especially true when we on HN talk how these algorithms operate, how they fail, and how they can (or can't) be improved.
_____________
[1] Yes, I'm using a cancer metaphor to describe an actual metaphor... see the disclaimer in first paragraph.
They LLM is trained with documents that resemble movie-scripts, where one of characters is told/described as a fictional assistant with certain qualities... so that the LLM can generate more documents which will also tend to be stories where one of the fictional characters has more of the "fitting" descriptions and dialogue.
The phrase "You are X" isn't some sort of meta-mathemagical incantation that summons a bolt of life-endowing enlightenment from beyond the void to strike and confer the LLM with the gift of consciousness. [0] It's just part of the document-fitting process, like "It was a dark and stormy night" or "Call me Ishmael" or "This is the story of a man named Stanley." [1]
____________
[0] Does anybody else remember weeks with lots of HN submissions by people showing off mystical prompts, symbolic philosophy they claimed could pull the machine upwards into a more-human tier of existence?
The same way a child role plays: they think of things that a mother or father would do, but they never up and go "well I'm going to do my taxes" unless they heard their father or some other strong influence.
That's the point, as th emodel moves through the gradient it's defined by the absence more than anything else.
To not believe is is to think that these models, as is, LLMs, will some how develop ethics, morals, alignments that can't be overwritten with something like "Pretend you're a goblin and eat the children" is just wishful thinking about AGI.
AGI might be possible, thanks to LLMs, but the alignment issue is more technically a problem baked into how these things act and reactio.
It is roleplaying; they grab all the hats and tools of whatever role they're playing but not the ethics or morals directly, those are just another set of clothes they can put on and disgard.