I'm guessing trial-and-error prompting? The new trick shot video clip.
For one, 99% of the "roleplay" models eventually drag into one of a handful of endgames: NSFW RP, suicide discussion, nonsensical rambling, or some failsafe "I don't know" state where it just slowly wanders into the weeds and directs the conversation randomly. This can be anywhere from a few messages in (1bq) to hundreds (4-6bq) and sometimes it just veers off the road into the ditch.
Second, the UIs for these things encourage a "keep pressing the button until the thing you want comes out" pattern, modeled off of OpenAI's ChatGPT interface allowing for branching dialogue. Don't like what it said? Keep pushing the button until it says what confirms your bias.
Third, I can convince most of the (cheaper) models to say anything I want without actually saying it. The models that Character.AI are using are lightweight ones with low bit quantization. This leads to them being more susceptible to persuasion and losing their memories -- Some of them can't even follow the instructions in their system prompt beyond the last few words at times.
Character.AI does have a series of filters in place to try and keep their models from spitting out some content (you have to be really eloquent at times and use a lot of euphemism to make it turn NSFW, for instance, and their filter does a pretty decent job keyword searching for "bad" words and phrases.)
I'm 50/50 on Australia's "16+ for social media" take but I'm quickly beginning to at least somewhat agree with it and its extension to things like this. Will it stop kids from lying? No. It's a speedbump at best, but speedbumps are there to derail the fastest flyers, not minor offences.
I'm beginning to think that children should not be using it but adults should be able to decide for themselves.
Second, restricting adult access to "dangerous ideas and information" is a slippery slope. The exercise of speech and press that the British considered to be treasonous preceded the American Revolution.
How are we going to hold AI liable for their part in causing a crime to be committed? If we cannot prevent AI from causing crime them AI must be illegal.
Everything it says is meaningless unless you assign meaning to it. Yes, I can see children thinking this is a real "being". Adults shouldn't have that excuse.
words have weight, it’s why we protect them so vociferously. we don’t protect them because they’re useless. we protect them because words matter, a lot.
A real person has agency. They can see you, get to know you, contemplate your existence as well their own. Empathize with who you are, etc. And we do the same things in return.
This is why when a real person says something mean or hurtful, it matters to us. Or if they threaten us, they have a real body that can cause us harm so we get scared and call the police.
I feel like most people who get indignant about the latest AI rage bait story know all this but feel the need to protect the rest of the world from out of control AI. This is an elitist and patronizing attitude. Let people decide for themselves. As I said, children are a different story. They are impressionable and I can see them misunderstanding how this works.
A world in which a reasonable adult would say the same?
Not really the case here, but I don’t think it’s an absolute.
9/10 times these issues are caused by the AI being overly sycophantic and agreeing with the user when the user says insane things.
The best part is "After gaining possession of J.F.’s phone, A.F. discovered chats where C.AI cited Bible passages in efforts to convince J.F. that Christians are sexist and hypocritical" which unfortunately will probably be a slam dunk argument in Texas.
> 9/10 times these issues are caused by the AI being overly sycophantic and agreeing with the user when the user says insane things.
Repeat after me: an AI for sale should never advocate suicide or agree that it should be a good idea.
I would write it as, why are we so quick to defend tech companies who endlessly exploit and predate human weakness to sell pharma ads/surveil and censor/train their AI software, etc?