Weird people trying to fall in fake love with AI

Awful headline you added.

https://blog.giovanh.com/blog/2023/03/17/replika-your-money-... explains:

But this is a product that’s targeted at and specifically addresses the needs of people who are lonely and thus specifically emotionally vulnerable, which should make it worse to inflict suffering on them and endanger their mental health, not somehow funny.

...

As cringe worthy as “AI girlfriend” is, there’s no question that’s what they sold people. They advertised it, they sold it, and they assured users it wasn’t going away even while they actively planned on killing it.

I'm curious. Why does the AI do that?

Is it as simple as there is some fraction of romantic conversations in its training corpus that involve a breakup, and so there is a chance that it will attempt to autocomplete a response into a breakup conversation? Or is there something more complicated going on?