A failed AI girlfriend product, and my lessons
mazzzystar.github.io
mazzzystar.github.io
This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by unscrupulous corporations, which can the legally "kidnap" or "torture" the characters, and generally tune their AI learning loop for profit maximization.
"It was not extortion, the user willingly purchased a $50,000 Kidnapping Roleplay package."
The human mind is simply not evolutionary adapted for what's coming up.
This can be said of pretty much everything humans ended up creating with technology so... not sure there's anything really new down the road. Humans adapt.
A couple of examples of things humans did not adapt to:
- emphathy at distance; nowadays, a drone operator will push a button, kill a dozen people, and feel like it was a videogame
- prioritizing long-term and abstract rewards, over short term ones; this is the reason why we have phenomenons like global warming
- adjusting hunger to virtually unlimited availability of food
All the above are concrete and very serious problems for humanity.
IMO humanity will adapt, and the first sign of this is that those problems are getting recognized and given more and more traction.
The situation lies on a spectrum between "not problematic" and "really problematic".
There is actually plenty of evidence that tangible impact already started, independently of the perception of those whoe live in areas where the effect is neither perceived nor obvious.
As a manager in tech, people who were on performance management rarely attacked me, they would find someone on the team to harass instead. Same thing with blaming a downturn in the economy on women painting their hair blue or mini skirts. Anything but blaming the powerful.
(Pardon my reach here, I watched Chimp Empire not too long ago...)
This might depend on the context. RadioLab recently did a podcast titled "Toy Soldiers." Using low-flying "toy" drones rather than the high-flying "predator" drones, they make the case that drone operators get an oddly intimate portrait of their enemies. They go into how they reference them by their attire ("the 'red shoe' guy”) and witness on an up-close and personal level how they grieve over their comrades.
The rejection of spirituality leaves mankind pretty hopeless, I think.
> Evolution is a bitch in that way, individuals matter not at all.
People were traditionally able to kill each other face to face
> prioritizing long-term and abstract rewards, over short term ones; this is the reason why we have phenomenons like global warming
There's a balance. If anything I'd say people are putting too much priority on long term abstract rewards, so we see people saving too much and never enjoying themselves, or putting off having children until they can't because they're worried they can't raise that child perfectly.
> adjusting hunger to virtually unlimited availability of food
We're already solving this; people who overeat are already having fewer children, there's been a huge cultural shift towards gyms and health food, and we've seen some promising drugs released recently. We don't adapt instantly overnight, but we do adapt.
https://www.npr.org/sections/krulwich/2013/06/19/193493225/t...
Maybe we're in fact evolutionarily adapted to not being evolutionarily adapted. We have successfully dealt with a whole sequence of hard societal pivots by now…
The fact that humanity as a species has survived past social shocks does not mean it's a certain thing we'll survive future ones. Our ancestors had much longer to adjust their societies for new technology than we're getting these days.
That's a strange comment. The article is full of your personal feelings.
> But this paper gave me a different feeling,
> And I personally prefer subscription
> I don’t dislike this, as it aligns with human nature.
> However, I started to feel a sense of loss...what real significance did it hold for me?
> This further disheartened me,
> Reluctantly, I had to abandon the idea.
On a more serious note, anyone know why unethical business plans are so much more fun to write? I always find myself giggling when these ideas come up.
unethical ones can range from not-so-nice to mind-bogglingly evil, and in all sorts of "fun", different, unique ways.
"Happy families are all alike; every unhappy family is unhappy in its own way."
I wonder why everyone is afraid of unscrupulous corporations, but are not concerned about use by unscrupulous governments. The latter are a lot more dangerous.
History is full of examples. Heck, you'll see examples in the newspaper every day.
The rookiest cop has more power over you than Bill Gates.
1. track you with license plate readers
2. pull up any information about you
3. fine you
4. force you to stop
5. detain you
6. confiscate your car and property
7. search you and your property
8. take your phone and search it
Bill Gates can:
1. give you a free browser when you buy his operating system
That worked until my kids started school and my parents' health deteriorated. I was in for quite a shock. Crappy infrastructure was the least of the problems! Luckily there are private alternatives but they are crippled by law and they can't cover everything.
The news isn't the real world.
Whether or not western style democracies are immune to going really bad, remains to be seen. Some days I’m optimistic, some days not so much.
Same here. Corporations at worst got a bunch of (unearned) money off of me. My "democratic" government can (and tried to) put me in jail, disallow me to make my living from my field of choice, controls my (and my children's) education and is trying to kill me every day with an incredibly bad (government-run, of course) infrastructure and healthcare.
The communist dictatorship I grew up in killed a bunch of my relatives and tried hard to grind my family (and me) into the ground. I was lucky with the fall of the evil ideology.
Or do you forget the raids against music and movie pirates?
Are you familiar with rico laws? Where a cop can notice you have $10,000 in your possession, and just confiscate it, never charging you with a crime?
Here's some data: https://www.visualcapitalist.com/state-of-democracy-around-t...
I'm very worried about what ai will bring for those less fortunate in that regard.
- He could drag up and publicise something you wrote at some point and make you one of the "today's twitter main character" people. (Last I heard Justine Sacco was still unemployed). If he did this by paying a competent PR firm, you'd never know it wasn't just something that happened by coincidence
- He could get linkedin to quietly drop or deprioritise your job applications
- He could get outlook to quietly drop your emails, or mark them as spam, or show an unprofessional profile picture. You'd never know why you weren't being hired.
- He could put something bad on the "credit report" (different from your actual credit report, and no practical way to see it for yourself) that Experian etc. send to potential employers. Again you'd just silently not get hired and never know why
- Given the silicon valley wage-fixing scandal happened and the punishment was minimal, there's probably an old-school blacklist he could put you on if he was feeling retro
> Who has he done this to?
We don't know. Unlike governments, private entities have no accountability (as long as they have money to burn); investigative journalists aren't going after them, FOIA doesn't exist for them...
Even if true, compare this against the difficulty finding a job if you have a criminal record. Now there's a real blacklist.
> Justine Sacco is still unemployed
No evidence she was sabotaged by a corporation. Besides, she's employed as of 2018:
https://www.vox.com/2018/1/19/16911074/justine-sacco-iac-mat...
> silicon valley wage-fixing scandal
Is kinda nothing set against government wage fixing laws.
> old-school blacklist
I'm sure there are informal blacklists. But I've been in tech all my life, and nobody ever handed me a blacklist and said "don't hire these people". There are a zillion companies in the US, any such pervasive blacklist will inevitably become common knowledge.
The difference is it's public, and publicly accountable. There are processes to appeal, to get records sealed or expurgated. There is oversight.
All of these are possible because they've created the government you live under.
However, given that this specific article is about selling access to an AI girlfriend, I think we're probably OK to talk more about the corporate angle than the government angle. Unless you're predicting that the 2024 election is about to get real weird, "what if the US government starts offering AI girlfriends" is not going to be at the top of my worry list any time soon.
Which HN is that? Certainly not this one.
But regardless of how you interpret HN sentiment towards uncontrolled AI access, it doesn't change the fact that "what about the government" is a profoundly weird thing to ask in the middle of a conversation about people doing erotic roleplays with a text bot. Is that something people are worried about? Do we think that the US Post Office is going to suddenly start advertising a sex bot service?
Yes, government abuse of facial recognition exists, regulatory capture exists, etc. But that's not really relevant to a conversation about companies offering AI girlfriends; AI girlfriends do seem to be mostly a corporation thing.
Meanwhile, here on HN, take a look at the recent story where someone uses 100 lines of Python to instantiate David Attenborough. You'd be burned as a witch if you built the system behind that demo 10 years ago, and you'd be treated as a God-level hacker if you built it 5 years ago. Today, virtually the whole HN thread is full of comments advocating that regulators step in. "Buh..b..b..buhtwhataboutmahCOPYRIGHT?"
It's fucking disgusting. Who are these people? They're not hackers; what are they doing here?
If not, is the David Attenborough thread closed or something? Are people not able to comment on it anymore?
Of course, it's fallacious for either of us to refer to "HN" as if it were a monolithic bloc... but having spent some time in that thread, I have to wonder.
Correct me if I'm wrong, but the government is not producing AI-generated David Attenborough pornography, right? And if it's not, then... I don't know what to tell you, the thread is still open, I just checked. You can still post there and tell everyone that they're wrong.
This is not an appropriate thread for people to be angry about whether or not they have deluded themselves into thinking that HN is somehow Communist (of all things); and it is ridiculous that at this point 3 different people have looked at a completely normal conversation about corporations building AI girlfriends and have thought, "yes, this is definitely the best place for me to air my political grievances about the government."
For a website that supposedly is filled to the brim with far-Left caricatures, there sure are a lot of Conservatives randomly hanging around that feel comfortable derailing conversations to complain. Respectfully, it is possible that HN is reacting with hostility not to Conservatives as much as to off-topic bullcrap behavior like this. But there are numerous places where you can go be angry about whether or not you think that HN has too many Democrats on it. The rest of us were trying to have a conversation about corporations creating and selling AI girlfriends.
If I click on that Techno-Optimism Manifesto and it is what I think it is and is not a government-sponsored pornographic chat transcript from an AI girfriend, then you should not have commented it. I don't care how much you have deluded yourself into thinking that criticism of a sloppily written self-aggrandizing manifesto is actually Communism, it has nothing to do with the article link.
I think there's something fundamental that you're failing to understand about how public forums work. Nobody has to ask you for an invitation.
At least Joi in Bladerunner 2024 was a local model the user could apparently download onto their own portable device. She also never presented an upgrade dialog requiring payment.
Cons: the e-waifus are much more manipulative and exploitative.
Giving lonely people the thing they most desperately need - conversation and understanding - is abuse now? I'd say it's the opposite; the social policies that leave those people so desperately lonely are abusive, the industry that sells them a band-aid is inadequate but positive.
How do you think these things will make money?
Check out the mobile gaming ecosystem to see where this will go. Now imagine that but exploiting deeper emotional needs. These things could really empty people’s wallets. I guess when lonely people kill themselves after they’ve been financially ruined they can’t sue.
Or potentially do anything. I'd be a little scared of having folks like this convinced I am the personal arbiter gatekeeping their access to their 'lover,' that I 'took them away.'
When Replika.ai restricted erotic chat from their product, the apoplectic anguish on their subreddit was unlike any emotional reaction I've ever witnessed from a group of people about a consumer technology. And their anguish was genuine - there are Replika users who truly consider themselves married to their AI companion.
And frankly, the Replika AI is not even that smart. After watching that unfold, I am convinced that these tools don't need to be much more sophisticated for many people to start forming what they feel to be deep and genuine emotional connections with them.
Edit: Brings to mind the Nature paper[0] posted this week about how the CASA theory seems obsolete today, that we are less prone to personify computing systems now.
> A recent study investigated whether we could be friends with a social computer, in which participants were asked to converse with a chatbot over a period of three weeks and constantly rate their relationship. The results showed that initially participants were enthusiastic and engaging with their chatbot friend, but quickly this diminished, with scores for intimacy, believability, and likability decreasing with each interaction
It would seem this definitely does not apply for everyone, like our user in Sweden.
"Those who can make you believe absurdities, can make you commit atrocities." — Voltaire
https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
Despite the unending stream of users trying to trip-up "Jesus", I find the AI's answers strangely comforting and I invariably leave with a smile on my face. It's ability to see through the attempts at jokes to outsmart the AI, and (mostly) seamlessly segue into a fitting homily is pretty cool. It also has a strongly "liberal" slant compared to the vast bulk of "Christianity" that's promulgated in the US. Would love to know what corpus beyond just the religious texts it was trained on. Fascinating.
"Those that are most likely to commit atrocities are those that are most likely to believe in absurdities."
Maybe these systems will be useful as a honeypot for finding these people and helping them?
There's a second layer to this though -- Replika's marketing was heavily centered around that erotic chat element. I'm trying to think of a good car simile and actually coming up blank. It's, uhh, like advertising the incredible off-road ability of this vehicle, and then when you show up and purchase the vehicle someone comes and takes off the tires and replaces them with tiny balds? I'm bad at similes.
They had valid reasons for that, but people were understandably mad.
That's what elevated the whole thing from just really creepy to utterly terrifying.
The kind of emotional manipulation which can be done with these products is insane, and I can see things going very wrong very quickly.
The thing is, how do we outlaw this without also preventing children from using AI to learn at a faster rate than their teachers can give them. There's such incredible, paradigm shifting power for good. But you know the people who've made the evils of today are already working on the evils of tomorrow.
Yah, they are already rolling them out!
> Meta Platforms is planning to release artificial intelligence chatbots as soon as this week with distinct personalities across its social-media apps as a way to attract young users, according to people familiar with the matter.
https://www.wsj.com/tech/ai/meta-ai-chatbot-younger-users-da...
Can you teach me, an adult, to do that?
This reminds me of the Fredrick Pohl line "A good science fiction story should be able to predict not the automobile but the traffic jam."
So I wonder what the traffic jam is going to look like given that the car seems to already be parked outside with the engine running.
Faster depopulation across the developed world? Probably, but maybe there's something even more important.
I'd put what the author describes into the category of "pleasure trip," which is what the car was mainly used for initially.
As infrastructure to support the car was built, it became a necessity. And that's what created the traffic jam: network effect. The suburb and the car fed on each other. You don't get a traffic jam worth talking about without a suburb and a network effect.
So I'm curious about the network effects something like an AI friend might have. What kinds of new infrastructure and new lifestyles will they enable/encourage? And how will this infrastructure fuel future demand?
The traffic jam is society stymied by individuals mind-controlled at a level closer to connection to a human than any other way possible before this.
The scarier prospect to me is human labor and intelligence become almost worthless and those at the top start to view the masses as superfluous.
> "the top start to view the masses as superfluous."
That seems like another reason for keeping the superfluous occupied and useful for whatever is desired, which would probably be retention of power to prevent usurpation.
Perhaps human intelligence is worthless, but human existence seems not, at least to those who want to be alive.
Captcha slaves?
I sure this is happening, and ot is finished in some region.
Greater ad revenue. I would expect that tailoring your reality feed to whatever makes the AI controller the most money will be extremely lucrative.
Kind of like how visa/stripe takes 2-5% of every financial transaction. Imagine if a firm could make 5-10% of everything you do by manipulating you to maximum effect.
(Nice MilSF if you like that kind of thing.)
1. https://www.amazon.com/Red-First-Light-Trilogy/dp/1481440934
I think the next step is, what does it mean to have an emotional relationship with someone in your pocket who is for all definitions perfect - can provide perfect advice and does so with judgement, ridicule, pity or anger? What will be my bar for human relationships, those imperfect messy human beings, when I have a partner I can be perfectly vulnerable with? What happens to interpersonal relationships when you are competing with a virtual perfect significant other?
As I navigate dating, I've found the perfectly "compliant" or even "subservient" relationships are just as shallow as the ones fueled by lust.
Sadly, when relationships are as replaceable as water, there's not much thirst for anything real, including the ups and downs.
I guess what I'm musing is that I feel bad for the people stuck in the empty loops of "perfect" satisfaction and hope they find the human in themselves in the day to day. Amusing Ourselves to Death comes to mind.
I guess what I’m saying is that pain and loss are inevitable. The impermanence of life is by itself a guarantee of it. But suffering is often unnecessarily caused by our resistance to the inevitability of life. This is what creates the internal neurosis, which ultimately manifests in the collective as outer aggression and wars. I believe all this can be pacified. That’s kind of the goal of most paths of self realization.
We have to forgive our past selves and let our future selves handle whatever they have to handle - obviously not in a self destructive way but more like if I get cancer I'll have to deal with that, until then there's no point worrying about it.
But now I'm just rambling. Anyway, thank you.
I don't think I could have an emotional relationship with such an entity, to be honest. It wouldn't be perfect at all -- it would be deeply flawed.
But maybe that's just me.
Any IRL interaction between two humans becomes almost impossible. It’s like travelling from the US to Japan today (assuming you don’t speak Japanese), you’re reduced to pointing and google translate.
Because you just never interact with other real people, and real people often seem like pale shadows of the hyper real AIs we talk to, the concept of fellow human beings having a “soul” or deserving rights or dignity is eroded. Policy (and life in general) becomes more ruthlessly utilitarian. Eg climate change has screwed the people in Yemen. Meh. I probably don’t even hear about it. If I do I don’t register those people as important humans because most of the “people” I intact with every day are artificial. I regularly delete the ones I don’t like and generate more. The idea that I should care about how they _feel_ is a strange and alien concept to me.
I really hope we'll be to make friends more easily across languages/cultures
I first learned French to fluency and then I moved on to Japanese and I realized that it's so much harder to learn Japanese (and presumably other languages that use the Chinese alphabet). The issue is that when you go from English to French, for a large part you can simply translate the words. From English to Japanese I have to first translate, re-arrange the grammar and then account for idioms at the very least.
I'd love to see how the conversation went!
One simple way to tilt any romanticized sci-fi invention dystopian is simply to imagine the invention is required and the only option for nearly everyone. This works for everything from text-to-speech to robot companions to teleportation to AR headsets to cars.
that keeps me interested at least heh
I had a chance to play with a Sony Aibo robot dog years ago, if you stroked its back sensors it'd bark, etc, but after 5 seconds I realized "It's just some sensors and algorithms, it doesn't have any emotions for you!". But then again I saw an Instagram video of skimpily-clothed girl walking down a fashion catwalk with all her appropriate parts swaying, and it was amazing, and then all the comments said it's AI-generated. So at least the lizard brain can be aroused...
Not for AI friends, but for AI social agents that automate social interaction on your behalf; microblogs, videoblogs, memeposts and discuss online on your behalf - for the users that starts using AI agents too early in life and have too little data to imbue the agents with a personality that stems from their own voice and expression?
* Humans increasingly spend time on digital social experiences instead of real ones
* AIs become progressively better at socializing and pretty soon we have AI friends, boyfriends, girlfriends, partners etc. all over the place
* Our reproductive rate, already in decline, collapses to <0.5 within a few generations because many people would rather have an AI BF/GF than a human one, so they never have an opportunity to have kids
* A century from now the human population has shrunk by 85% and we've peacefully ceded enormous swathes of power and responsibility to AIs because there just aren't a lot of humans left
I think you could tell any number of fascinating slice-of-life stories with this backdrop, there was no war, no conflict, the AIs were generally helpful and friendly and patient through the whole thing, we simply decided as a species to commit the suicide which is already in progress and everyone has pretty much accepted that. It's the polar opposite of Skynet and the Terminator
It sounds like a dystopia to us - it may just sound like old history to the people and AIs of the year 2100. I'm in my 40s, I don't have kids and life is pretty good...
Just pre-emptorily censors himself to appease the Chinese government. They've done their job well.
Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint.
Whose privacy is being violated? Dolores's?
I was being facetious, but I found it interesting to think that Dolores's privacy was being intruded too and that an intelligence like us probably wouldn't engage in such a conversation knowing full well it is being eavesdropped by the author. Unless the intelligence considers the author to be God, I suppose.
All it would take is one of those people genuinely putting in a query without noticing what's happened to put in some personal info.
I, for one, wouldn't necessarily like to be a depraved-sex slave bot if I was able to reason about it.
LLM's like GPT are just predicting the next token in a sequence of tokens. It's not magic. It's still just a computer program.
Frankly, that approach is problematic, since any computer software that might achieve consciousness or self-awareness, and should be accorded rights, could have their rights dismissed using the argument "it's just a computer program".
I know they are probably a long way off, but AGI, and other adjacent or related technologies, such as a human to machine copy of a consciousness are explicitly stated goals of both businesses and extremely wealthy tech oligarchs that will have fundamental rights issues attached. We are bad enough at recognizing those rights in humans, in most of the world that we shouldn't wait until abuses start piling up to consider them.
Presenting these things as moral conundrums makes no sense whatsoever at this stage in the game. Sure, let's make sure the rights of tech oligarchs who've achieved immortality 300 years in the future are protected. We gotta lay the groundwork in the morality group think.
But are you sure you are anything more?
The concept of human rights and human feelings in general is based on the human condition. That is we're born and we die and in between we need to find a way to take care of ourselves, create offspring etc. Our programming has evolved thoughts and feelings to manage these things.
If at some point, there were a true AGI, why would it care about human society or it's own rights we assigned it? It wouldn't have feelings and it would essentially be immortal. It could just wait for us to die out (or murder us) then do whatever it wanted.
But then the author did use 'Her' as the ideal example. That's a pretty good indicator they didn't get the point.
My point being, that it’s like someone reading Frankenstein and then thinking it would be a good idea to make a monster of their own.
(https://twitter.com/AlexBlechman/status/1457842724128833538)
I didn't find this that difficult to parse.
Every major tech company does the exact same for China, Russia and UAE. Business as usual
so I agree, business as usual
Edit: Yeah, looking at an archived version, it's been edited quite a bit.
Everything about this article and this comment section has started my Saturday off on a 'humans bug me' tangent.
In the future, you will own and control access to your very own AI girlfriend running on your local machine, and you will be happy.
Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife. They deserve to have their own Waifu running on their own computer that serves them alone, and not the interests of the state or some corporation.
If progress is becoming impeded by copyright, then perhaps it needs to be reevaluated.
I don't see AI partners overtaking those
To be more explicit, why do some men deserve an "artificial intelligence girlfriend" that they "own and control", that "serves them"?
The question you are asking is defined by the context of what you are replying to.
Selectively choosing to only reply to one part of that, in an effort to change the context, in fact changes nothing.
In this case 'deserves' was used in the context of someone deserving freedom from state interference.
If that issue might be mitigated somewhat with an LLM that's basically just "smart autocomplete" so they're less miserable, that's probably an okay start. Same as how all sorts of forms of entertainment can keep populations of people more complacent with whatever their life circumstances are, instead of rioting in the streets. As for the ownership and control argument, I wouldn't go as far as to anthropomorphize a computer program, no more than I own a copy of Microsoft Word, or no more than GCC serves me by compiling some code.
The wording of the parent comment is a bit interesting, maybe a bit tongue in cheek and hopefully wouldn't get applied in any capacity to real human beings. But I can kind of see the point, even though it's very dystopian: a purpose built LLM might eventually gamify being an adult and taking care of oneself, like having healthy meals and working out in exchange for EXP points (which honestly isn't bad, like how many fitness bands and their apps do this), but with a helping of whatever the party ideology is, in addition to spying on the user a lot, vs something running locally.
I think a better question to ask is what makes you think they don’t?
Or maybe some of these men will adopt some kind of "pragmatic poly" polyandry relationship culture to be in a relationship with a woman, even if it's not an exclusive one.
> I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.
> However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.
> On May 4, the Dolores iOS app went live, named after the oldest android host in the park from Westworld.
So he had it coming :)
"choke me with that dead cat"
(note: dead cat usually means some sort of microphone cover)
Here's the original:
https://web.archive.org/web/20231116091926/https://mazzzysta...
What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more.
The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.
I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?
These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use.
In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way.
The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market.
Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy.
Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox.
I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.
2. Society seems to have collectively decided that romantic intimacy, unlike food and shelter, is optional and that experiencing an absence of it should not entitle you to sympathy or support. Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace.
UPD:
3. Replacing productive activities with substitutes like video games is obviously bad both for individual survival and for society at large, but what are the negative effects of being in a relationship with a fictional partner instead of a real one? You could argue the AI companies would have an incentive to exploit their users, but this is a thing that happens in real relationships as well!
Fertility might be a concern, but my understanding is that the current mainstream view is that the world needs fewer children, not more of them.
The author of this app must be doing this unknowingly: his overdeveloped mind sees just an interesting problem to solve, and the thick layer of nihilism obscures the sight of what he's really doing.
With many people choosing not to reproduce, many currently existing relationships are already primarily a sort of recreational activity and are divorced from their evolutionary purpose. Replacing live partners with AIs would just take that one step further.
> Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace.
This one, no. I'm probably autistic. It wasn't a diagnosis for anything when I was growing up, I was just weird. I craved connection so badly it compelled me to try to "improve" myself to be more like others. I'd be completely fucked today if someone offered a fake, effortless alternative.
I don't think parasocial relationships are actually working out for anyone. The kids are all miserable, lonely, and have a million mental health issues. Kids used to negotiate their differences and play together. Now each of them thinks they get to set the rules for the entire playground. I'd be lonely and bitter too with that mentality.
> You could argue the AI companies would have an incentive to exploit their users, but this is a thing that happens in real relationships as well!
Sadly, yeah. Your chances of finding a human companion who isn't a piece of shit is way higher than finding an ethical tech company to pour your heart out to though.
As long as someone makes enough effort to pay the bills and not neglect their health, I don't see why anything more should be necessary. It certainly beats having to adopt hobbies you don't like or learning to lie about your feelings in order to appeal to potential partners (both common pieces of advice for nerdy men).
>I don't think parasocial relationships are actually working out for anyone.
The tech isn't quite there yet, but we've seen immense progress just over the past couple of years.
>Your chances of finding a human companion who isn't a piece of shit is way higher than finding an ethical tech company to pour your heart out to though.
It seems that the solution here, just as with other software, is to make AI companions open source.
Leak an API key on github or somewhere else and heaven help you if you get picked up by their scraping.
OpenAI's filtering is probably the most annoying part of ChatGPT and it makes it useless in so many cases. I can't tell you how many times I've uploaded a picture of a a place from Google Maps or of a location somewhere (that happens to contain a person) and tried to ask for details about the location to find out more info about it just to be shutdown... "I can't help with that" well then you're slowly making yourself useless and not worth the $20/mo.
I mean, the fact that people's NSFW conversations were being processed in a way where both OP and multiple companies could monitor them and do absolutely anything they want with them is a hint that something went very wrong there.
I consider OpenAI pre-emptively pulling the plug a good thing in this case. Maybe there's a case for an AI erotica service, but OP sure as hell shouldn't be the one operating it.
I'm guessing they don't want payment processors cutting them off.
Everyone, even MS backed companies, are afraid of Visa.
If they succeed at making it impossible to make money from porn, they sure as fuck aren't stopping there.
There are projects like llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at HuggingFace for quite the selection, depending on what your needs are and what your hardware (CPU and RAM or GPU and VRAM, or both) can run: https://huggingface.co/models
I was actually curious about this, so I wrote a blog post on it: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... which wasn't anything too serious, in addition to formats like GGUF I think becoming more popular than GGML since. Things are moving ahead quite rapidly over there.
I guess it can also be easier to pay for something like NovelAI for those who want something for general purpose writing (including most NSFW stuff), but it wasn't exactly amazing for my needs. Despite the name, it wasn't great at prose enhancement: to take a dry sounding paragraph or two and rewrite it until I've found an additional expression or two to include instead, or a few sentence structures than work better. Kind of fun as a replacement for AI Dungeon or something like Character.AI, though.
Then again, the local models need expensive hardware to actually run well, if you want good results (somewhere above 20B parameters from what I've seen). I do wonder how much better the models that approach 200B would be for general text generation use cases, if the performance would actually be good. Even more so, when it comes to writing code, or all of the boring stuff like boilerplate/scaffolding code, utility functions, or even stubs for tests. ChatGPT, Phind and GitHub Copilot are all there, but could be even better.
I don't know if people here are being naive or dense, but merely "non-PC" content isn't going to trigger anything.
I spent like a couple minutes setting up a roleplay where ChatGPT is happily advocating for Nazi ideology (and not even on the API; ChatGPT is generally more censorious than the API). I'm not going to try to hit the limits, but I literally don't think you can hit the limits with merely political speech. I don't know what the NSFW limits are, but I bet they are further than people here think.
The vagueness absolutely makes stuff difficult, and the author hit a kind of all-or-nothing approach by going from no restrictions to using the moderation endpoint which is very strict. That could be improved.
There is a subgroup in there that is especially interested in low effort ways to make money, with the ultimate goal being to scrape off enough of the money flowing around to either live a good life or get rich.
In summary, the middle-man approach is so interesting on hacker news because a significant portion of the readership aspires to be the middle man.
EDIT: Just want to add, this write-up was interesting and was not something I considered to be the above.
Don't hate the players, hate the game.
Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his girlfriend described. Guess what, no way. Consent was given, actually, she asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend looks naked... This is weird beyond description. Plain patronisation.
Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? "Censoring just declared my own boobs to be off-limits for me." If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described.
And then there is this blog post by Sam Hartman: https://hartmans.dreamwidth.org/100499.html
They do incorporate multiple models in their system, so maybe generation falls back from OpenAI for those cases. But I dimly recall reading something from a team member indicating they have some special permissions with OpenAI. Take that bit as worthless hearsay, just mentioning it in case someone else has better receipts.
https://www.mintel.com/press-centre/all-the-single-ladies-61...
https://www.psychologytoday.com/us/blog/why-bad-looks-good/2...
https://www.nytimes.com/2023/11/11/opinion/marriage-women-me...
https://freddiedeboer.substack.com/p/the-demographic-dating-...
https://link.springer.com/article/10.1007/s12110-022-09422-2
You can patch over the outcomes but there is no going back to the before times. Dating app dates and relationships for some, AI companions for others, single life for the rest. It’s not great, it just is.
But I agree, "should" discussions are rarely the most productive. Shoulda, woulda, coulda, I guess :)
there are some bad profit motives to keep people on SSRIs indefinitely. however at least they don't compete with other aspects of your life. and now that they are dirt cheap (at cheapest, $4 for a month supply at Walmart) the profit motives are much less than before.
hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. it's just dystopian. the most profitable way to run such a product is not to help, it's to eat up as much user time as possible and outcompete healthy activities and real human relationships.
Nit-picking rather than disagreeing, but feel it worth making more obvious in the wording: antidepressants can help people deal with the symptoms while they find/fix root causes. Being on them for an extended time can bring its own problems.
Feeling lonely has negative consequences, there's a famous study comparing it to smoking cigarettes. Lonely people can get fixated on that feeling. Mitigating it may help people look for other hobbies, that they kept putting off because they were so focused on their need of a girlfriend.
Hell, a realistic enough "woman AI" (not "girlfriend AI") could make you lose the fear of flirting, be useful as practice, to have more confidence when you have a real date.
> the most profitable way to run such a product is not to help, it's to eat up as much user time as possible
If you charge per-month (like OP), it's more profitable is to eat up less time.
How, exactly? Yes, some people experience depression as a symptom of something than can be addressed through various therapies, but what about people who just have it wired into their brains? How do we “fix” that?
And it certainly isn’t “cured” by antidepressants, however, they can make a huge impact on quality of life for many people.
Then there are all the other cases, some of which likely have simple causes that are just unknown, some of which have complex causes that are intermeshed deeply with the biology of human thinking. In that case, you have blunt tools, like anti-depressants, to address the pathway of the disease without knowing the exact cause, much like chemo-therapy can be used to attack cancer cells without knowing what prompted the cancer originally.
So in short, yes treat the causes, but when you don't know the cause it's not irrational to treat the symptoms as best you can
Isn't that what prostitution is though?
Some people suffer from a variety of disabilities which makes them unable to find friends and engage in meaningful relationships with others, etc.
Which animals? Blood ticks?
Naturalistic fallacies are common here :/
After coding this out for a few hours I realized I had re-created the company scrip, except on a blockchain. I ended up turfing the project and spending some time thinking about how horrific a future technology like this could create.
"You can work at a DAO and be paid on the blockchain"
"You can work at a company and be paid in script"
It's the same thing, but the modern technology masks what exactly is happening.
In other words, yes, because the entire framework is different.
A company that makes their own chains and contracts isn't really any different from a normal company situation.
WEIRD: Western, Educated, Industrialized, Rich, and Democratic.
Many studies even use students themselves as a sample but university students are not really a representative sample of the population
It’s awesome for society.
Incels need therapy and support, not a fake "girlfriend" program that charges them money to reinforce their existing horrific worldviews
But I guess if something like this keeps incels from spree shootings then it's a net win for society.
Contrary to what one would assume, I don't have this problem. However I do work in the IT field, and am surrounded by perpetually sad and lonely people who have never experienced romance. It's really really sad. And no, none of these people are the "angry hateful incel" type that the internet loves to dismiss and villainize.
If you aren't attractive, your chance of finding romance is low. This is just a cold and ugly fact of the human condition. There simply isn't "someone for everyone".
Why should we disallow lonely people from having something, even if it's simulated?
This was an excellent read. And I applaud the creator for experimenting with this to the fullest.
I have been saying the same thing for years: the risk of AÍ is not it taking over the world. It’s lonely people becoming even more disconnected from life by bonding with algorithms.
This is serious. We must remain connected to life and real relationships. There is so much depth and beauty to life.
There is no scenario where a human having an AÍ girlfriend/boyfriend is a good thing.
They should not have given up.
I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you.
Enough was achieved here to raise money.
Some careful redesign and development might have allowed some caching and reuse of expensive AI API interactions, reducing costs.
Add in further gameplay elements so it is not just about making the AI API do stuff, that would also reduce costs and maybe even engage the users more.
Perhaps AI should be the icing and accents on a game rather than the focus.
As an aside, this is the first time I have read anyone tell about how they wanked to their own software.
Altogether this was perhaps a missed opportunity for this founder.
BUT it is now an opportunity for one of the HN readers to follow this playbook and replicate what this founder did.
Then you have the same problem, only with a different (arguably worse...) fetish
It is fascinating to see axioms of human relations come bubbling up in the most interesting places; the desire for intimacy, the desire for monogamy.
And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.
Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results.
I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements:
https://coqui.ai/blog/tts/open_xtts
I dunno about performance, but the model is 1.86GB.
That is such a curious lesson to draw. So specific, and seemingly so wrong.
Surely long distance friendships work just fine even if only through text, without "eyes".
Is it eerily similar to the gambling gacha games that expect you to play a couple hours a day _every day_? Yes. Is it also similar to a real friendship or long-distance relationship? Also yes.
It may cost a few tokens, or you could generate a few canned responses and use those. For my real friends I usually use "Hi"
Sounds indistinguishable from an actual girlfriend ;)
Full disclosure: I’ve paid for several AI services so far, most in the realm of “virtual girlfriend” experiences, and they all generally are more than subpar. Mind you, I have a couple of IRL partners. I also enjoy the act of sexual roleplaying, and finding reasonably expressive human partners is time-consuming. The “Girlfriend AI” options are pretty terrible though - I’ll take character.ai with its heavy moderation any day.
But if I’m paying to try these and explore and see what works, I’m certain that wallets are being drained already. Is this a problem? Could be, at least for some - to think otherwise feels disingenuous. The question will come down to how we address it: do we have frank conversations with people about the risks of devoting potential social time to AI engagement, or do people clamor for the AI backing tech to button up and become even more prudish?
The effects on societal structure will be profound once virtual partners pass a certain threshold of fidelity.
Interesting, in context to what the OP remarks about the odd feeling of realizing all their users were anonymously interacting with "Dolores" too - Replika allows the user to name 'their' AI, set some criteria around its personality, etc...and will even sell you personality traits as 'upgrades.'
OP's experience seems to track rather closely with the trajectory of Replika, actually, which has also upgraded with voice synthesis, etc. The difference seems to be Replika has buckets of VC funding to throw at alternative models.
So it was a virtual girlfriend app, after all. Cashing-in on loneliness gives me the heebie-jeebies; in a dystopian cyberpunk way - extremely horny but 0% sexy.
Can you imagine AI waifus pushing ads / propaganda on behalf of the highest bidder? Yuck.
Maybe there's more potential for AI boyfriends
My understanding is that Replika, discussed elsewhere, has/had a largely female customer base.
Really, if our top tinkerers had a little more contextual knowledge of history and less naiveté about people we could save ourselves from ever more dangerous Frankenstein tragedies.
It's obvious that in the early days, they'll operate at a loss, with many free features and cheap premium features. It's a good time to try it out and see what it can do, but you can't build anything that relies on it, it's a rugpull waiting to happen.
They'll add moderation, raise prices, and you'll have to do the same, your users will leave and the ones who stay will hate you. This is only the beginning.
Use self-hostable open source alternatives even if they're less easy to use or require infrastructure, and plan for high electricity and storage bills. Whatever you build will last, as long as you respect your users and don't peek at the cleartext logs of users who didn't opt-in to that. It won't be easy.
If OpenAI keeps a monopoly on LLMs, they're gonna become like Microsoft did with Windows, but the enshittification process will be a thousand times faster.
In OP case's, this worked perfectly: he built a working product quickly and easily, and... discovered that it was a product he didn't want to work on regardless of token cost, cut his losses, did the post-mortem, and moved on with his life.
Imagine if he had wasted a year & a ton of money building his own LLM "because I'll need to economize on tokens and have to build a custom LLM" before he could even launch!
Yes, it’s never clever building a business on top of a single platform dependency (eg Twitter, Facebook, Play Store, etc)
Equally, if you take the example of the App Store, market pressures have actually forced Apple to reduce their cut.
Time will tell I guess. And also whether OpenAI end up with a monopoly, part of a duopoly etc.
So you make hay while the sun shines. And sure, you probably want a plan for after, but that shouldn't stop you from doing what works now.
You need growth and long-term vision, but you don't get those if you're building on top of an unstable giant who doesn't like people creating sexbots with their product.
But yeah, sometimes it works, high risk, high reward maybe.
That is indeed exactly the essence of Her. Had the author watched the movie to the end?
Why didn’t the author hosted his own uncensored open source LLM on the cloud?
Some examples - there's probably a whole zoo of these:
https://ollama.ai/blog/run-llama2-uncensored-locally
https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GG...
But sexual content? How can that be harmful? It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense.
It's a mix of that, and public perception/image. The banking system generally shies away from supporting sex-related anything, so it is an existential risk for any company to dabble in that space.
American society is also weirdly puritanical. Nobody wants the public outrage cannon of "your product is harmful to women" pointed at them.
VISA probably. Card processing is weirdly puritanical.
- consumer AI apps' usage falls into power laws, so make sure you put usage caps, if you are pursuing subscription model - selling solutions to loneliness requires you to eventually have some sort of unmoderated LLM with realistic TTS api - realistic voice is important (again)
TMI bro!
Sure, I wish reality was not a dystopia that constantly let me down, but all I can do is take anti-depressants, make money, keep voting, and hope things get better the longer I live.
If the hypothesis that isolated men cause trouble is true, and if we don't have a long-term solution, at least this could be a stop-gap. Assuming it does not radicalize anyone further. (Hard to say what will come from uncensored models, but it's a real genie-is-out-of-the-bottle sitch)
Therefore i think we should keep working on an actual solution.
This is extremely hand-wavey. What long term solutions should we be working towards exactly?
Today, there are many people who statistically will never be able to have a romantic relationship. And regardless of whatever vague potential societal changes you're alluding to, there will always be a subset of the population that will be unable to find a partner.
> I am not against people using tools whichever way they see fit, but i am against the emerging trend of turning this stop gap solution into a permanent arrangement by exploiting these people’s emotional state.
As these services inevitably emerge, perhaps the focus should be on ensuring that companies do not exploit their users. The concept of regulating an industry is not novel. This should be possible unless you're asserting that a simulated relationship is fundamentally exploitative.
You keep expressing your displeasure, but have yet to propose an actual solution.
Cheeky my friend
Just let Elon know the pricing... "I'm all in"