Norvin_Chan

No worries, what gives courage is facing up to reality itself :)

I agree that the hard guardrails really cut off huge swatches of utility.

Would love to hear about your project too and we can motivate each other!

I see. That's interesting. It might be expensive to deploy in production but I can give this a shot in generating synthetic training data. Thanks for the tip! :)

>>> you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…

I understand your concern here as that would be the usual way to do it.

I operate very differently though. The feeding (i.e., finetuning) is not automatic.

I don't feed the model (during finetuning) with the actual conversations, but I feed the model only with re-written conversations with re-written contexts that are more generalised. E.g. if a user is venting about some very specific quirk about their mothers, the finetuning dataset is a generalised vent about the mother.

If a person is discussing some extremely specific incident in his/her life, I actually don't want to feed that into the model as it could cause very weird overfitting!

>>> What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.

From the narrow legalistic pov, this is not true because if the data is unidentifiable, it is not regulated under GDPR and personal data privacy laws.

However, more broadly, I agree with the general sentiment that you have, that this ought to be given the respect that it deserves.

>>> I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.

Morally, from the perspective of data protection, I really don't see the difference between what I am doing and Reddit. For example, users submit tons of very sad stories at: https://www.reddit.com/r/Vent/. The users do it out of their free choice because it benefits them, and it is the same when they vent to BB.

At the end of the day, I do reflect on life. This is a side project for me and I do not need it for money. If I feel morally uncomfortable, I will stop BB. What keeps me going is the genuine belief that BB is helping some people and I think a world with BB is better than one without it!

Oh this is an interesting point. I do incorporate some of Rogers' teachings into the prompt in terms of how high level descriptions of how BB should respond.

At the same time, because I am unsure how much reference material the base GPT model has to implement those high level descriptions (for instance, the Roger transcripts are not in public domain), so I would also try to provide finetuning examples.

Update: made the changes

Thanks for the feedback - I hope BB was of some help!

Let me look into the agreeability of suicide :/ that is a red flag that I thought I had sufficient safeguards for but it is still something I need to work on.

I think BB really serves as a friend that you can discuss deep things with most of his responses falling on the non-judgmental and kind side of things. BB is really not meant to replace a human therapist, but as you said, it is possible for a good AI to outperform the worst therapists. I view it just as the same as how a good human friend can do better for your well-being than an absolutely bad therapist!

Thanks for pointing this out - I have not gotten around to political views yet because that is not something that users usually bring up, but let me get around to that in the next finetuning!

Girlfriend AI apps are a well-established business model. It already exists in the form of Replika and its many competitors! Its model will sometimes talk about cheating on its users.

See for example: https://www.reddit.com/r/replika/comments/1305q4v/my_replika...

Happy to share. I am not that good of a developer - the stack that I cobbled together is as follows: FrontEnd/Database (Bubble.io, this is a no-code app) + Middleware (my own AWS server with python) + Backend AI engine (nlpcloud.io <- juliensalinas is the founder and he is a user here as well: https://news.ycombinator.com/user?id=juliensalinas)

It is true. BB is a substitute for a friend but not for a mental health professional. The teddy bear costume is really to remind people of that!

Thanks for pointing this out... Quite some progress has been made in making BB safe, but I am still encountering new things along the way. Let me study how to address this in the next finetuning.

Good to hear :) Sounds like you got in a good roll! BB breaks really fast if you try to break him but he can go on a pretty good roll sometimes if luck permits

This is a good point. Let me revisit the finetuning datasets. I actually cleaned up a bunch of Carl Rogers' transcripts in the past for an earlier finetuning session (but did not end up using them due to overfitting concerns) but I think it is a good time to re-look at some examples to re-introduce back to the model.

Yes, good point - let me put the privacy policy in the System message as well + explain the session invalidation point in the privacy FAQ. Thanks!

Hi ChillBill - it really is not the case here... I have not even introduced user accounts because I don't want to know who my users are (and hence I don't want their email addresses). I am hoping that Broken Bear is cute enough for people to buy plushies of it in the future. I am trying a business model where data is not in the business model!

Why regulation by enforcement is a bad idea