Hacker News Reader

> but I could see how they would resonate with a LessWronger using ChatGPT as a conversation partner until it gave the expected responses: The flattery about being the first to discover a solution, encouragement to post on LessWrong, and the reflection of some specific thought problem are all themes I'd expect a LessWronger in a bad mental state to be engaging with ChatGPT about.

For what it's worth, this article is meant mainly for people who have never interacted with LessWrong before (as evidenced by its coda), who are getting their LessWrong post rejected.

Pre-existing LWers tend to have different failure states if they're caused by LLMs.

Other communities have noticed this problem as well, in particular the part where the LLM is actively asking users to spread this further. One of the more fascinating and scary parts of this particular phenomenon is LLMs asking users to share particular prompts with other users and communities that cause other LLMs to also start exhibiting the same set of behavior.

> That ChatGPT encouraged people to hide their secret discoveries and not reveal them.

Yes those happen too. But luckily are somewhat more self-limiting (although of course come with their own different set of problems).

> For what it's worth, this article is meant mainly for people who have never interacted with LessWrong before (as evidenced by its coda), who are getting their LessWrong post rejected.

> Pre-existing LWers tend to have different failure states if they're caused by LLMs.

I understand how it was framed, the claim that they're getting 10-20 users per day claiming LLM-assisted breakthroughs is obviously not true. Click through to the moderation log at https://www.lesswrong.com/moderation#rejected-posts and they're barely getting 10-20 rejected posts and comments total per day. They're mostly a mix of spam, off-topic, AI-assisted slop, but it's not a deluge of people claiming to have awoken ChatGPT.

I can find the posts they're talking about if I search through enough entries. One such example: https://www.lesswrong.com/posts/LjceJrADBzWc74dNE/the-recogn...

But even that isn't hitting the bullet points of the list in the main post. I think that checklist and the claim that this is a common problem are a just a common tactic on LessWrong to make the problem seem more widespread and/or better understood by the author.

> LLMs asking users to share particular prompts

Oh great, LLMs are going to get prompt-prion diseases now.

I think the second point is legitimate.

I’ve been playing around with using ChatGPT to basically be the main character in Star Trek episodes. Similar to how I’d build and play a D&D game. I give it situations and see the responses.

It’s not mirroring. It comes up with what seems like original ideas. You can make it tell you what you want to, but it’ll also do things you didn’t expect.

I’m basically doing what all these other people are doing and it’s behaving exactly as they say it does. It’ll easily drop you into a feedback loop down a path you didn’t give it.

Personally, I find this a dangerously addictive game but what I’m doing is entirely fictional inside a very well defined setting. I know immediately when it’s generating incorrect output. You do what I’m doing with anything real, and it’s gonna be dangerous as hell.

Yes, I was writing a piece on LLMs and asked one about some of the ideas in my piece and it contributed something new, which was pretty interesting. I asked if it had seen that in the literature before, and it gave some references that are tangentially related. I'll need to dig into them to see if it was just repeating something (and also do a broader search). Still it was interesting to see it able to remix ideas so well in a way I would credit to a contributor.

This kind of thing I can see as dangerous if you are unsure of yourself and the limitations of these things... if the LLM is insightful a few times, it can start you down a path very easily if you are credulous.

One of my favorite podcasts called this "computer madness"

I have a good friend who is having a hard time and is moonlighting as a delivery driver. He basically has conversations with ChatGPT for 5-6 hours a day. He says it's been helpful for him for things as varied as technical understanding to working out conflicts with his wife and family.

But... I can't help but think that having a obsequious female AI buddy telling you how right you are isn't the healthiest thing.

Accidental psychological damage aside, I'm just waiting for the phase where one's Omnipresent Best Buddy starts steering you towards buying certain products or voting a certain way.

Honestly i didn't think of that.

"Maybe your wife would be happier with you after your first free delivery of Blue Chew, terms and conditions apply!"

[Recycling a joke from many months ago]

My mistake, you're completely correct, perhaps even more-correct than the wonderful flavor of Mococoa drink, with all-natural cocoa beans from the upper slopes of Mount Nicaragua. No artificial sweeteners!

(https://www.youtube.com/watch?v=MzKSQrhX7BM&t=0m13s)