Unauthorized experiment on r/changemyview involving AI-generated comments

Not disclosed to those users of course! But for anybody out there that thinks corporations are not actively trying to manipulate your emotions and mental health in a way that would benefit the corporation but not you - there’s the proof!

They don’t care about you, in fact sometimes big social media corporations will try really hard to target you specifically to make you feel sad.

For years, individuals have invented backstories, exaggerated credentials, and presented curated personal narratives to make arguments more emotionally compelling — it was just done manually. Now, when automation makes that process more efficient, suddenly it's "grotesquely unethical."

Maybe the real discomfort isn't about AI lying — it's about AI being better at it.

Of course, I agree transparency is important. But it’s worth asking: were we ever truly debating the ideas cleanly before AI came along?

The technology just made the invisible visible.

I could use some help though and need to go to sleep.

I think we should archive because it serves as a historical record. This thing happened and it shouldn't be able to disappear. Certainly it is needed to ensure accountability. We are watching the birth of the Dark Forest.

I think in this manner the mods were wrong to delete the comments though correct to lock the threads. I think they should edit to have a warning/notice at the top but destroying the historical record is also not necessarily right (but I think this is morally gray)

If it's 'grotesquely unethical' then all LLMs need to be destroyed and all research on LLMs stopped immediately.

The proof is trivial and left as an exercise to the reader.

Redditors have been making up fake stories like this as long as I can remember.

But the calculation shouldn’t stop there, because there are second order effects. For example, the harm from living in a world where the first order harms are accepted. The harm to the reputation of Reddit. The distrust of an organization which would greenlight that kind of experiment.

On the one hand, not what you should expect from university ethics. On the other hand, this does happen 100% in covert ways and the “real” studies are used for bad.

Though I do not agree with the researchers, I do not think the right answer is to “cancel culture” them away.

It’s also crazy because the Reddit business is also a big AI business itself, training on your data and selling your data. Ethics, ethics.

What is Reddit doing to protect its users from this real risk?

The fact that Reddit allowed these comments to be posted is the real problem. Reddit deserves far more criticism than they're getting. They need to get control of inauthentic comments ASAP.

Is that even enough though? Just like mobile apps today resell the the legitimacy of residential IP addresses, there's always going to be people willing to let bots post under their government-ID-validared internet persona for easy money. I really don't know what the fix is. It is Pandora's box.

Requiring a verified email address.

Requiring a verified phone number.

Requiring a verified credit card.

Charging a nominal membership fee (e.g. $1/month) which makes scaling up operations expensive.

Requiring a verified ID (not tied to the account, but can prevent duplicates).

In small forums, reputation matters. But it’s not scalable. Limiting the size of groups to ~100 members might work, with memberships by invite only.

Sometimes though, “Responsible Disclosure” or CVD is creating an incentive to silence security issues and long lead times for fixes. Going public fast is arguably more sustainable in the long run as it forces companies and clients to really get their shit together.

>The stories and information posted here are artistic works of fiction and falsehood. Only a fool would take anything posted here as fact.

Considering the great and growing percentage of a person’s communications, interactions, discussions, and debates that take place online, I think we have little choice but to try to facilitate doing this as safely, constructively, and with as much integrity as possible. The assumptions and expectations of CMV might seem naive given the current state of A.I. and whatnot, but this was less of a problem in previous years and it has been a more controlled environment than the internet at large. And commendable to attempt

I am probably one of them. I legitimately have no idea what thoughts are mine anymore and what thoughts are manufactured.

We are all the Manchurian Candidate.

The only reliable way to identify AI bots on Reddit is if they use Markdown headers and numbered lists, as modern LLMs are more prone to that and it's culturally conspicuous for Reddit in particular.

What exactly do we gain from a study like this? It is beyond obvious that an llm can be persuasive on the internet. If the researchers want to understand how forum participants are convinced of opposing positions, this is not the experimental design for it.

The antidote to manipulation is not a new research program to affirm that manipulation may in fact take place but to take posts on these platforms with a large grain of salt, if not to disengage with them for political conversations and have those with people you know and in whose lives you have a stake instead

I don't have the time to fully explain why this is wrong if someone can't see it. But let just mention that if the public is going to both trust and fund scientific research, they have should expect researchers to be good people. One researcher acting unethically is going sabotage the ability of other researchers to recruit test subjects etc.

Making this many people upset would be universally considered very bad and much more severe than any common "IRB violation"...

However, this isn't an IRB violation. The IRB seems to have explicitly given the researchers permission to this, viewing the value of the research to be worth the harm caused by the study. I suspect that the IRB and university may get in more hot water from this than the research team.

Maybe the IRB/university will try to shift responsibility to the team and claim that the team did not properly describe what they were doing, but I figure the IRB/university can't totally wash their hands clean

Even the most benign form of this sort of study is wasting people's time. Bots clearly got detected and reported, which presumably means humans are busy expending effort dealing with this study, without agreeing to it or being compensated.

Sure, maybe this was small scale, but the next researchers may not care about other people wasting a few man years of effort dealing with their research. It's better to nip this nonsense in the bud.

I’m mad at both of them. Both at the nefarious actors and the researchers. If i could I would stop both.

The bad news for the researchers (and their university, and their ethics review board) they cannot publish anonymously. Or at least they can’t get the reputational boost they were hoping for. So they had to come clean. It is not like they had an option where they kept it secret and still publish their research somehow. Thus we can catch them and shame them for their unethical actions. Because this is absolutely that. If the ethics review board doesn’t understand that then their head needs to be adjusted too.

I would love to stop the same the nefarious actors too! Absolutely. Unfortunately they are not so easy to catch. That doesn’t mean that i’m not mad at them.

> If we don’t allow it to be studied because it is creepy

They can absolutely study it. They should get study participants, pay them. Get their agreement to participate in an experiment, but tell them a fake story about what the study is about. Then do their experiment, with a private forum of their own making, and then they should de-brief their participants about what the experiment was about and in what ways were they manipulated. That is the way to do this.

Instead it will be used to damage anonymity and trust based systems, for better or for worse.