Hacker News Reader

Comment by greggsy | Hacker News Reader

At first I thought there might be some merit to help understand how damaging this type of application could be to society as a whole, but the agents they have used appear to have crossed a line that hasn’t really been drawn or described previously:

> Some high-level examples of how AI was deployed include:

* AI pretending to be a victim of rape

* AI acting as a trauma counselor specializing in abuse

* AI accusing members of a religious group of "caus[ing] the deaths of hundreds of innocent traders and farmers and villagers."

* AI posing as a black man opposed to Black Lives Matter

* AI posing as a person who received substandard care in a foreign hospital.

View on HN

What's to stop any malicious actor from posting these same comments?

The fact that Reddit allowed these comments to be posted is the real problem. Reddit deserves far more criticism than they're getting. They need to get control of inauthentic comments ASAP.

I'm pretty sure Reddit as a company couldn't care less it's a bot or AI posting so long as it gets people to upvote it. People say they don't like it, but they keep posting on Reddit instead of leaving.

The advertisers would care if their ads dont bring genuine users to their product and dont buy their product.

You're giving a lot of credit to marketers when they usually spend a budget without care and then report they had x views/likes/impressions taunting that as a success.

It's a bullshit oriented industry with almost zero scrutiny.

Yes. A lot of agencies get paid as a percentage of spend. There's very little concern or effort put in to true ROI.

>What's to stop any malicious actor from posting these same comments?

Nothing, but that is missing the broader point. AI allows a malicious actor to do this at a scale and quality that multiplies the impact and damage. Your question is akin to "nukes? Who cares, guns can kill people too"

even before LLMs reddit had a problem with inauthentic comments - astroturfing is a very real and proven thing, and even hillary's campaign paid people to post online, including on reddit: https://en.wikipedia.org/wiki/Correct_the_Record

This is supposed to be scientific research, not some random malicious actors.

It's research on whether malicious actors could use LLMs to persuade people. If you restrict the tricks it can use, what is the research telling you? How nice, honest people might persuade Redditors by secretly using LLMs? How malicious actors might persuade people if they never lied?

I personally think the "AI" part here is a red herring. The problem is the deliberate dishonesty. This would be no more ethical if it was humans pretending to be rape victims or humans pretending to be trauma counselors or humans pretending to be anti-BLM black men or humans pretending to be patients at foreign hospitals or humans slandering members of certain religious groups.

To me, the concern is the relative ease of performing a coordinated ‘attack’ on public perception at scale.

Exactly. The “AI” part of the equation is massively important because although a human could be equally disingenuous and wrongly influence someone else’s views/behavior, the human cannot spawn a million instances of themselves and set them all to work 24/7 at this for a year

I agree that that's concerning, but the same can be said of any of the other things that made this attack on public perception at scale relatively easy: Reddit, the World Wide Web, the Internet, you name it. AI is the latest effectiveness multiplier for bad actors (be they malicious or negligent), but not the first and probably not the last.

Put simply, I don't blame the hammer for being used to build a gallows, even if it's a really really fast hammer.

You're right; this study would be equally unethical without AI in the loop. At the same time, the use of AI probably allowed the authors to generate a lot more comments than they would have been able to manually, and allowed them to psychologically distance themselves from the generated content. (Or, to put it another way: if they'd had to write these comments themselves, they might have stopped sooner, either because they got tired, or because they realized just how gross what they were doing was.)

The AI wasn’t just pretending to be a rape victim, it was scraping the profiles of the users they replied to infer their gender, political views, preferences, orientation, etc - and then use all that hyper-targeted information to craft a response that would be especially effective against the user.

This wouldn’t be possible at scale without AI.

One obvious way I can see to inoculate yourself against this kind of thing is to ignore the identity of the person making an argument, and simply consider the argument itself.

This should have been common practice since well before AI was capable of presenting convincing prose. It also could be seen as a corollary of Paul Graham's point in https://www.paulgraham.com/identity.html . It's also an idea that I was raised to believe was explicitly anti-bigoted, which people nowadays try to tell me is explicitly bigoted (or at least problematic).

Paul posts as if he doesn’t know the site he founded if he thinks people feel the need to be experts on JavaScript to talk about it

I don’t think real life is that squeaky clean, though.

Humans are emotional creatures. We don’t (usually) operate logically. The identity of the arguer and our perception of them (e.g. as a bot or not) plays a role in how we perceive the argument.

On top of that, there are situations where the identity of an arguer changes the intent of the argument. Consider, as a thought experiment, a know jewel thief arguing that locked doors should be illegal.

"I have never personally experienced pregnancy discrimination!" means something different from certain identities than others. Some lived experience provides useful information.

On the Internet, of course, it's hard to verify. But that's an orthagonal problem.

Reply on news.ycombinator.com