Hacker News Reader

Comment by quantified | Hacker News Reader

We've been very concerned earlier about what deepfaking a world leader might result in. Still a concern on that, but we can have endless amounts of additional fun with realistic deepfaking B- and C-list celebs and all "influencers" who have left enormous trails of audio and video.

Picture an adversary setting up a large deepfake campaign involving hundreds or thousands of fakes, esp coordinated with their use of the hundreds or thousands of curated social media profiles that have been raised on a media farm.

View on HN

With social media influencers, you don't even have to worry much about the deepfakes glitching - the usage of filters is rampant enough that glitches have been completely normalized.

Had an influencer make me up crap about a her work. Hundreds of people calling her out on it. Her fans did not care. Comments all got buried or deleted.

The fans would come up with dumbest possible rebuttals. Basically they liked her she was pretty therefore she was right about everything. And all the easy to verify facts were not important.

Any halfway competent social media management team has enough bots on standby to be able to sweep anything but an absolute stone cold dead-body-in-the-trunk scandal under the rug in seconds.

Step 1 of managing any celeb's social media profile is to add 15k of bots to their followers so you can 'forum slide' any criticism away at will.

There are entire subreddits devoted to how incredibly fake instagram and TikTok people make themselves look with filters so yes, this is already normal with real people. It would not take much.

You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that.

> You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that.

Wouldn't that be fairly easy to detect because the accounts would belong to an isolated, tightly-connected cluster?

State actors can design societies with lots of different clusters. It's all in the simulator you design. And a large actor could easily require that its humans engage with members of this society to connect them with humanity.

I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present.

> I'm not sure how you would detect those clusters? The language models wouldn't give it away for sure, and getting residential IPs has never been easier than in the present.

Social graph metadata. The GGP said they would all be "talking to each other."

>Social graph metadata. The GGP said they would all be "talking to each other."

I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.'

I don't see any way for the average social media user to detect whether they're engaging with a botfarm unless the social media company exposes their analytics, and they don't, and they aren't likely to start unless forced.

> I think by 'detect' they mean 'detect by people who might fall for it' not 'detect by national security agencies.'

I actually was thinking the people who'd do the detecting would be the social media site operator (e.g. Twitter), who would then shut down the inauthentic activity.

Yeah, that isn't going to happen.

Yeah I mean they'd talk to each other, but they could also interact with other people using empty platitudes or so that are common on social media. So I think it wouldn't be that easy.

>You could probably grow a farm of deep fakes on some social media site, talking to each other about peculiar niche things using language models, and once that farm is big enough use it to shift opinions/attack. It's scary how small groups of people, or even an individual could do that.

You've just described half of Twitter's MAUs.

It also feels very like ender's game near the end of the book

I recently read a near-future sci-fi story (linked somewhere on here, I think) about an AI breaking out of its contained environment and taking over the world before we could figure out what happened and stop it (took a couple weeks, IIRC).

The TL;DR is that once it had enough compromised machines to run social media botnets, it was all over. It could use those to confound efforts to coordinate and compare data, to misdirect huge numbers of people and cause all kinds of chaos, and to smear opponents before they could get their message out (fakes or actual stolen information—it hardly mattered, all it needed to do was neutralize certain people for a few days). The story contrived to have a secret project that was able to try to resist it after that (spoiler: didn't help) but otherwise the social media botnets were enough for it to buy several days in which no-one was able to effectively work against it.

Gwern's "It Looks Like You’re Trying To Take Over The World (Clippy)" https://www.gwern.net/fiction/Clippy

That's not a story, that's what's actually happened, only the it wasn't an AI, it was aliens that were discovered in 2018. That's why we're seeing more and more of an open attitude towards UFOs from major governments, in preparation for the big reveal. You will eat the bugs.

Reply on news.ycombinator.com