Generative AI could be an authoritarian breakthrough in brainwashing | Hacker News Reader

Generative AI could be an authoritarian breakthrough in brainwashing

thehill.com

Generative AI could be the perfect tool for a 1984 world.

Remove whatever is inconvenient from the record, replace it with 10x more generated ahistorical content. At nearly 0 marginal cost.

There will be lots of bad effects. If you don't think the NSA is investing GPTs for mass surveillance, think again. Unless they already are..

I would be disappointed if no one at NSA used "HYPNOTOAD" as a moniker for a related tool.

/jk

With the amount of tools named after pokémon in the leaks over the years, I figured they'd be nerdy enough to use Futurama

I honestly, don't see it making much of a difference. There's already more disinformation than a person can consume in a lifetime.

Which to me suggests teaching critical thinking, to people, to help ensure we are making decisions based upon worthwhile information.

The only problem with this is that it ends up being used disproportionately to criticize information which contradicts their beliefs. It's the same problem smart people have - they're more capable at rationalizing their beliefs than they are at deciding what to believe.

That's not critical thinking then, it's reactionary thinking. Critical thinking is a tool for challenging our own beliefs and assumptions.

I agree. Critical thinking is often paired with developing the methods or tools of critique. Instead of applying the tools of critique to one's own thoughts, these tools get applied selectively to ideas, especially ideas one doesn't agree with already.

For example, recognizing bias is a component of critical thinking. Rather than recognizing one's own biases, the reactionary thinker will instead tend to recognize bias in other's assumptions and arguments. They apply different standards to their own thoughts than they do to others'. You're absolutely right that this isn't critical thinking.

My hunch is that critical thinking doesn't work if there is isn't an space to be wrong. Even here, there's a tendency to "destroy opponents with facts, logic, and rationalism." I've participated in this myself. Frankly, it entrenches reaction instead of arriving at truth.

The subject finds it easier to retreat into the realm of the imaginary than to repeatedly confront a rupture in the symbolic order.

Apparently vast swaths of humanity are dumb as bricks and teaching them to "think critically" has somehow ended up with a bunch of of knee-jerk contrarians who think they've uncovered conspiracies from global warming to covid vaccines.

I've been told the "free marketplace of ideas" will sort these sort of problems out any day now. Unfortunately the FMoI has zero capacity to handle Brandolini's Law and has shown itself to be so useless it can't even set Flat Earth believers straight.

https://en.wikipedia.org/wiki/Brandolini%27s_law

smart people are just as susceptible

More so as you mentioned above. I know plenty of people who aren't that smart who know they aren't that smart and are humble and don't presume outside of their expertise. But there's a certain class of "educated" folk who think that because of their education they know everything. Let me say that troubleshooting software issues with doctors is absolutely the last thing you want to do. Please let me talk to the secretary who claims she knows nothing about computers.

at population level critical thinking is a dangerous hero myth. AI shows us clearly that our own faculties are very limited and there is no individual defense against propaganda and lies.

Treat everything as a lie. Boom - defenses go waaay up, and this avoids acceptance of lies.

People who live like this tend to be insufferable. It's intellectual auto-lobotomy.

To defend and clarify, another way to "treat everything like a lie" is to be a skeptic. And being skeptical doesn't necessarily require revealing one's skepticism.

To illustrate - one can be empathetic and choose to accept that other people have perspectives which work for those others, though not for themselves.

Keeping quiet about one's skepticism doesn't require one to have a superior attitude, or be arrogant.

On second thought we are likely talking about two different things.

You could make 100-1000000 fake profiles for only one person eventually and dominate their entire online world with fake bots.

Or think about this - prominent critic of the regime? Good luck getting anyone to find you online when there are thousands of AI-generated fake social profiles and personal websites claiming to be yours, all saying things that are vaguely plausible but not in any way substantive.

This has largely been my reaction to all the "But AI can be WRONG!" hysterics which seem to crop up. The internet is absolutely full of wrong information and there's no sign of it slowing down in any way. We don't dismiss the internet because it can give wrong answers. Why are some people so short sighted with these models?

Because we are now going through what could be described as another "industrial revolution" in the production of bullshit.

And given the state of the mainstream media this is really saying something.

it is bad, but it will get so much worse. even videos are easily/automatically generated showing people saying things they never said (i. e. speach of biden). and the actual problem will come from people not knowing/caring if it is fake, yet they vote accordingly

(sarcastic take on the article) Well, thank god it can only be used for Russian or Chinese disinformation. We just need to ban anything Russian or Chinese and we are safe! Do avoid mass outrage among our own western population, we can facilitate tools like ChatGPT to make it appear there's a broad consensus among the population for these measures, and to automatically attack all criticism with consistent talking points. Democracy is saved!

Reply on news.ycombinator.com