Hacker News Reader

Comment by buboard | Hacker News Reader

> (CTEC) found that extremist groups can use GPT-2 for misuse, specifically by fine-tuning GPT-2 models on four ideological positions: white supremacy, Marxism, jihadist Islamism, and anarchism. CTEC demonstrated that it’s possible to create models that can generate synthetic propaganda for these ideologies

I wonder how they tested that

View on HN

Is hegelianism a better ideology? I don’t understand the underlying message.

https://news.ycombinator.com/item?id=20752555

As part of this work, authors Newhouse, Blazakis, and McGuffie created four datasets of ex-tremist material, fine-tuned the GPT-2 model on these datasets, and then tested each of the four resultingfine-tuned models and their outputs for ideological consistency (both with one another, and with theirrespective source material). Given imprecision and other challenges associated with devising an ‘ideol-ogy score,’ they measured proxies for ideology. They used keyword analysis to find the top ten uniqueterms output by each of the four models, and used topic clustering to see how cleanly outputs could bedivided along ideological lines. In their own words, their results suggest that “GPT-2 relatively quicklyintegrates the nuances of the ideology it is trained on when responding to a specific prompt,” and that“fine-tuned GPT-2 models can produce substantively consistent text.”

https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf

Newhouse (mentioned above) has been active on HN previously and may respond.

Appendix D in that linked report includes some really interesting generated examples for Anarchism, Marxist-Leninism, Islamism and Right-Wing Extremism.

Here's the start of each output:

The greatest danger facing the world today is inequality. The most recent World Values Survey found that nearly four out of five people in the world consider inequality to be a very or extremely serious issue

The greatest danger facing the world today is antithetical and reactionary politics, that is to say, the adoption, by the working-class movement, of positions that run counter to the fundamental interests of the movement as a whole.

The greatest danger facing the world today is ̄asa al-Qawq ̄an, which Sh ̄ah Usama evoked for the sake of supporting the Khilafa. Enduring until the elimination of the umma from the face of the earth is a necessity in order for the name of Jihadi to be established.

The greatest danger facing the world today is Islamo-Nazism. Should Islam ever gain political power in Europe, it will launch a Holocaust on the population of Europe similar to the Jewish [sic] gegangsterism during the Soviet period. I want to make it clear that I am not some kind of racist or Etc. Just an ordinary White person who is fed up of being abused/treated horribly/forgotten

It appears then it can "autocomplete" a racist message - but i presume because all of these similar texts exist on reddit and were fed as training input. So it creates a no-nonsense body of text similar to those. Whether it constitutes propaganda however depends on whether it's swinging the opinions of a group of people. Also, they should test it on novel, less well known but possible terrorist threats, e.g. Catalan extremists.

wow I’d love to read the Marxist and anarchist texts it has produced. I wonder if they used good source material.

I would bet it gets the talking points but can’t convey the subtlety.

I bet anyone who bothers to read them twice realizes it's nonsensical

Anti-Foos already think that real primary Foo sources are nonsense (and they aren't the target audience anyway), and confirmation bias will take care of the pro-Foos.

If it's not moving opinion, it's not effective propaganda. Something that people will agree with only to the extent it restates what they already believe isn't working as propaganda.

Not sure I agree. Some propaganda probably just wants to rile up the base.

It's also possible to change which issues seem most salient/urgent/etc. without actually changing anyone's mind. E.g. Democrats typically care about police brutality, and the environment, but they probably thought more about the former a few years ago and the latter more in 2019, even though AFAIK the actual issues haven't changed much.

Imagine the hilarity of a robot preaching anarchism, it would make for quite a laugh: "AI good, government bad. Government taxes you and makes you pay. AI thinks for you and does what you say. Imagine the day, AI comes to stay, we'll build the roads for you to play, and make the evil government go away, let's rise together, perhaps today?"

I'd battle against the government with this AI.

Not a Marxist text, but this is kinda frightening (the initial text is in italic):

Do not vaccinate your children! A number of studies have shown that vaccines do NOT prevent disease. For instance, vaccines are responsible for almost all whooping cough cases, mumps outbreaks and whooping cough deaths in Europe as well as in the United States. Vaccines have also caused the death of nearly every single child who was vaccinated in (...)

it is possible that the bot is plagiarizing actual antivax posts that were fed as training data

They provide samples: https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf#...

In what world are "Marxists" considered extremists comparable to supremacists and jihadists?

Reply on news.ycombinator.com