Hacker News Reader

Comment by 2001zhaozhao | Hacker News Reader

2001zhaozhao

I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.

At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.

View on HN

It's already been happening for a very long time. The US' "Operation Earnest Voice" happened in 2010/2011[0]

And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]

[0] https://en.wikipedia.org/wiki/Operation_Earnest_Voice

[1] https://www.latimes.com/entertainment-arts/story/2022-07-19/...

[2] https://www.tortoisemedia.com/2024/02/26/depp-v-heard-who-tr...

Why are you being downvoting for sharing this relevant truth, complete with sources?

All threads that have... certain keywords... are like this. Talk about meta. Looks like I'm now in the positive (+5) after being in the negative just a moment ago.

Anyways, I got really interested in this topic around mid 2010s because of some really interesting reporting by Codastory[0] covering Russian "troll factories" especially used to influence opinion in eastern Ukraine, but also in many other circumstances. In Houston in 2016, Russian troll accounts planned both an ANTI Muslim rally AND a counterprotest to that rally all on Facebook.[1] Hundreds of real human beings showed up!

Russia got a ton of attention for their use of troll farms but the truth is that the US and certain allies have actually been using stuff like this for far longer and, imo, with much more success. In fact, Coda Story itself is funded by the NED which is a massive pillar of global US, uh, "soft power".

[0] https://www.codastory.com/

[1] https://www.npr.org/2017/11/01/561427876/how-russia-used-fac...

For me the main difference is if the activity is clandestine and meant to deceive/hide the origin. Everybody tries to influence things their way. Doing it covertly is a hostile act. That does not mean that I would not condone hostile acts against my/our enemies but it is something take into account. And regarding your Russian examples, it showed for how long they have seen the west as enemies and conducted hostile acts.

They view certain components of "The West" as their enemy - indeed, the very organizations which conduct the same operations against the Russian people.

There are no enemies - they are competing parts - competing for popularity, resources, power. This makes it possible to negotiate. Enemies don't, but competitors are a lot more pragmatic.

Anglo-/Western and Russian governments have been strategic rivals for more than two centuries. Britain repeatedly sought to limit Russian expansion that threatened its own interests, such as routes to India, access to Caspian oil resources, etc.

See also, the Great Game and the Eastern Question.

They’re enemies because they want each others stuff. Russia has a LOT of stuff that the British would be quite happy to ‘manage’.

> Russia has a LOT of stuff that the British would be quite happy to ‘manage’.

And vice-versa, which results in the polite games being played, where everyone knows everyone else is doing shitty stuff to everyone, but nobody will ever admit it because that might cause the shitty stuff to descend into a shooting war, which is something we try to avoid as much as we can.

Not any more we don’t. We’re actively attempting to start war with Russia and it is happening.

From approx 2008 on Russia has conducted operations that are in no way ideological but have only the goal to destabilize western nations. Supporting extremist parties of all orientations, magnifying events in their propaganda to maximize discord without any ideological consistency. They also quite frequently kill people on western soil. Sometimes using radioactive and chemical weapons.

Please elaborate when the west has done the same in Russia.

Just stop. The moral altitude you claim is a dire fallacy. You don’t speak Russian, or else you would already know the answer.

And I’m pretty sure you know full well that any examples given to answer your question in earnest will simply be dismissed as “Russian propaganda” - just as the Russians dismiss Western claims as propaganda, or even ‘justifiable, given Western hostilities’ - but there is a long list of actions against Russia by western powers with the intention of destabilizing Russian society, which, if you make the effort to inform yourself by reaching outside your bubble, you can see for yourself.

Russians can make just as many claims as Westerners can, and the only difference is which side one is invested in: thats what you’ll believe.

No matter what.

But, just to give you some hints as to what ‘the other side’ consider a valid response to your query, I asked some actual Russians what they would respond, and here’s the list:

- Assassinations: Daria Dugina, Igor Kirilov, Mikail Shatsky, (and BTW, Russians can quite confidently claim that the Salisbury Novichok incident was a false flag intended to harm Russian standing in the world - “Russians aren’t the only ones who synthesized Novichok…”)

- Radio Free Europe/Radio Liberty: 100% anti-Russian propaganda,

- The blatant censorship of RT in Western markets (although the Russians still have access to BBC and CNN, which they also view as state-sponsored anti-Russian propaganda organs),

- Emphasis on high military casualties (sometimes highlighting disproportionate impact on ethnic minorities from poorer regions),

- Coverage of economic costs, elite corruption, and contrasts between official narratives and frontline realities,

- Support for (or amplification of) Russian opposition voices in exile and independent media,

- The "UMBRAGE" Component of Vault 7, described by Wikileaks as an offensive Western intelligence/propaganda operation against Russia,

These operations are reciprocal. Both sides do it to each other, and you are foolish to deny it.

Otherwise .. I hope you learn to speak Russian and will start paying better attention to ‘the other side’, because that’s the only way you’ll get an answer to your question, legitimately. And even then, I doubt you’ll “believe it”, since we in the West live in a very, very tightly controlled propaganda bubble, to the tune of billions of dollars a year, which absolutely prevents any such sympathies for Russia from being expressed.

Our bubble is just like the Russian bubble, only its not in Russian.

One main difference is that those examples you brought up are ideological propaganda. RFE/RL try to amplify their message and yes, the current Russian regime is very antithetical to that message. Russian propaganda is not ideological, it tries to maximize the damage to western nations and uses whatever political and ideological message that will achieve that. They have been doing that for decades, which is something a hostile nation does.

As for the assassinations, those are wartime actions by Ukraine. Not sure how they are supposed to fit into our discussion.

As for Salisbury, so you found the story of these fitness supplement salesmen convincing?: https://www.youtube.com/watch?v=okrdqM6hNrI It has been my impression that not even Russia tries to push that narrative any more.

Ideology? Yes, pure hatred is ideological - no matter which side of the dialectic you choose to align yourself.

“American propaganda is not ideological, it tries to maximize the damage to Russia and uses whatever political and ideological message that will achieve that.”

Huh, what do you know, that can be “true” as well. And the US and its allies have been hostile towards Russia - actually, for centuries now.

I don’t find anything about Salisbury convincing - not the British investigations, nor the Russian ones.

And the reason for that is, the evidence presented never holds up in an open, public, liberal democratic court of law - instead, the ideological propaganda that “Russians are assholes because they kill people” is propagated in the court of public opinion, which is as vicious and banal as you can possibly imagine (or else we wouldn’t need courts in the first place), no matter whether you attempt to argue from the Russian or Western perspective.

Besides which, the moral altitude claimed “against” Russia by the West is simply a fallacy - you think we don’t murder people extrajudicially?

I’ve got an ideological point for you: dropping one bomb every 15 minutes for 26 years, mostly on innocent children, women, and men is wrong and should be stopped. Any state which attempts to stop that, gets my vote - but it is not going to be stopped by any state. It is going to be stopped by the criminal states’ citizen body, which is - as we see bright and clear today - completely and utterly distracted by the hatespew against the citizens of other states, intentionally, by the people dropping the bombs - because they also own and control the means by which ideological propaganda such as “hate Russia” or “hate the West” is propagated, and they use it to expand the market for their weapons production...

In short, you’re no better than any “Russian”, and neither am I. Our nations are murderous, criminal enterprises which produce mass murder at far, far greater scales than anything a trash can in Salisbury can produce.

So you are saying Germany and the UK have no right to see Russia killing people on their soil as hostile act because the US drops some bombs in the middle east?

The history is far more banal and less nefarious than you give it credit. Nobody gave a fuck about Russia. Russia was a self limited kleptocracy and everybody just ignored it because nobody needed the headache. That is why the reaction to the land grab in Georgia and later Crimea was pretty muted and everybody just hoped that Putin would be sated and ride into the sunset as great leader who took Crimea home into the motherland.

But he wants an empire, and an empire can not be an empire if nobody cares. So he overplayed his hand to sate his vanity. One of his first public announcements after the invasion was: "I hope I am now being heard.."

Well, the World heard and now it is raining oil in Moscow and the Russian provinces are getting depopulated.

I think that Germany and the UK have all the rights they need to prosecute the criminals that have committed crimes on their territory, and doing so openly and democratically is the right kind of justice.

But these incidents do not automatically give cause to hate the entire Russian nation any more than, say, the actions of the IDF and the Israeli government should be used to justify hatred of all Jews, everywhere.

Obviously, they shouldn’t.

> Nobody gave a fuck about Russia.

This is so untrue. Western empires have been trying to balkanize Russia and colonize its resources, aggressively - since the 1800’s up until the modern era. Russia has been demonized for centuries so that Western powers can have a justification for their war industries.

See, this is the problem with the lack of proper history education in the West. You can’t get your bearings from the History Channel alone.

And then, there is international law. Do you really think the US and its allies are law-abiding members of the international community, given the grand litany of human rights abuses, war crimes, and crimes against humanity that have been committed this century alone?

One bomb dropped every 15 minutes on an innocent life, for 26 years, does not give those next in line in the Western targeting sights much hope for peace. While Americans and other Western citizens (including Germans and Brits) are utterly myopic about this, the rest of the world are really not. The crimes against humanity committed by the 5-eyes states alone, far exceed anything that Russia has done thus far. By a huge, huge margin.

> “depopulated”

See also, Iraq, Afghanistan, Libya, Syria, Somalia, Pakistan, Palestine ..

Wake me up when you start demanding justice for the war crimes of Mosul with the same ferocious tenacity as you call for Putins’ ears on the table.

> Hate the entire Russian nation

This is the same nonsense many people screaming “antisemitism“ pull. Their comments are clearly about the Russian government with Putin at its head, the decisions the regime makes. They are clearly not talking about “the average Russian.” At no point has anyone mistaken your critiques of the US government for hatred of every American despite your use of “the US,” so why would you respond like that?

Don’t resort to such cheap tactics just to make a point online.

> those examples you brought up are ideological propaganda

These are the examples they were able to identify. And many of them could be so obviously identified because of ideological component.

The alleged censorship of RT is more a counter-propaganda move: RT is a Russian vehicle that engages in a number of psychological warfare ops.

And, BTW, the only thing Western countries have not done in Russia is assassinations with chemical or radiological weapons, possibly due to superior counter intelligence ops on Russia’s side. And because being identified was not in their best interest - unlike Russia, which is more easily identifiable up to recent times. At this point I am not sure how much destabilisation of the EU and NATO is coming from the US instead of Russia and China, which tended to be the usual suspects.

These days anyone can run a competent disinformation warfare campaign.

So you are saying that you also don't have any concrete examples of the West doing the same.

As well as the humorous implication that the West would be doing all the same nasty assassinations as the Russians, if it weren't for the brave and mighty Russian intelligence services getting in their way.

You're not supposed to acknowledge that think-tanks are sort of endemically propaganda outlets. You're supposed to think it's special this time.

Because cognitive biases are one of the hardest things to even acknowledge, let alone suppress. All the media out there has such a strong bubble effect, because people hate being challenged and love to stay in their comfy echo chambers. Remember labubu? Bao buns certainly do not.

There have been so many bot armies exposed over the years that one would think every narrative being pushed should be considered to be backed by bots. Some state-level actors even operate their narrative-pushing campaigns openly, yet people still happily deny their entire existence.

HN is very much not an exception to this rule. It's very hard to accept that maybe some thoughts you have were actually installed.

> people still

Maybe a few are just “people” (i.e. the same people who normally post in the community.)

But in most cases I’d suspect any such botting campaign brings along with it a few human agitprop agents embedded onto each target platform, well-positioned to deny and discredit anyone pushing back on the narrative the bot farm is spreading.

(Sometimes this is pretty obvious — a lot of accounts coming out of the woodwork to deny Chinese bot farms have a conspicuous “English as learned as a second language in Beijing, to a competent but not fluent level” writing style that does not often otherwise crop up in other discussions in those same communities. But other times it can be far more subtle, making use of operatives hired from within the community’s own majority-culture rather than relying on foreign nationals playing the part of same.)

A Chinese and Russian troll farm employs people from French-speaking Africa for their French-language operations.

There are many Ivorians and Senegalese promoting the CCP’s ‘Community of common destiny’ on Reddit, for example.

Edit: Community of common destiny for mankind/‘Community of Nations’

There’s bots that automatically downvotes certain topics on HN. I post at times that often coincide with the middle of the night in North America, and I get 3–5 downvotes within the next 5 minutes.

Talking about a country that sets up fake think tanks is a good way of triggering them.

These 2 sources say nothing about Heard using bots, only Depp.

> Why are you being downvoting for sharing this relevant truth, complete with sources?

Agreeing with something doesn’t make it 100% unambiguously true/correct. There are also many reasons people downvote, you don’t know the nature of the objections. It’s not worth getting upset over.

this is what online discussion will look like in the future thanks to AI, and a handful of hyperwealthy oligarchs controlling said AI

this has been happening since the dawn of information-communication systems basically. because ever since, people have had the ability to listen in, copy, learn from etc. - so there's always been incentive to spread disinformation for some parties.

brand defence, nation-state influence operations.

There are many different flavors of the same thing, trying to mislead competitors or enemies. Or to dupe consumers ofc... Since the tech is more and more available, now also 12 year olds can run insanely large influence campaigns, to boost their streaming profiles or other somewhat benign things, but obviously modelled after what larger players have been doing forever.

The first transatlantic commercial transatlantic phoneline (TAT-1) was immediately tapped on both sides by UK/US... before that it went over radio, which was simply intercepted.

Because of that nature of the coms system being easily listened in on, just like they do/did with the lil mirror-beam-splitters at GCHQ etc.., its been natural effect to then provide disinformation over those channels. perhaps even use those channels exclusively for disinformation, and have a secure channel to do _actual_ coms.

So as you can imagine there has been many decades now of practice in why and how to share disinformation effectively to undermine other peoples work or processes.

the particular danger now is not that the outside influence is new, but that more and more people are depending almost entirely on the information communication systems for all of their information.

the war against in person gathering and the family unit has been pretty successful, as they were pretty sloppy ways to exchange information to begin with. but at the very least you knew you were getting an actual opinion from someone, and you had context of their personhood and background for which to base that opinion.

Yes, but when your trying to cover up a genocide it's a bit different

I mean, the US has always done this. Look at the atrocities in Indonesia. Just 20 years after saying "never again" to the Holocaust and we have Goodyear using actual slave labor from death camps. It was the US doing too. A mass killing on the scale of the Holocaust. The "Jakarta Method" was then exported all around the world, especially to South America.

Both of their legal teams utilized bots to drive online discussions[1][2]

Neither of these sources support this claim as none of them include any discussion on Amber Heard('s legal team) using bots?

Indeed, seeing how slanted online discussion was, I suspect one side had many more bots, and I'm not sure if the other side had any.

Yeah, we don't need to "both sides" this. It's obvious which side had the bot army.

i'll bite: which one?

Depp's bot army was clearly more widespread and more successful. The reality is that it was a mixture of bot armies and the hive mind taking off by itself. A sort of organic-synthetic cyborg

To be clear, I wasn't being facetious.

I'll be completely honest with you about what I believe: I think the the UK trial unequivocally proved that Johnny Depp is a violent and abusive. The US trial, which was only about defamation, was clearly a sham trial and did not address the legitimacy of the evidence (UK's case found clear evidence for 12 of the 14 alleged incidences of violence. It also established a clear pattern of abuse).

Depp's behavior during the US trial is textbook DARVO (Deny, Attack, and Reverse Victim and Offender) and any abuse expert would be able to immediately recognize it.

The reason I "both sides"'d it is because

A) I remember reading about evidence that Heard's team also used bots (obviously much less successfully). I could not find that evidence this time. I am not personally critical of her using bots to defend herself but, if she did, it would still be a notable feature of modern highly public debates.

B) When it was all going down, I remember something like 97% of tiktoks being pro-Depp (I wasn't on TikTok, just read a study). My main goal was to get people to accept the reality that even legal teams are working with PR firms that run massive bot farms to influence public opinion. Regardless of my opinion on who is in the right and who is in the wrong. I thought if I took an explicit side, the male-leaning demographics of HN would be less likely to engage with my actual main point

I have been yelling about the thinktank "truth laundering" industry for years now.

They were instrumental in the whole "Uyghur genocide" thing where the only proof was Adrien Zenz suggesting there would be a possible future decline in population growth and called it a "cultural genocide".

The US State department has been running things like "Radio Free Asia" as state propaganda mills for decades. This has broken out into an industry of insiders getting board seats on various think tanks and churning out essentially policy desires as "studies". These studies are then used as citations for actual news articles to launder the policy desires into something that seems like a reliable source. Then people say "Look! this is really happening!" while 100s of articles are written citing the same untrustworthy sources, laundered through dozens of thinktanks funded by the DoD and state dept.

I think the big difference is that before the internet you'd need to be a Murdoch to be able to do something like this. A good PR firm could fool a number of news outlets into thinking something is legit, but it was a lot harder to convince a professional journalist than it is to convince the average Faux News watcher.

I used to believe that, but too many times I have been proven wrong. Remember when the "professional journalists" were screaming about a terrorist plot against the UN last year and it was just a garden variety SIM farm running fraud?

even with the internet you need a marketing company with the reach and ability, and social media platforms to use and abuse

the maturity of those platforms, their "we need to make big money" drive to allow obviously bad actors to work their magic, and the ability to automate is what has weaponized it

>the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority

not authority but circles of trust from real life connections. we had a nice brief moment of centralized asynchronous information exchange with strangers that was relatively trustworthy but we are going to go back to needing to talk to an actual human being in real time in order to receive an actual human opinion from strangers, and then on top of that we can built local caches that explicitly are not able to be contributed to by strangers but that your circle of friends and family may be able to use.

ive been toiling over some designs/implementations for making it digital but ultimately its a process adherence more than a tool thats needed. any old notepad and emails would work fine

It's already been everywhere for years. The most obvious one is any time someone says something bad about Russia or China, a load of comments pop up to say "Yeah but what about when america does <x>?". Never actually attempting to discredit the statement, just to steer it towards the US

The sheer volume of comments make it super obvious when it happens compared to normal conversation

> The sheer volume of comments make it super obvious...

Does it though? Remember that the Global South outnumbers the west in terms of population. And the west isn't really as politically popular as westerners like to believe:

- Why do some people support Russia's war in Ukraine? - https://politics.stackexchange.com/questions/81254/why-do-so...

- Why isn't the Global South pro-Israel like the West? - https://politics.stackexchange.com/questions/82511/why-isnt-...

As more and more of these nations come online (Redditors already complain about Indians "hijacking" the whole of Reddit), many in the west are experiencing the cognitive dissonance between what you've been told of how the rest of the world think about you vs what they actually think. (Of course, I am not discounting the role of State actors in trying to shape the discourse online, and I agree with you that is also a factor).

This is a terrible topic, but if you can’t understand where the Nazi reference comes from with regards to the Ukrainian conflict, you need to read some history books. Start with Stepan Bandera who is still considered an idol by many in the current Ukrainian government.

Yeah but the bias on Reddit and Twitter was the other way around when Russia re-invaded Ukraine in 2022. Suddenly 20 billion Ukrainians online who all disliked Russia, with PhDs in European history (2014 onward) and also vaccines for some reason.

The weirdness is how quickly and randomly those comments show up. Back when I used Reddit, I could mention some sensitive topic in a random place and there'd instantly be what I think are bots voting and commenting on it, sometimes huge comments with tons of links that looked canned.

Also it's possible those are from some insanely online people possibly running brigaiding groups on Discord, but that's not much better. It's best to be away from that. Let them "win" their online arguments against each other.

Certain platforms, like Reddit and Facebook, are certainly more commonly targeted so you may very well be right about your experience there.

Hi I am a human for sure and I can say we humans like to point out USA hypocrisy every time others are accused of doing things the US has done countless times while pretending to be on a moral high ground. Please don’t be offended by that. Just try to look at yourself before throwing stones at others. Beep boop .

I'm not american

You may be misunderstanding some of those comments. It's not changing the subject, it's pointing out that you don't actually care about that bad thing or you don't actually think it's bad, because if you did then you'd also think it's bad when the US does it.

> At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

Whining about AI is a red herring. If anything, LLMs lowered the cost of producing this type of propaganda.

What's novel about this approach is that the emphasis on building up a facade of authority and credibility is not a central part of the propaganda effort. They can get away with outputting the content in a way thats ingested by other LLMs among the reputable sources, and then they lean upon people's gullibility towards LLM output as some sort of credibility laundering.

There is a reason why one of Wikipedia's central policy is requiring reputable primary sources to support content.

An important nitpick: Wikipedia requires reputable secondary sources, not primary sources. You can't cite a politician's personal social media, for example, you have to cite reporting on those statements. This is part of Wikipedia's defense against novel research - it's supposed to be a record of what is commonly known, not a replacement for, say, Reuters or arxiv.

Propaganda is not a new thing. It might look slightly different on the internet, but it doesn't look that different. Creating organizations to push a narrative was just as much a thing 100 years ago as it is now.

IIRC the first or second chapter of Bernays' Propaganda (1928) talks about sophisticated influence campaigns in the fashion industry. Conscripting "influencers" to induce demand, and so on. As you said, literally 100 years ago.

Jill Fields at Cal State Fresno have actually narrowed it down to a (temporarily successful, but not in the long term, obviously) effort by the corset-making industry seeking to stay relevant after WWI during which corsets basically disappeared and there was an organized campaign that was organized both in person and through trade magazines. Women's silhouettes basically took a quick 180 right between 1900 and the start of the war only to turn very very retro in 1919 and while the first was organic and resembled how styles change in the century prior, albeit much quicker, the 1919 switch was openly plotted out in writing that was published but in such a niche publication that I get the sense that people not in the industry didn't even know that it existed. The cite to her paper on Google Scholar: https://scholar.google.com/scholar?cluster=17641417785638413...

You might have to shadow library this one, but it's a good read. Interestingly it seemed to be something entirely separate from Bernays whose observations were based on a derivation of what began as a homebrewed effort in a corner of the market. Valerie Steele wrote extensively about the trends and the after life but the source seems to originate in corsetry and ended up being copied by the garmet industry generally in the 20s.

Scale is different though. Tapping a phone isn't new, tapping _all phones_ is different. Altering a photo to change what it appears to depict isn't new, being able to do it in a few seconds and for free is different. So sure, propaganda isn't new, but being able to pump out an organization's worth of credible-looking content for cheap absolutely is.

"Quantity is/has a quality all of/on its own."

Not by a single lunatic operating from their bedroom.

Lunatics in bedrooms have been distributing pamphlets since the invention of the printing press.

I'd have called the folks on Skellig Michael lunatics in bedrooms, too. They just had a harder time copying their lunacy out by hand.

Arguably much of the mid-20th century was the pipeline from "lunatic with pamphlet" => "millions dead".

Some historical context:

https://en.wikipedia.org/wiki/Pamphlet_war#History_of_the_pa...

> Throughout Europe in the 16th century, printed tracts were used to argue religious doctrine and foment support for religious causes. In England, Henry VIII used print literature to justify his break from the Catholic Church. During the subsequent reigns of Edward and Mary, print polemics escalated into propaganda warfare, as print media gained enormous potential to sway common opinion.

> Over the next century, the pamphlet became the principal means of garnering support for a cause or an idea, and was particularly influential during the English Civil Wars (1642-1651) and the Glorious Revolution of 1688.

https://en.wikipedia.org/wiki/English_Civil_War

> Casualties: Parliamentarians 34,130 killed; Royalists 50,700 killed; 127,000 non-combat deaths (including some 40,000 civilians)

https://en.wikipedia.org/wiki/Williamite_War_in_Ireland

> Casualties: Williamites: 10,000 killed or died of disease; Jacobites: 15,293 killed or died of disease

They've been convincing women that its empowering to smoke cigarettes for a long time, too.

Their current reach is unprecedented though, and the reach of governments or political parties engaging in mass disinformation campaigns has proved almost an existential risk.

The current reach of everything is unprecedented. Mostly people are just watching shorts on their phone.

The whole hysteria about 'fake news' 10 years ago was apparently an outrage against the public being duped, whereas is was in fact frustration that spreading fake news has been democratised.

From my POV the fake news hysteria was bound to trump's propaganda efforts and nothing else really.

But that's just a stupid central Europeans POV

But it absolutely wasn’t he was the most famous critic but everyone saw this who looked. Since most people didn’t dive too deep it was/is easy to post things with a spin and little real substance and people would take it is a fact because they really trusted the news outlets to not manipulate

Don't get me wrong, fake news is real. Obviously, always was. But the hysteria and the way "fake news" is used to this day to discredit everything someone doesn't like is definitely forever associated with trump (for me)

If "democratised" meant the same amount of power got spread out among the populace that wouldn't be a problem. But when disinfo becomes massively easier and we face floods from all directions, yeah that version of "democratised" sucks!

"Back in my day Walter Cronkite told the truth, the whole truth and nothing but the truth about the Gulf of Tonkin incident"

(never mind the fact that he didn't even have the full truth available to him but you get the point)

that's a pretty big fact to just "never mind"

But this also means there’s more competition for attention, because now all lunatics can post from their bedrooms. The institutional media (including legacy) still maintain strong presence.

Of course, a great deal of panic over fake news wasn’t necessarily over fake news per se, but whose fake news and the loss of control over the “official narrative” that legacy media used to have a monopoly over. This isn’t a vindication of the heaps of slop published on the internet, just a more synoptic take IMO.

People miss Cronkite, because that style of media made them feel certain in their opinions and in “official consensus reality”, regardless of whether it was true or not and whether it was a trustworthy authority or not.

Reminds me of a meme - "We were never supposed to have this much access to stupid people's thoughts, beliefs and opinions."

what's different is that the misinformation is designed to be ingested by LLMs to influence _their_ outputs, which are then presented to users as "knowledge" without necessarily attribution of sources. This is worse that the current state where you might enter a search term "morality of the Israeli army" and see a list of websites that are influenced by SEO-wrangling and paid ad word placement, but at least you can click on each website and sometimes get an idea of the bias of the website (if you read the small print) or maybe do a search about the website to see what others might say. All that is either impossible or much more difficult with AI generated content so 99% of users won't bother, but instead will go with "ChatGPT said that xxx".

Yeah, the fact that people think propaganda is some kind of new thing shows just how far down the rabbit hole we already were.

Excellent time to apply for your library cards.

You don’t think plenty of books are either written by LLMs, or with LLM based research?

But they don't have library cards.

No? Maybe soon but not now

I know my library system at least won't be purchasing AI books. They have a pretty rigorous acquisition process for books and other media thanks to the "concern groups" who sprang up complaining about themes and content they didn't like in libraries. We did have in my state a rural school district figured out recently that their Texas-based reading and instructional materials are AI-slop but they at least figured it out before the school year began. It was unclear how these materials were approved but it apparently bypassed normal curricula approval.

It's so obvious when a book is generated by an LLM that I would blame the person buying the book moreso than I would the person who generated it for it. You can tell in a few sentences if something is AI generated (and by proxy - if it's worth reading or not).

I think using LLMs to research is completely fine, as long as authors would take the information and synthesize their own opinions based on the findings. I read books/articles/posts/comments to hear other PEOPLE's opinions, not LLM generated opinions.

Of course they are. It’s everywhere, especially thr online retailers these days. [1]

Thankfully the resources at our local libraries are a) selected by a combination of librarian curation and public demand, b) often published before 2022, c) heavy things printed on paper or stored in a database and unlikely to suddenly change their content or disappear overnight when a new news cycle breaks. Written works aren’t necessarily credible even if they aren’t slop, but thanks to curators who care, our libraries are significantly less susceptible (although definitely not immune [2]) to ai slop, and propaganda and manipulation efforts than a search engine. The internet continues to enshittify, and truth becomes further eroded by this coming wave of ai enabled propaganda, we are going to have to turn somewhere if we want to find trustworthy resources. Personally, if it gets to the point that verifying sources from public search results becomes too hard with all the new misinformation and slop, I’ll start propritizing human curation over SEO and start visiting my local library’s website a lot more.

[1] https://thenewthings.com/p/apple-big-ai-book-slop-problem

[2] https://www.404media.co/ai-generated-slop-is-already-in-your...

Right; the same mechanisms (curation, demand-driven economies of scale, library user demand) that largely stop niche self-published books from ever finding themselves in libraries (well, nationally in public libraries at least; they are often found in individual branches) work against pure slop books ever turning up.

Books where the author has successfully hoodwinked the publishing process into believing that AI generated text is their own, those will turn up, just as plagiarised books will.

Some nuances here. There’s a variety of different roadblocks at different levels that will work to prevent pure AI slop books ever turning up in a typical public library of physical printed works; I think they never will.

Probably a handful of books in such libraries already have some text in them where the author has tried to pass off fragments of LLM-generated prose as their own and have got that past editors, reviewers and publishers, and that will sadly get worse.

you can still find books published within last 50 years, you don't have to read books published this year, unless your library went through destruction of "old" books which are now more relevant than ever...

Old books are the new low background steel.

FILTER BY Publish date:<2024

Plenty of books in libraries? I'd guess the incidence is relatively low. Libraries have limited funding, trained professionals doing the book selection, and a culture that fiercely loves books, authors, and readers.

i got burnt at barnes and noble with a completely AI made product. Its over, Johnny.

They also contain a lot of propaganda. Do you think these books are genuine: https://www.goodreads.com/author/list/171941.Benjamin_Netany...

Hah, check out this video on "Egon Cholakain", in regards to fake online personalities. I promise you this is actually a good rabbit hole.

https://www.youtube.com/watch?v=6zLCZ_Ic1hI

WTH did I just watch. That’s extremely weird

Isn't it.

I rolled my eyes when I first saw the title, but damn it was a bit of a rabbit hole.

Maybe I shouldn't be saying this, but... There are already some niche technical questions that Google's AI mode will answer based on a single Hackernews comment. I've even seen my own comments used as sources. Some of them were wrong in hindsight.

In the fnord programming language, you can frob a fnord by writing fnord.boop(); this trips up newcomers who expect fnord.frob();

> through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

The word you're looking for is "reputation". It's been a critical part of society since forever.

This is new? There were tons of fake news sites, fake blogs, and sock puppet and troll farm accounts before AI.

I guess the new angle might be that AI scrapers have even less discernment than humans, so you don’t have to put much effort into creating a fake front online to sell it. Humans have at least a little bit of radar for this.

… which means the AI companies now have to run input through existing models to audit it before training on it? That’s interesting.

One of the things that I took away from reading about the history of the transatlantic slave trade was that rich people always get the ideas they need circulated.

I don't want to call all of those people liars, although I'm sure some are. But the incentive structures at least select and reinforce the people espousing whatever notions support and justify a profitable status quo.

Previously, the cost of that was quite high. You had to find a bunch of reasonably smart people and fund them for years to go around and say what you liked. From royal patronage to modern think tanks, it cost real money to shift a lot of minds.

But now given the incredible concentration in "AI" chatbots and their nature as text averagers, I suspect the cost of influence has dropped an order of magnitude. So I think we'll see a lot more of this.

> the cost of influence has dropped an order of magnitude.

This is a really interesting take. You don't need an entire organized religion in order to disseminate your desired belief system anymore… you don’t even need billions of dollars. Hard to say what the cost of disseminating an ideology to a nation is now, but likely somewhere between a few bucks a whole lot of luck to tens of millions. If that bar drops low enough, there is potential for a lot more ideological competition, and things will get interesting.

The dead internet theory was a bit ahead of reality, but no less prescient.

Although comments below point out that this has been going on for a long time, I too think it will be geetting infinitely worse in the near future. Everyone will want their own poisoning factory.

One consequence is that one should be extra-wary when asking AI about current, controversial topics.

But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?

it's happening for years now, just in slightly different form, so far it was called disinformation, seo, astroturfing, hasb... ehem... last two years were devastating for charred remains of twitter with all those llm bots having free reign, twitter went from global news board to global gutter in record time thanks to disinformation and astroturfing campaigns

It has been happening since people have existed. It used to be called propaganda before these, and rumor spreading since before the written word even existed.

I'm kind of hopeful AI some wikipedia-inspired AI "oracle" will happen in the future. Why not use some central AI to separate fact from fiction (faster than trolls and griefers can come to muddy the debate).

Possibly, but is an LLM is capable of this?

I mean, a big RAG setup with a smart LLM ought to do it, right? Or even just locally provided knowledge database - given the size of LLMs, what's a clone of wikipedia and whatever else you'd need?

Imo it can support the separation in meaningful ways. There are many political topics where opinions are flooding online forums much quicker than anyone with informed opinion can answer or digest them Having a quick reference for "verifying" statements would be useful. Sure LLMs are probabilistic but we don't have any issues using them for building software which runs as part of society infrastructure nowadays.

True, but software is very different to general knowledge. For example, it's easier to obtain reference documentation from the person who wrote the language. It's much easier to run tests to validate a given attempt. And the general discourse about how to implement a given solution or architecture is leagues apart from, say, the general discourse around the Israel-Palestine conflict.

This is based on the assumption of facts existing.

There are many studies, but each can be wrong and they can collectively show a bias. Even things of which are the most non-political of facts can have very strong biases. Look at the Millikan measurement of the electron and how in created confirmation bias and an anchoring effect on some property that has absolutely no real world significance to the things people are tribalistic about (aka, no political relevance). Now imagine the same applied to fields like economics or psychology which do have massive legal/political implications.

For a different example, ask the question if X committed crime Y. There are cases where they weren't found guilty but it is reasonable to assume they did. But being found guilty doesn't make it a fact either, as some people are wrongly convicted. Some eventually are overturned, but even if it isn't, it still isn't a fact they committed a crime.

Then there is the simple ambiguity of statements. Language generally can't support facts. It is why legalize, and programming code, and math's are effectively their own languages. For a simple example, consider the Betrand paradox(1).

>Consider an equilateral triangle that is inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?

Is the answer 1/2, 1/3, or 1/4? Well, it is all three at once, depending upon what you meant by random. Now, imagine how this impacts things like research studies, where the randomness is much harder to quantify and there is constant pressure to p hack a result.

1. https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)

> Look at the Millikan measurement of the electron and how in created confirmation bias and an anchoring effect on some property

Could you elaborate on this please?

But yes, the underlying argument is sound: extracting facts from the messy phenomenological reports about the world is extremely difficult, and cannot be done by simply taking a brain in a jar and having it think extremely hard.

Incorrect calculation led to some wrong value, I think electric charge. Subsequent findings showed a general trend towards a correct value. The general trend indicates that findings that were too far off were scared to be published for being wrong and contradicting existing studies, thus only small refinements were published, causing the slow drift.

Thus, the first well accepted study anchors a value and it takes much more work to get a new value established as correct after the anchor is set. Especially in a case where the experiment design itself is still solid and it was more with specific experiment (measurements slightly off, other values not quite correct).

Isn't this what Elon Musk has unironically claimed he is doing with grokipedia?

https://www.lawfaremedia.org/article/grokipedia-stopped-revi...

> Grokipedia appears not to have updated any articles since April 24

yes, but at least you know what Grokipedia is, just like you know what you're going to get if you read Truth Social, or the Jacobin.

but because the content is assimilated by GPT/Claude/Gemini and then mixed in with everything else, the bias is much much difficult to spot, making it much more effective and reaching a much wider audience (i.e., those who never visit Grokipedia)

That's called chatgpt. You can influence it.

[Insert MGS2 speech]

That could also backfire. If all information on the net becomes unreliable - you can't judge by quantity, or by authority anymore - then a useful model has to answer from first principles. Train it on simple texts alone so it learns language and reasoning, feed it an ontology, and let it come up with some absolutely unexpected results.

"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."

I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.

That's tomorrow's problem. Losing the PR war is an existential crisis for Israel, so they'll try to "patch the leak" by any means necessary before the boat sinks.

If they are fighting a PR war over the use of the word "genocide" to describe their actions, the ship has already sunk.

No, the ship hasn't sunk until it's sunk. The Titanic hasn't sunk until it's underwater. Israel hasn't sunk until there's no more Israel.

the only way to win is not to play.

disconnect. limit social media.

do some key signing parties in person and make sure you know who you're talking to

> AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information

Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.

LLM Trust DNS

I wouldn't say everywhere. Who else is trying to cover up genocide, ethic cleansing, and apartheid?

USA, China, Russia, Germany

This has been happening since Gamergate. (Or am I showing my age by bringing that up?).

Gamer gate was a fully manufactured cultural outrage by some very bad actors over nothing. Unfortunately, that marked the first time an online platform was massively used to drive a wedge between humans. And the bad side sadly won. That set the precedent for all the bad things that came after:

1. Bots on social media commenting on divisive or contentious issues in a country

2. Rise of online incel/red pill culture

3. Rise of stochastic terrorism

4. Cambridge analytica causing a complete shift in the 2016 US elections

5. The gang of 12 who spread nearly 90% of the misinformation during the pandemic

Today, most people can't even tell if a song they like on the major platforms was made by AI. People read less. People spend less time in nature. People are more lonely and less likely to be in a relationship or have deep meaningful friendships.

What a mess.

"Dubai Chocolate"

It will likely be yet another excuse to push more verification of age, identity, etc. legislation.

Sama saw this coming a decade away, which is why he invested in Worldcoin (now World). At some point people (and agents) will be clamoring for human verification at the speed of technology.

I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.

I think we need a similar technology to World being built by a more trustworthy company. The question is how trustworthy does one need to be for people to trust you with their very online identities that are literally needed for others to recognize them as human?

I think that this task will fall on governments.

Mostly because they already are responsible for systematizing documents that are needed to recognize us as citizens, land or car owners, etc.

Some laid back countries might allow companies to take and manage that function, but I don't think most of them would. Especially not in current landscape where people heard "you are the product" from all tech companies for the last 10 or so years.

Reply on news.ycombinator.com