New UNESCO report warns that Generative AI threatens Holocaust memory | Hacker News Reader

New UNESCO report warns that Generative AI threatens Holocaust memory

unesco.org

36 pointst0bia_s49 comments

Asking a large language model about facts is like asking an image model for photos. That's not what it's for, or what it does.

It's a bit weird that we're building tools that make the large scale pollution of the information space approach a marginal cost of zero. I get that it's a bit useful for artists, and for helping dyslexics write emails or whatever, but it's highly doubtful this will be worth it.

We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?

> That's not what it's for, or what it does.

Then again, it has been marketed as such, and does not tell people that it's lying. Image models don't have that problem because they never claimed to answer truthfully.

I can’t wait to see what happens as the term info pollution gains transaction.

We already know that generative performance degrades when they learn from generated data (don’t remember the exact findings).

I think even calling it lying is too kind. It just has no relation to truth.

> why would we insert this not-even-stupid noise into the information space?

Because US tech would "collapse" otherwise.

Big Tech's stock price is entirely propped up by AI hype. Metaverse didn't go anywhere so they need a new bubble.

US tech startups are even worse. The VC culture is completely incapable of building useful companies, AI is the last hype. Just a rat-race to grab the last funding and get bought out before the end of ZIRP kills the startup scene entirely.

There's no real demand for any of this shit. Even the web spam renaissance is mostly hustle-bros losing money. The writing is already on the wall with investment slowing down, and AI product after AI product bombing.

Hey, collapse is unfair. Correct is better.

They will go from trillions to slightly fewer trillions.

Some will go to jail.

Why do you think this? Nobody will go to jail. Maybe some paltry fines will be paid and some CEOs ousted, and that's about it.

I mean, it's likely that _some_ fraud is going on, and that tends to become more apparent as the money runs out (for instance see SBF, or the various Ponzi schemes that ran out of road during the financial crisis), so when the AI bubble bursts, yeah, probably a few people will go to jail. But it doesn't seem like a significant factor.

Someone will oversell this. There is too much greed.

> large scale pollution of the information space

It's incredible, this attention economy. Google anything and you'll have word-vomit articles that don't answer your question - something like "Actress XYZ boyfriend" and you get:

> XYZ, XYZ has emerged as a rising star, captivating audiences with her talent and charm. While her professional life is an open book, her personal life remains a subject of intrigue. Join us as we delve into the untold love story of XYZ, exploring her journey from boyfriend to husband.

With even a wrong headline photo of the actress, and zero information about her boyfriend/husband. At least the above paragraph is still coherent.

Why? Because they can embed ads on the webpage, so they can earn money from Google...

Can we admit that this was the state of the Internet content before LLMs and their introduction just accelerated and existing problem? I feel like the conversation around the information pollution is turning into "AI did it" and we need to take a lot more responsibility for letting it happen in the first place.

I agree, it wasn't AI, or well, much stupider AI / bots that put words together (we had that before GPT - Were they AI? Just like a high-end Nokia was a "smartphone" before iPhone came along...).

But this new AI is making it simpler and will probably accelerate the bullshit pollution.

On that note, today I got 2 Instagram video ads of 2 different national politicians saying they're offering something to "just 50 people", and it was to me obviously deepfake videos. But there are vast numbers of people who will fall for this scam. But hey, at least these ads pays for Zuck's yachts (yeah he has two, 1 for him to enjoy and 1 behind it that carries supplies/sleeps guests)...

https://www.autoevolution.com/news/mark-zuckerberg-buys-russ...

Who is “we” here again? I’m not sure why even google engineers would want to take credit for management decisions to screw up search. And it’s deliberate because bad search results means more search! The approach I usually recommend for protest is.. use google exclusively to search for ChatGPT, then leave google. Since they haven’t heard users saying for the last decade that we pretty much prefer classic google search circa 2010 to the garbage engagement and growth hacking monster that they have become… perhaps they’ll hear this version of protest better.

Sure, but tipping points matter. If the amount of AI slop causes people's behaviors to change, then it's still interesting/concerning.

We're already on the verge of losing reliable search engines. Will this bring back 90's-style Yahoo sites?

Yea, Google/Ads is the problem for this specific case. Just that Google didn't fight it and Google also was pivotal in creating the modern LLM/AI tech. They are the people who are destroying the internet from many fronts.

It's being adopted by organisations who should be guarding data integrity, preserve sources of truth, and pay utmost attention to PII or corporate and state secrets. I have to congratulate operators of genAI on their marketing prowess, but the results are really worrying and it's only going to get worse. We run real risk of serious disruption to the lives of people wrongly banned from access to shops and places, charged with offences they did not commit, their credit scores ruined and no straight path to rectify the situation. The tools that ought to be only used for academic research have been given to people with no background allowing them to understand how those tools work by cynical people looking to make a quick buck from chaos others will unleash on employees, patients, citizens. Meanwhile the AI operators have all the raw data and can do anything with it.

> We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?

Because people will make money from it and the political will to resist such a thing does not exist. Besides the cat is already out of the bag.

What the actual fuck?

People think the earth is flat—does Generative AI threaten our core understanding of astrophysics?

If flat-earthers had the numbers and motivation of revisionist neo-nazis, they might very well be able to decrease popular (not academic) understanding of astrophysics.

You don’t need ChatGPT to rewrite WWII history. I choke every time I read “pacifist Japan militarize in response to expansionist neighbor” in American media.

This article gives me "What an odd thing to say" vibes. I guess the focus on antisemitism and holocaust memory is because

  Published in partnership with the World Jewish Congress

Which is still an odd choice when it comes to AI. I think this article should be more framed as "World Jewish Congress considers AI dangerous".

I mean.. everyone who can string two words together does...

Absolutely. This can be applied to anything and everything in its output, and this is why I take a very dim view of how quickly this has been trotted out, yet how little understanding the average person has of the maths behind it.

You should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English.

Even asking it questions about COVID can yield some nonsensical responses in regards to politics.

Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.

>especially if you're talking politics beyond the USA or Canada

I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries.

I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information.

Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs.

This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.

The extra danger is when we don't know it comes from an LLM. If we think a human wrote it, we're less skeptical.

I've had a few less tech-savvy friends tell me dubious things this past year and asked where they heard it—usually "this neat app I found called ChatGPT". eyeroll

History rewriting has been going on for awhile now. No need for LLM and "malicious actors to seed disinformation and hate-fueled narratives".

I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Which UNESCO demonstrates.

The problem is the Holocaust was the first time industrial techniques were applied to killing large groups of citizens. The unique evil is the tools, bureaucracy and mundanity of mass murder. The horror was kids in the wrong family tree being shipped off to well engineered gas chambers - not the details of ideologies and personalities. We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use.

> The Historical Figures App allowed users to chat with prominent Nazis such as Adolf Hitler and Joseph Goebbels, and falsely claimed that individuals such as Goebbels were not intentionally involved in the Holocaust and had tried to prevent violence against Jews.

I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).

> I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness.

Misinformation is a real, serious threat.

> I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).

I was born and raised in a city where one of the largest concentration camps was built and run by Germans. Members of my family were murdered by Germans following orders of Hitler and told that it was necessary to do so by Goebbels. Then Russians came and repurposed the camp to imprison people who fought Germans and ruled the place for close to 50 years by proxies. I have no desire to see genAI or "works of art" showing Hitler or Goebbels as innocent or even likeable characters. Holocaust was a real thing. The fact that other genocides happened before and after does not change the reality or the legacy of it. Hitler puppets saying he was innocent are not just denying the facts, they are prepping people for another round of persecutions, and not just Jews. If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.

> If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.

Just taking a feel of the political winds but - if I'm sent to a camp in my lifetime it will probably be by people calling me a Nazi. Assuming an internal group does it. It is a favoured insult right now. Although the odds are still comfortably low as far as I can tell.

And this is the issue. Privileging a hate-cult of Hitler and Goebbels is a distraction from the actual problems which are policies, capabilities and incentives. It doesn't seem to be an effective barrier to hate - in fact philosophically it encourages it over more useful focuses.

Calling someone a Nazi is the the favourite thing Nazi copycats do. Reminding of what Hitler and lots of Germans (and Austrians) did is not practicing cultish behaviour, it is a vaccine against those who want a repeat of what had happened.

Yes, but no. People who are unable to morally reason will go "I'm not a Nazi so what i do is justified" and then do the same bad shit all over again. Ignoring the lessons about Hitler and Goebbels because they are convinced they are better. Very focused on fighting the baddies that they never question if they are just another sort of baddies themselves.

> We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use.

I don't think that's even the lesson. That's too detailed. That's not what happened in Cambodia or China. It's not what happened in Indonesia. It's not what happened in the Ottoman Empire.

The problem is the same everyhere: how_wrong_you_are_about_facts * power * conviction, these are the three variables that multiply together to produce mass killings. Mao didn't have an out group (well.. that wasn't the big death toll at least). Mao didn't have registers. He just made people destroy the farming and millions starved.

We have to care about truth. Then we have to limit power. Then we have to be doubtful about what we know so we make small changes and slowly.

Reply on news.ycombinator.com