Hacker News Reader

Comment by t_mann | Hacker News Reader

Assuming it'll be true, I'm not sure it has to be just a bad thing. We got used to treating videos/photos as the kind of objective evidence that they never were. It's not hard for an image to be both factually correct/unaltered, and yet giving a distorted view of reality. Cropping out parts, leaving out some bigger, non-visible contexts, such distortions can be really easy to miss, too. How faithful an image is depends a great deal on how it was put together and presented. So what we should really be asking ourselves when we see images is how much we trust the sources. A proliferation of AI fakes could make the need for such reflection more obvious.

View on HN

Two issues with that:

I) There will be a "danger window" where the fact that audio-visual evidence is meaningless has not yet fully registered with too many people.

II) It's perfectly possible that audio-visual lies are just inherently more convincing to the human psyche than written or spoken lies. We know instinctively that we can not trust what others say blindly, but we do instinctively trust our own eyes and ears. Note the etymology there: Blind trust is trust without seeing. When you see you don't need trust anymore.

For a classic example of this, our justice system tends to—in general—treat witness testimony as highly reliable. After all, the person was there. They saw it with their own eyes.

Of course we all know rationally that witness testimony is generally terrible. And yet we don’t actually seem to care about that when push comes to shove.

The "science" behind much criminal forensics is often laughably absurd. Even where there is valid theoretical foundation, in many cases the practice voids validity of much of the results.

And that's before we consider the psychology of witnesses, criminals, juries, judges, law enforcement, etc.

It's terrible, but it's reliably terrible. You always have exactly one layer of distortion between the evidence and reality: one human being. And that human being can be held accountable for willful misrepresentation.

(Hearsay, when there is more than one human in the chain, is treated differently.)

I think your right, the tricky part is always navigating these technological changes.

If you imagine life in colonies here around 1776, there was no photographs to doctor, no Zapruder film, or Nixon tapes, and yet newspapers existed and people lived in a relatively high trust society. People generally had faith in contracts, the law, and the government; but not a lot of ways, as an individual, to verify much at all.

I think eventually we’ll have to go back to that model of simply picking individuals and institutions to trust. Hopefully it’ll make people more selective.

> People generally had faith in contracts, the law, and the government; but not a lot of ways, as an individual, to verify much at all.

How do you know this was the case in 1776? Wasn’t there a revolution around that issue?

Slight non-sequester, but the DS9 episode ‘In The Pale Moonlight’ comes to mind here in this thread

WW3 or some other high-intensity conflict could be potentially launched by highly coordinated deepfakes. Imagine creating some videos offending an aggressive religion with many combatants or generating event videos that didn't happen and managing narrative as the uproar progresses etc.

None of that strikes me as meaningfully different than regular old propaganda.

Quantity and speed are qualities.

Getting hit with a snowball could hurt you but is most apt to annoy you. Getting hit with an avalanche is apt to kill you. But hey, they are both snow, right?

However, it's unclear what plays the role of mass in this analogy.

When it's one video of an angry mob killing bystanders, it's easy to dismiss.

When it's hundreds, from different viewpoints, it's much harder.

...so people are forging videos of mob justice? I'm not sure what your point is.

Computers are just fast calculators, right?

Cars are just faster horses, right?

Ships are just bigger canoes, right?

Photographs are just quicker made paintings, right?

Or… perhaps there is something different which makes practical what hasn’t been practical before? AI propaganda is the car of the propaganda horse of days past.

For all of human history, the only way to know how an event happened is to either have witnessed it yourself, or heard from someone else. It's only the last 100 years or so that we've been able to record exactly what happened as audio, video, or photography. And still, it's possible to doctor any of these. That's not to mention that you can craft excellent propaganda even by using this "real" recorded media, just by cherrypicking and presenting it in a certain context. I don't think AI generated media is going to a massive sea change in misinformation.

edit: downvoters should reply instead

I'm not denying that and I am concerned about that possibility. The thing is, I don't think you need deep or any fakes at all for that. What it takes are people willing to believe lies and consume hatred (fwiw, WWII was based on huge lies that didn't need much forgery). The more people are questioning hatred they're being fed, the better of a chance we have.

People believing lies or not isn’t the problem. The problem is that lie to truth ratio, admittedly not low enough even today, will rapidly approach 1. Finding anything trustworthy will be impractical, so people will stop trying.

This is very scary. Being able to tell what’s true underpins western democracies. With everyone having plausible deniability from everything we’re going to have to… change things.

People to people information from direct witnesses will go up in value because faking physical environments is expensive, while untrusted or unverified sources will become pretty much useless. In person settings will matter more again - back to older models in a way.

I don't think that needs to be true, though. Let's go with the UK since the article mentions the British PM, there are trustable sources like BBC and other media whose journalists can verify events on scene can work with trusted photographers, have contacts with whom to cross-check claims,...

Yeah, some hoaxes will make it through the process, but some will be rectified later, and I don't see the ratio approaching 1. And those media are not hard to find. Taking the example from the article, there's no sustained uncertainty about whether the PM endorses a get-rich-quick scheme, and I don't see that coming, no matter how good the fakes become.

>> trustable sources like BBC and other media whose journalists can verify events on scene can work with trusted photographers, have contacts with whom to cross-check claims

They could, but do they? Evidence says no - they are perfectly happen to run with lies if it fits their narrative - and then weeks later, tucked away someplace, issue a retraction - after the damage is done.

The BBC has an agenda, FOX news has an agenda, CNN has an agenda, MSNBC has an agenda, PBS has an agenda - they are all perfectly willing to lie to gather clicks or eyeballs - not sure AI is going to meaningfully change anything in that regard.

Having an agenda isn't necessarily bad in and of itself. Wanting to maximize eyeballs or clicks is an incentive problem that can be addressed.

> there are trustable sources like BBC

The BBC may have been trustworthy at some point, but that's a thing of the past.

Here's a well known example of them purposely deceiving their viewers in meaningful ways in recent history (sic).

Fortunately they were called out for it:

https://www.youtube.com/watch?v=904xN8YIQd4

Now do Blue Peter and Shaun the Sheep.

Not sure what you're referring to?

Ahhh, so that's where we get a Ministry of Truth... ;)

Wait till you realize 1984 was the optimistic take…

Ever seen the movie Wag the Dog?

Lots of wars started that way before AI: Spanish-American ('remember the Maine!'), Vietnam (Gulf of Tonkin), Iraq (nuclear weapons program), etc.

AI would seem to make it worse. I think it will also get worse because warmongers can automate propaganda and, with social media, reach people directly.

I think misinformation is already out there. I can only guess what extent to which common knowledge on world events is false, but it's not zero. There is an interesting, though not that academically rigorous body of internet analysis on things like Pallywood and the Charles Jaco bluescreen video from the Gulf War that are a good starting point.

Agree. It will be like text, which has always been trivial to conjure or alter.

It will be a blip in history that that there was a period in which producing a particular image required its contents to have existed in reality.

What we need going forward is a way to know provenance, where an image came from and what edits have been made. Then people can trust particular sources, as they do with text.

The Content Authenticity Initiative is working on that. https://contentauthenticity.org

This use case is a compelling use for blockchains https://research.adobe.com/news/content-authenticity-and-ima...

yes agree - this was also discussed ten years ago in some math circles. So, trying to agree but note that these are problems that different projects have taken on over a long time, also.

> It's not hard for an image to be both factually correct/unaltered, and yet giving a distorted view of reality. Cropping out parts, leaving out some bigger, non-visible contexts, such distortions can be really easy to miss, too.

In some contexts, that's true, especially if the taker has full autonomy on how the photo/video is taken, and how the story is told.

But there are other contexts (fixed cameras like CCTV, videotaped performances for auditions, a footage that includes the complete process of a car crash, etc.) in which this is not a thing. Faking it would require editing visible content, which is made way easier than before by machine learning, and ultimately reduced their value as evidence.

I'm not sure I particularly want to go back to the time when evidence like these didn't exist.

I guess you’d have to have a rigorous chain of evidence really.

It's not just about cropping out relevant parts of a video (which can absolutely still happen with CCTV, btw). Like I said, there might be some bigger context that wouldn't even be visible if you had a full 3D rendering of the whole scene, like information that some of the actors had that no bystander would have and that completely alters the interpretation of the actions. My point was that fully objective video evidence was never a thing, and this is a good opportunity to reflect on that.

When teenagers are making sexually explicit deepfakes of classmates, that's objectively a bad thing, because it is extremely damaging. People are using it to blackmail and extort too. Consent, ethics and law should be at the forefront of the conversation regarding AI, not whether we trust the contents of an image.

Laws are used by humans to blackmail, extort, and damage.

When you have an idea how to cancel out human emotions fucking up the high minded poetry, platitudes, euphemisms, coded language society relies on to function let us know.

Studies show that we kill each other in random acts of violence; spurned lovers, and the boss who fired the office shooter without understanding how close to the edge they are; we kill each other over those things at the same rate we always have.

Our language just implants mind viruses that obfuscate the mechanics of reality. Something we evolved into intuiting (enough heat, water, food) before language. Language just gets in the way. It creates hallucinated problems that only exist given the language they exist.

AI isn’t a problem if we can’t understand it. Let it emit whatever Anglo-gibberish it wants. Just unplug it when it mouths off.

AI won’t be the problem people fear it will be because most know it’s just a dumb machine under the hood. They aren’t lost to a puerile hallucination like IT narcissists who can’t cope with reality as-is.

But if everyone knows AI exists, the negative points kind of go away, because that also gives everyone plausible deniability. How are you going to blackmail me with sexual images when everyone knows that AI can easily create that? I think that actually makes the AI objectively a good thing, because it prevents blackmail with real images, too

No, the negative points don't go away whatsoever. Normalizing the sexualization of minors is never not going to be a problem.

If you take a photo of a minor and then use that photo to generate CSAM, even if it is "fake", that's not in any way OK on numerous levels. There's the legal, moral and ethical aspects of it, and then there's the issue of consent. Can you imagine people normalizing it for themselves and no longer being attracted to adults? Fuck that rabbithole. We already know about the negative side effects of porn addiction in terms of real sex.

These things are damaging by their very existence. Can you imagine the bullying and rumours that are possible now because of technology like this being so readily available? It doesn't matter if it's real or not. People used to crappily MS Paint/Photoshop heads of female minors onto "bikini babes" and pornstars when I was at high school, that didn't make it OK, even if it didn't look real in any way whatsoever. It has a massively negative effect on the person it is of.

It doesn't prevent anything, quite the contrary, it enables everything. Look at the proliferation of fake news already in the past few years, people know they shouldn't believe everything they read on the internet, but people parrot internet fake news constantly. People act on it. People get killed over it. People kill themselves over it.

> and then there's the issue of consent.

You mentioned this twice within two messages. What exactly is your agenda here? Consent is entirely irrelevant in the context of CSAM, since the consent of a child does not authorize anybody to produce pornography of them.

I don't support the idea of deepfakes, yet despite agreeing with most else you said I'm more disturbed that you're in favor of normalizing the idea that anybody exercising their right to expression is beholden to the feelings of their subject.

You do realize what chilling effect that has on basic discourse? Consider the absurdity of someone photoshopping foreskins back onto Israeli dudes' nudes in protest of the (non-consensual!) RIC practices of the Jews.

>So what we should really be asking ourselves when we see images is how much we trust the sources.

I mean, that's what you want as a best outcome, unfortunately I don't see that is going to be the outcome any time soon.

https://en.wikipedia.org/wiki/Hyperreality

Further embracing of the hyperreal by society at large, coupled with authoritarian governments using manipulation tactics like the firehose of falsehood will continue to erode trust in social institutions and disengagement by the masses.

We cannot compare an out of context picture to an artificial video created with just a prompt.

First of all, video is accepted as evidence in almost all countries judicial systems. Proliferating fake video hard to distinguish is going to be a nightmare on that front.

Besides judicial systems, the reputation of countless people has been permanently ruined/damaged by some "leaked video" that went trough some kind of "trial by media". Even if a week later the video is confirmed to be fake, is too late. Once it's out there, the damage is done. Imagine these video to be created by anyone, from edgy teenagers to governments or political parties.

Secondly, there's a difference between an out of context picture that can be abused by bad faith actor and the ability to very easily generate countless misleading pictures AND videos by these actors. Disinformation can and will multiply.

Average social media user had a very difficult time recognizing fake news, ragebait content, etc even before the AI revolution. Things can only get worse when fake videos will start to spread everywhere. Imagine bad faith actors weaponizing VIPs, actors, tv personalities, etc to send misinformation to the people that do not have the tools to defend themselves. Imagine your average facebook user scrolling trough the feed and finding a fake video about some political misinformation. It's the same thing that already happened with fake news, but an order of magnitude bigger.

Reply on news.ycombinator.com