Many AI researchers think fakes will become undetectable
economist.com
economist.com
Getting hit with a snowball could hurt you but is most apt to annoy you. Getting hit with an avalanche is apt to kill you. But hey, they are both snow, right?
When it's hundreds, from different viewpoints, it's much harder.
Cars are just faster horses, right?
Ships are just bigger canoes, right?
Photographs are just quicker made paintings, right?
Or… perhaps there is something different which makes practical what hasn’t been practical before? AI propaganda is the car of the propaganda horse of days past.
edit: downvoters should reply instead
This is very scary. Being able to tell what’s true underpins western democracies. With everyone having plausible deniability from everything we’re going to have to… change things.
Yeah, some hoaxes will make it through the process, but some will be rectified later, and I don't see the ratio approaching 1. And those media are not hard to find. Taking the example from the article, there's no sustained uncertainty about whether the PM endorses a get-rich-quick scheme, and I don't see that coming, no matter how good the fakes become.
They could, but do they? Evidence says no - they are perfectly happen to run with lies if it fits their narrative - and then weeks later, tucked away someplace, issue a retraction - after the damage is done.
The BBC has an agenda, FOX news has an agenda, CNN has an agenda, MSNBC has an agenda, PBS has an agenda - they are all perfectly willing to lie to gather clicks or eyeballs - not sure AI is going to meaningfully change anything in that regard.
The BBC may have been trustworthy at some point, but that's a thing of the past.
Here's a well known example of them purposely deceiving their viewers in meaningful ways in recent history (sic).
Fortunately they were called out for it:
AI would seem to make it worse. I think it will also get worse because warmongers can automate propaganda and, with social media, reach people directly.
I mean, that's what you want as a best outcome, unfortunately I don't see that is going to be the outcome any time soon.
https://en.wikipedia.org/wiki/Hyperreality
Further embracing of the hyperreal by society at large, coupled with authoritarian governments using manipulation tactics like the firehose of falsehood will continue to erode trust in social institutions and disengagement by the masses.
I) There will be a "danger window" where the fact that audio-visual evidence is meaningless has not yet fully registered with too many people.
II) It's perfectly possible that audio-visual lies are just inherently more convincing to the human psyche than written or spoken lies. We know instinctively that we can not trust what others say blindly, but we do instinctively trust our own eyes and ears. Note the etymology there: Blind trust is trust without seeing. When you see you don't need trust anymore.
Of course we all know rationally that witness testimony is generally terrible. And yet we don’t actually seem to care about that when push comes to shove.
And that's before we consider the psychology of witnesses, criminals, juries, judges, law enforcement, etc.
(Hearsay, when there is more than one human in the chain, is treated differently.)
If you imagine life in colonies here around 1776, there was no photographs to doctor, no Zapruder film, or Nixon tapes, and yet newspapers existed and people lived in a relatively high trust society. People generally had faith in contracts, the law, and the government; but not a lot of ways, as an individual, to verify much at all.
I think eventually we’ll have to go back to that model of simply picking individuals and institutions to trust. Hopefully it’ll make people more selective.
How do you know this was the case in 1776? Wasn’t there a revolution around that issue?
It will be a blip in history that that there was a period in which producing a particular image required its contents to have existed in reality.
What we need going forward is a way to know provenance, where an image came from and what edits have been made. Then people can trust particular sources, as they do with text.
The Content Authenticity Initiative is working on that. https://contentauthenticity.org
This use case is a compelling use for blockchains https://research.adobe.com/news/content-authenticity-and-ima...
In some contexts, that's true, especially if the taker has full autonomy on how the photo/video is taken, and how the story is told.
But there are other contexts (fixed cameras like CCTV, videotaped performances for auditions, a footage that includes the complete process of a car crash, etc.) in which this is not a thing. Faking it would require editing visible content, which is made way easier than before by machine learning, and ultimately reduced their value as evidence.
I'm not sure I particularly want to go back to the time when evidence like these didn't exist.
When you have an idea how to cancel out human emotions fucking up the high minded poetry, platitudes, euphemisms, coded language society relies on to function let us know.
Studies show that we kill each other in random acts of violence; spurned lovers, and the boss who fired the office shooter without understanding how close to the edge they are; we kill each other over those things at the same rate we always have.
Our language just implants mind viruses that obfuscate the mechanics of reality. Something we evolved into intuiting (enough heat, water, food) before language. Language just gets in the way. It creates hallucinated problems that only exist given the language they exist.
AI isn’t a problem if we can’t understand it. Let it emit whatever Anglo-gibberish it wants. Just unplug it when it mouths off.
AI won’t be the problem people fear it will be because most know it’s just a dumb machine under the hood. They aren’t lost to a puerile hallucination like IT narcissists who can’t cope with reality as-is.
If you take a photo of a minor and then use that photo to generate CSAM, even if it is "fake", that's not in any way OK on numerous levels. There's the legal, moral and ethical aspects of it, and then there's the issue of consent. Can you imagine people normalizing it for themselves and no longer being attracted to adults? Fuck that rabbithole. We already know about the negative side effects of porn addiction in terms of real sex.
These things are damaging by their very existence. Can you imagine the bullying and rumours that are possible now because of technology like this being so readily available? It doesn't matter if it's real or not. People used to crappily MS Paint/Photoshop heads of female minors onto "bikini babes" and pornstars when I was at high school, that didn't make it OK, even if it didn't look real in any way whatsoever. It has a massively negative effect on the person it is of.
It doesn't prevent anything, quite the contrary, it enables everything. Look at the proliferation of fake news already in the past few years, people know they shouldn't believe everything they read on the internet, but people parrot internet fake news constantly. People act on it. People get killed over it. People kill themselves over it.
You mentioned this twice within two messages. What exactly is your agenda here? Consent is entirely irrelevant in the context of CSAM, since the consent of a child does not authorize anybody to produce pornography of them.
I don't support the idea of deepfakes, yet despite agreeing with most else you said I'm more disturbed that you're in favor of normalizing the idea that anybody exercising their right to expression is beholden to the feelings of their subject.
You do realize what chilling effect that has on basic discourse? Consider the absurdity of someone photoshopping foreskins back onto Israeli dudes' nudes in protest of the (non-consensual!) RIC practices of the Jews.
First of all, video is accepted as evidence in almost all countries judicial systems. Proliferating fake video hard to distinguish is going to be a nightmare on that front.
Besides judicial systems, the reputation of countless people has been permanently ruined/damaged by some "leaked video" that went trough some kind of "trial by media". Even if a week later the video is confirmed to be fake, is too late. Once it's out there, the damage is done. Imagine these video to be created by anyone, from edgy teenagers to governments or political parties.
Secondly, there's a difference between an out of context picture that can be abused by bad faith actor and the ability to very easily generate countless misleading pictures AND videos by these actors. Disinformation can and will multiply.
Average social media user had a very difficult time recognizing fake news, ragebait content, etc even before the AI revolution. Things can only get worse when fake videos will start to spread everywhere. Imagine bad faith actors weaponizing VIPs, actors, tv personalities, etc to send misinformation to the people that do not have the tools to defend themselves. Imagine your average facebook user scrolling trough the feed and finding a fake video about some political misinformation. It's the same thing that already happened with fake news, but an order of magnitude bigger.
When you see _anything_ on the internet these days your first instinct should be to ask 'did any of this actually happen?'
If you think about it, it's sort of like a real life GAN algorithm.
And then sign the images in hardware so that you can prove an image came from a specific camera and hasn’t been altered?
A much harder to break system is one uses a secret only exists on each individual phone + the generic algorithm that iphones use, but this comes back to individuals being traced.
There is actually a way to prevent individuals from being traced while having a strong membership proof. This can be achieved by issuing signatures on a relative generator created in an exponentiation mix together with a set of output pseudonyms. One can picture it as strands braided together where it is hard to trace the link between the input and output strands, while it is very easy to tell that such a link exists.
More on that can be found on:
However if it came out that manufacturers were secretly storing a "fingerprint" of the noise profile of each image sensor and sharing it with the government, I'm afraid I wouldn't be entirely surprised
Remember https://en.wikipedia.org/wiki/Machine_Identification_Code
This could work for other manipulation. Change the colors in the image. Then if someone wants proof, offer the original and describe the transformation.
https://www.dpreview.com/news/9855773515/sony-associated-pre...
Here's some code that can be played with that relates to steganography in images: https://github.com/RoliSoft/Steganography/tree/master
It will be people who need to learn how to sign and encrypt and manage their keys.
I don't believe it will be possible in practice to "granularize" trust like that. You trust a person or an institution (a person is already more ideal though). Going even further will not work in practice.
Photo/video signing in hardware might actually make things worse by creating a false sense of security. It's better for people to question everything they see rather than trust everything they see.
A hardware key proves which camera it came from, and a blockchain hash proves at least when the camera first connected to the internet after taking the photo. So, if anyone tries to claim your photo as their own, you need only point to the blockchain to prove it's actually yours. It also proves you didn't use any beauty filters outside the camera hardware security bubble.
https://news.adobe.com/news/news-details/2022/Adobe-Partners...
I predict it's going to go mainstream in a new iPhone model eventually. Consumers will become sold on the idea of authenticity. They are sick of fakes and filters, but didn't yet have any way to prove something is real.
But this is essentially the same problem statement as DRM: provide the user with some cryptographic material, yet limit how they can use it and prevent them from just dumping it. Logically, there will always be a hole. Practically, you can make it very annoying to do, beyond the ability of the average consumer [1], but someone will probably manage, and given the scenarios we often talk about (e.g. state-backed disinformation), there will be plenty of resources spent on doing that. The payoff will cover the cost
Paradoxically, one could argue that a "95% trustworthy" system could actually do net harm. The higher people's trust builds in it, the greater the fall damage if someone manages to secretly subvert it, and use it on just the right bit of disinfo at the right time (contrasting my footnote about DRM)
[1] Hence why claims that DRM is a complete failure miss the point a bit - it's not needed to stop everyone. Perhaps we can crack some given DRM system, but the fact that you even have to download a new program is enough to stop a massive amount of consumers from bothering
Give a valid reason to do otherwise. I can't think of one, unless you want to mislead people. Corporations can still have their chatbot customer service, and add that tagline. The only reason not to is so their customers think they are talking to a human.
Also, it's not clear that an individual can do it nearly as well as a well-resourced company with scientists and lots of processing power.
The way we detect them is if this becomes a problem, the authentic entity will speak against it and the society will rectify. Just as these fakes are a non-problem in practice, AI fakes will be a non-problem too.
Society is already speaking against stolen content and fakes produced using it. How are we rectifying this?
The societal implications of undetectable fakes are off the charts.
What we will need today is to start signing things digitally in the same way, which I think will require very good, decentralized pki, unlike what we have with email.
Essentially, different pki should work like dns, and find the pki for someone, and cache it., so that your signature works in china and the moon. Updating it should work the same way, the old one should always work and new ones should have the old pub keys appended to them so any public key can be used to verify all past signatures.
[1] https://securelist.com/operation-triangulation-the-last-hard...
The point is that tracking the provenance of digital items is certainly going to be more important in the future. A signature by Apple silicon is one piece of evidence. It could also be signed by the photographer’s identity. The account who uploads it gives another piece of evidence.
Unfortunately, we should probably not trust anonymous content in the future.
This seems like a copyright problem to me, a legal one, not a technological one
Guess a few decades of big tech enshittification will do that to the psyche, lol.
I have wondered if this would happen to CCTV / Ring / security cams for a while, but yet to have seen it.
DMA attacks anyone?
The comments in this article have deeply studied the method you're suggesting.
State actors, for example, don't need to use public models for their generators anyways, so it's not like non-state actors would have true-positive images to train models on to counter said actors' models.
Using AI image generation for misinformation is by far the vast minority of its use.
And if the output is stylized, it's going to be much more obvious if it has come from an AI model, because a human is going to have a much harder time reproducing a specific ML model's style on-demand (e.g. if their art teacher asks them to sketch a face in the same style, to prove their homework wasn't faked).
> Just as machines can be trained to reliably identify cats, or cancerous tumours on medical scans, they can also be trained to differentiate between real images and ai-generated ones.
But the conclusion the researchers made was detection at some point wont be reliable and fakes will win
I'm talking about a detection model that only detects images produced by one generator/source model, so rather than a one-to-many process of having one tool check an image for many potential source generators' artifacts, you'd have many individually-trained detection models, each being run against a single image/text block.
Who is the threat in your scenario?
Where most ordinary people run into trouble is decidedly very well down that particular foodchain.
Is anti-virus really going to fix this?