AI fakes duel over impeachment of Vice-President in Phillipines
factcheck.afp.com
factcheck.afp.com
If these same ideas were expressed by Vtubers (virtual youtubers, anime-like filters for people who want to do to-camera video but are shy or protective of their privacy), it would not be troubling, as everyone understands that fictionalized characters are a form of puppetry an can focus on the content of the argument.
But using generative video to simulate ordinary people expressing those ideas is a way of hijacking people's neural responses. Just pick the demographic you wish to micro-target (young/middle/old, working/middle/upper class, etc. etc. etc.) and generate attractive-looking exemplars for messages you want to promote and ugly-looking exemplars for those you wish to discredit.
The speed with which you’ll be able to create and disseminate propaganda is mind blowing. Imagine what a 3 letter agency could do with their level of resources.
Expected reaction: every camera manufacturer will embed chips that hold a private key used to sign and/or watermark photos and videos, thus attesting that the raw footage came from a real camera.
Now it only remains to solve the analog hole problem.
Now you have another problem -- a signature is unique to a device, presumably, so you've got the "I can authenticate that this camera took this photo" problem, which is great if you are validating a press agency's photographs but TERRIBLE if you are covering a protest or human rights violation.
I don't think the CIA will have any problems
- Watermarking is nearly useless as a way of conveying that information, either visibly distorting the image or being sensitive to all manner of normal alterations like cropping, lightness adjustments, and screenshotting.
- New file formats are hard to push to a wide enough audience for this to have the desired effect. If half the real images you see aren't signed, ignoring the signature becomes second-nature.
- Hardware keys can always be extracted in O(N) for an N-bit key. The constant factor is large, but not enough to deter a well-funded adversary. The ability to convincingly fake, e.g., video proof you weren't at a crime scene would be valuable in a hurry. I don't know the limits, but it's more than the 2-10 million dollars you need to extract a key.
- You mentioned the analog hole problem, but that's also very real. If the sensor is engineered as a separate unit, it's trivial to sign whatever data you want. That's hard to work around because camera sensors are big and crude, so integration a non-removable crypto enclave onto one is already a nontrivial engineering challenge.
- If this doesn't function something like TLS with certificate transparency logs and chains of trust then one compromised key from any manufacturer kills the whole thing. Would the US even trust Chinese-signed images? Vice versa? The government you obey has a lot of power to steal that secret without the outside world knowing.
- Even if you do have CT logs and trust the company publishing to them to not publish compromised certs, a breach is much worse than for something like TLS. People's devices are effectively just bricked (going back to that 3rd point -- if all the images you personally take aren't appropriately signed, will a lack of signing seem like a big deal?). If you can update the secure enclave then an adversary can too, and if updating tries to protect itself by, e.g., only sending signed bytecode then you still have the problem that the upstream key is (potentially) compromised.
- Everyone's current devices are immediately obsolete, which will kill adoption. If you grandfather the idea in, there's still a period of years where people get used to real images not being signed, and you still have a ton of wasted money and resources that'll get pushback.
Etc. It's really not an easy problem.
What does it even mean that hardware keys are extractable in O(N) time? If there's some reasonable multiple of N where you can figure out a key, your cryptosystem is broken, physical or not.
It's also very straightforward to attach metadata to media and wouldn't take a format change.
Can you expand on that a bit? Wikipedia's coverage on this seems mostly historical and copy protection focused.
There's a fair bit of public information out there on theoretical techniques (e.g. https://www.intechopen.com/chapters/71851), but I'm not deeply familiar with what's actually used in industry, for example by imatag.
One critique I can lodge against this is that to me it reads like the security model in this scheme trusts the venue to not tamper with the projection equipment. This may not map well to everyday camera recording situations, where the camera owner / operator may have a vested interest and capability in tampering with the camera itself.
The point isn't to completely, 100% close the analog hole here (and at least one piracy group seems to have leaks of raw movie data despite all of this security), but it's effective at making compliance the least costly option for almost everyone involved worldwide. It's one of the major reasons we've seen a shift by pirate groups to preproduction leaks or alternative methods.
The problem isn't the cryptography. It's the hard part of cryptography, key management. In this case, the key is in the hands of the attacker, written down for all to see, and the only real defense is that it's written in a tiny "font" next to a bunch of other data.
You're right, that's a broken model. That's exactly what I'm saying. It's kind of like the current craze of using a phone number on login for "security" -- doing so increases the cost to an attacker by a fixed amount ($10-$15 or so) and is only reasonable for security (as opposed to selling out to data brokers) if the value to the attacker of the account in question is less than that. The security doesn't come from actual impossibility guarantees but by trying to stack a big enough constant factor onto the thing that nobody actually cares to break it.
> metadata vs file format change
Ah, exif and all the other slightly underspecified and incompatible metadata formats that no vendors fully agree on and are stripped by various middlemen and not relevant when, e.g., iphones auto-lossily-compress the formerly signed images when emailing and whatnot. We _could_ use those, but I don't think the software work involved to actually make that technosocial system work correctly is less than a new file format (e.g., with progressive rendering and something conceptually equivalent to signing each progression to account for compression use-cases).
> persistent media watermarking
Last I checked, failure rates, both type I and type II, are in the single-digit percentage ranges or worse, and that's with very low bit rates. Is the SOTA much better than that now?
Honestly, couldn't tell you what the failure rates are for SOTA in the real world as I don't work in that industry and numbers about anything DRM-related are kept quiet. Single digit error rates would be fantastically useful to the kind of person that thinks that's a good technology though, and it's usually pretty tunable.
It has always been about trust in the authors.
The main difference is petty fakes would be cheap. I.e. my wife could be shown a fake portraying me for whatever malicious reasons.
What makes you think fake videos will have an outsized impact?
Picture - “this could be out of context” (this is used constantly in politics and people fall for it anyway)
Video removes the question of context and if the person actually did it. So now instead of printing about it or showing an awkward picture from a certain unflattering angle I can generate a video of your favorite politician taking upskirt photos on a city bus.
As the tech gets more and more realistic we’re increasingly straining the average persons ability to maintain presence of mind and ask questions.
Video - “There’s no way to tell whether this is AI-faked or not.”
Don’t get me wrong. I hope this level of fake media causes people to stop taking things at face value and dig into the facts but unfortunately it seems we’re just getting worse at it.
The problem isn’t the fakery, it’s this speed of dissemination on algorithmic social media. It’s increasingly looking like the modern West’s Roman lead pipes.
Verifiable authenticity might just be the next big thing.
If they can't have both, some people prefer confirmation to authenticity. But that's far from a universal preference.
I would like to suggest a different lens to look at it through. In my understanding, we merely had a very brief period of time where a probability of mass audiovisual media from an untrusted source being an inaccurate portrayal of events was low enough we very rarely bothered to actively think about it and consider if it could be "fake". Or I would rather say "artist depiction", to make it match pre-mass-photography world more accurately.
All that changes is this probability. It used to take a lot of resources to make a good realistic-looking media, now it's becoming a commodity. What required a resources available only to a few, is now rapidly becoming cheap. That's really all what's happening - the rest is all the same, our senses can still lie, and we can still believe in things that never objectively took place.
And it's only about the realistic-looking audiovisual media. Other types of media (such as texts or non-realistic graphics) had those issues since forever. And we're still mostly fine (though the rapidly increasing scale of mass communications poses their own challenges, of course, but that's a related but different issue).
There are at least two ways I beleve this will be approached:
1. Updating mental models of trust, so we don't eagerly believe random people posting videos on the Internet anymore. Not easy (especially for old timers, as we are tend to be quite firmly set in our ways of thinking), but next generations would be born to the new reality and misplaced trust won't be an issue again. The obvious danger is the instability during the transitional period - societies have huge inertia, so a few decades could get quite messy.
2. Attempt to legally or technologically keep said probability low. Trusted cameras, AI regulations, extra online policing - those kind of things. The obvious dangers are that this could have all sorts of weird implications for freedom of speech, personal computing accessibility, and could also contribute to increasing power disparity.
Whatever happens in the future is unknown. All we can do it to try our best to steer it in direction we think is right.
IMO all digital content is going to have to be signed so the provenance trail can be crawled by an AI across devices.
https://aditya-advani.medium.com/how-to-defeat-fake-news-wit...
From the headline, I was expecting VP candidates slapping each other in the face with a glove and then facing off at dawn with loaded pistols.
In real life, other humans are not machines you can put kindness tokens in and get sex out. AI, on the other hand, you can put any tokens at all into AI and get sex out. I'm worried that people will stop interacting with humans because it's harder.
Sure, the results from a human relationship are 10,000x higher quality, but they require you to be able to communicate. AI will do what it's told, and you can tell it to love you and it will.
for some values of "will".
Which gene do you think encodes for having the hots for AI models?
You remind me to a reporting I saw on Taiwanese schoolchildrens' career goals. Most reported aiming for the semiconductor industry. Crazy how the local gene pool works, what a coincidence.
What about us mortals that want to deal with the problem in our lifetimes?