A new way to detect ‘deepfake’ picture editing
lightbluetouchpaper.org
lightbluetouchpaper.org
Considering that training a model costs thousands of dollars, and that people reuse models instead of training from scratch, I somewhat disagree with this in general, for now.
But this specific technique implies the attacker sees the result, which means they can try different models, until one does not produce artifacts.
Ok, so that's tongue in cheek and we'd see some artifacts there but the general principle works. You could intercept the signal from the CCD, or just extract the signing key from the camera's ROM, etc etc.
That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue.
For the case of avoiding deepfakes, you need to avoid the attacker extracting a key from any of the millions of cameras that are sold every year.
It shouldn't be easy to extract it and that's it. It's even less difficult than keeping the DVD/Blueray keys secure because each device has a separate key, so if a line of devices gets compromised easily it's easy to spot.
Then you put a legal framework around what can be presented by media, the requirement for signature collection and so. And one of problems with photo/video authenticity is essentially solved.
I think there is scope for a simple self regulation here to start us off. I would love to see Reuters or the BBC start to publish their raw footage with the hashes. It is a question of starting the ball rolling
It doesn't really seem likely at all to work in practice.
[0] https://lilianweng.github.io/lil-log/2021/07/11/diffusion-mo...
Any scheme like this would be eventually broken somehow, so it's altogether a bad idea imho
[0] Normally you accomplish this by taking several photos and combining them
Like I said, I don't think it's practical...
A. Writing my own software "camera" that does the same gyrations a real camera would do to sign its raw image, but applied to an arbitrary unverified image file on my computer?
or
B. Printing out an unverified picture and taking a picture of that picture (with good lighting etc so that it is not obvious), with a camera that makes it verified "real"?
B. seems like it would work.
1) Upload image with web-based user dashboard.
2) We strip existing metadata (for privacy and all of the reasons everyone else does).
3) Our API generates a unique identifier for the asset (UUIDv4 in hex).
4) The unique identifier is embedded in the image with XMP.
5) A SHA256 checksum of the entire file is generated.
6) Via the API the checksum is associated with the unique identifier (along with some other stuff).
7) (Optionally) the unique identifier and checksum are added as JSON to IPFS via a pinning service and (essentially) an NFT is minted for the JSON verification data on the Polygon blockchain.
8) The user gets sharable[1] and iframe embeddable[2] links and the ability to post directly to various social media networks, etc.
When the links are viewed our Javascript reads the unique id, fetches the stored checksum from the API, and generates a new checksum of the image in the browser. If the checksums match a clickable icon appears in the top right of the image with additional information about the image, links to the IPFS and blockchain links, etc. Users can change the additional metadata at anytime and it updates instantly.
Long term goals are hosted javascript verification library, browser extensions, mobile SDKs, potential browser/OS integrations, native plugins for popular authoring/editing applications, and so on.
Happy to answer any questions!
[1] https://share.tovera.com/preview/c65b0658ab6e4d89963b1e0a319...
[2] https://share.tovera.com/embed/c65b0658ab6e4d89963b1e0a319a1...
Then if there is an adversarial model built for that, we will just create a model to detect images that have been changed to hide the model they were built with.
Then we will build a model for the adversarial AI on top of that and so on.
https://arxiv.org/abs/2106.00660
> First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be designed to reconstitute a watermark if the editor had been trying to remove it. Second, we show that our markpainting technique is transferable to models that have different architectures or were trained on different datasets, so watermarks created using it are difficult for adversaries to remove. Markpainting is novel and can be used as a manipulation alarm that becomes visible in the event of inpainting.
You'd need to know all the models that one might use to tamper with a picture and modify the image in such a way that it screws them all.
And the first thing that will happen is that people are going to train their models with these new 'tamper-protected' images...
[1] https://mothership.sg/2021/03/japanese-biker-actually-man/
Q: Why didn't Hitler take a taxi?
A: He was more of an Ubermensch.
Hmm... so if I ask it to add an apple to the scene it just looks for an apple and says "yep, found an apple".
That is not an algorithm to detect fakes, it is an algorithm to detect predetermined edits.
Steganography/watermarking has been not only known but used for practical purposes for a long time.
Watermarking can be used to find portions of the photo that have been tampered with.
It is actually pretty simple. And using AI for this is pretty stupid in my opinion.
You just disperse a little bit of additional signal in the photo, maybe divide it into lots of blocks (they don't need to be rectangles and they can overlap). Think of it as every small piece of the photo having its own signature embedded.
If you don't know how the signature was embedded or what key was used for it, you are going to disturb the signature and your editing attempt will be foiled. Not only that, but you will be able to tell which parts of the picture were touched and how so you can tell whether the picture was just cropped, brightness changed, or something else happened to it.
As you see, no need for AI...
Actually, it is probably even worse than having deepfakes proliferated. In this case people would expect the photos could have been manipulated. On the other hand if only handful of players can do this it can be used with much more impact.
solution: you use a denoising filter to reconstruct the watermark pixels to plausible original values. Profit!
this: instead of just simple obvious watermarks, you can instead encode visually indistinct fake-noise that deliberately confuses denoising neural networks.
They claim “We find that we can target multiple different models simultaneously with our technique.”, ie. it is reasonably generic.
how? Eh, that’s complicated, look up “adversarial neural networks”, there’s a fairly high level overview here: https://towardsdatascience.com/how-to-systematically-fool-an...
I fail to see how this can actually combat against someone with the patience to manually paste out copyright marks using something like a clone brush, or even a normal brush ?
If someone wants to steal your work, they will. There's no sure-fire way to stop them.
Anything that can help us differentiate authentic images from manipulated ones, especially when it comes to news, is imo a welcome addition in the arsenal of societies attached to the concept of verifiable facts.
Except it wouldn't change anything. Facebook doesn't care about fakes and people wouldn't care whether the picture someone shared hasn't been tampered. Also whoever manipulated the picture could just sign it again probably.
The challenge is establishing an author's credibility in the first place. If I trace a cert back to joe schmo off the street or some empty front business, then I'd probably not trust the content.
For instance, say that Joe makes a deepfake and then signs it with his key. Sure, it's beyond doubt (assuming keys weren't leaked, etc) that Joe either took or created the picture, but that doesn't in itself tell you whether Joe made a deepfake or not.
It's the same as in supposed blockchain logistics operations. If Fred the farmer says he harvested x bags worth of grain but some were stolen before he could ship them, there's no way to mathematically verify whether the theft actually took place or the harvest just came up short.
In both cases you're going to need some kind of monitoring, and that's the purpose deepfake detection algorithms would serve.
Besides, what will the camera sign? The produced JPG? The raw file? What about rescaling? Do we want ZKPs that a JPG was achieved by nothing more than re-scaling and tone-mapping another JPG or RAW file? Those ZKPs are going to be massively big and slow to verify.
If we get to a point in society where unsigned images, or images signed from a questionable source are looked at skeptically, then that alone is a step in the right direction. I don't know the final solution and I don't claim to know it, but cryptography seems like a step in the right direction.
Main comment here:
But maybe we'll just have to limit cameras outputting NFT videos, or signed frames as the only source of trusted media