Cryptography may offer a solution to the AI-labeling problem | Hacker News Reader

Cryptography may offer a solution to the AI-labeling problem

technologyreview.com

45 pointsrntn38 comments

> What’s more, since C2PA relies on creators to opt in, the protocol doesn’t really address the problem of bad actors using AI-generated content.

… or how to waste money on useless regulation and fear mongering. As long as people have access to open source tooling to edit JPEGs, there is no way any watermarking system works towards political goals. But I am sure consulting and tech companies working on the project are keen to do forced sales of their software.

solution: make sure people lose access to such "dangerous" tooling

this will surely make Adobe stock rise even further. it's a win-win! (and yet... I have a sensation we all lose)

/angry sarcasm.... I just keep reading awful news lately

This appears to simply be a tool that adds additional signed metadata to the content which is trivial to strip from the file, allowing malicious users to not have that "AI generated" label show up... Optional metadata is not really a protection mechanism except in walled garden ecosystems.

Isn't this what part of the point of cryptographically signed artifacts such as via GPG is for anyway? Historically Linux package managers explicitly avoided using TLS / SSL for distributing binaries over networks because they wanted the userbase to build a habit of verifying signatures and checksums provided by the distro maintainers at every step of the process as part of the shared responsibility model.

Google the fingerprinting and traitor tracing techniques used by media studios. If a user screen records their HBO stream, or uses a cam in a theater, the studio can identify exactly which user it was streaming to or exactly which time and theater showing it was recorded in. This fingerprint is resistant to image warping/flipping, bitrate downsampling, image desaturation, cropping, frame drops, added noise, and nearly everything else.

It seems to me like the opposite solution is more robust. Rather than putting digital watermarks on AI-generated content, put them on human-generated content, and you can treat anything that doesn't have one as possibly AI-generated.

How is that more robust? It seems to me it would be more robust to have it be on AI-generated content, where it can be done automatically. Also, there are fewer AI generators than there are people, so the total effort would be lower.

It's AI that's presenting the problem here, why burden uninvolved others to provide a solution?

Because AI media generation can scale up way past human media generation, and probably will.

Because it's much easier to remove a cryptographic signature than it is to falsify one, and there's a greater incentive in passing a generated file as being created by a human than the opposite.

But it's a lot harder for people to do this than for a computer, which means people largely won't.

Also, I still don't see how it's fair and reasonable to put this sort of burden on innocent others when they aren't the ones making the problem.

With software support it doesn't have to be any harder than just saving a file. You set up your keystore once and then your production software does the rest. I'm sure there's a lot of popular digital artists who'd like for people to be able distinguish their art from generated stuff that imitates their personal style.

>Also, I still don't see how it's fair and reasonable to put this sort of burden on innocent others when they aren't the ones making the problem.

Reality is what it is. There's no point in arguing about what's fair or not fair, what matters is what solves the problem. If you were fighting your evil clone and I had to shoot the fake one, would you say "why should I have to prove I'm myself? My fake should just turn himself in", knowing you're risking getting shot because he'll do the exact same thing?

> I'm sure there's a lot of popular digital artists who'd like for people to be able distinguish their art from generated stuff that imitates their personal style.

Sure, and this sort of approach makes sense for them. I'm thinking of everyone else. Not artists, but ordinary people doing ordinary things.

> what matters is what solves the problem.

True, and fair enough. But I don't think putting this burden on humans would actually solve this problem, because not enough humans can or will do this.

Ordinary people just send files to people they know and have no interest in proving the authorship of those files, nor are the people who receive them interesting in verifying the authorship. Hell, I have GPG set up on at least two computers and I've never sent a signed message to another person.

This is a problem only for people who publish content and want to make sure everyone knows it was they who made something, and for people who consume/use content and want to make sure that a given piece of content was made by a human.

> This is a problem only for people who publish content and want to make sure everyone knows it was they who made something, and for people who consume/use content and want to make sure that a given piece of content was made by a human.

Right, which includes an awful lot of ordinary people.

But if the proposal is intended only to cover the more visible people, that's fair enough for now. We still need a more general solution.

In addition to siblings notes, we need to assume that a large chunk of AI generated content will be generated by bad faith actors using custom implementations. If you're generating a lot of images, it's going to be cheaper to run your own infrastructure.

Also, many digital asset management pipelines are homebrew hacks built into bespoke CMSs and are terrible at maintaining metadata.

Given the extreme value of the protections of modern copyright, should there not be more methods to prove it? If there was a public key registry to allow cryptographically verifiable creation dates that contain detailed metadata, why, as an artist interest in preserving and protecting rights to my art, wouldn't I?

Wasting that on AI is the ideological equivalent of leaving ASCAP voicemails of my farts.

Been saying this for awhile. As dumb as some nfts are, I do think that having a public registry that logs a paper trail for all human-generated media is a necessary solution in response to AI.

Working on this: https://proofivy.com/

Better title: "AI companies propose using C2PA to identify AI-generated content"

TIL that’s the standard for when eg a camera signs a photo, and then editing tools can further sign to continue the chain of provenance.

My money is at (a) this won’t be universally used and (b) “laundering” AI content is just about removing the signature, and nobody will ever care. In fact, the signatures will be removed unintentionally by copying, downscaling etc.

Even if all media editors and viewers of sorts were to use it, like some sort of authoritarian wet dream, breaking that DRM would be top priority for hackers and since many if not most keys are client side it’d be trivial to crack and spoof anything.

If you want to prove provenance without centralization of power like certificate authorities, you could do it with a blockchain. I'm sure someone has implemented this.

Alice wants to prove to everyone that she took a photo. Or, at least, she wants to attest that it is hers from a given time on.

1) Alice takes photo at t=0

2) Hash is calculated of photo

3) Hash is signed with Alice's private key

4) Signed hash is uploaded to Ethereum blockchain (probably bundled with thousands of others to save money, or on some other cheaper/faster blockchain)

5) Bob can verify that Alice had this photo starting at t=0

Combined with other information (like "This is a photo of events that happened on Tuesday starting around 11AM"), this could be useful in the context of journalism or whatever's replacing it on the web.

It doesn't help with determining how Alice took the photo - with her camera or with her version of Stable Diffusion. Only that at t=0 Alice uploaded the hash of the photo she had before it.

Exactly.

What your parent described is a type of trusted timestamping and one doesn't need blockchain to implement it.

[0]: https://en.wikipedia.org/wiki/Trusted_timestamping

Thanks--I didn't know the generic term.

Yep, true. I can imagine cases where it is relevant, though. Like...something happens in the past, and people only become interested in it at some point in the future.

For instance, Alice is accused of a crime that happened a month ago. There exists photo / video of Alice that is attested this way proving she hasn't committed the crime, and this is taken by Bob who seems to be an uninterested third party.

It doesn't preclude criminal Alice from setting up this situation in advance given her intent to commit a crime, but it does seem to make it way harder.

While any kind of "digital notary" service would be enough to prove the minimum age of the data, but that's all it proves.

Consider a scenario where I generate a bajillion "predictions" of different winning lottery numbers, and I only unveil the one matching prediction--with provable age--while keeping all the non-matching ones hidden.

All good points, thank you

What if you take a photo of a photo, or hack the camera to feed in images to the sensor?

That is slow, and slow is not that bad.

Gen AI at high speeds is the bigger issue, IMO...

If one found a way to hack a camera to produce arbitrary signed images, I bet this could be automated to be pretty fast.

This seems like an obvious solve. Produce cameras with HSM (hardware security modules) that cryptographically sign the image using the existing certificate infrastructure. Get browser vendors to visually indicate signed images. Now you have a "class" of images that are known to be produced by taking a photo with a verified device.

Except you suddenly lose the ability to resize, edit, crop etc without having to have a “trusted editor”.

And the analog hole is still there. You can always just point your expensive HSM equipped camera at a high quality print (like a film telecine) of an edited image and it’ll have a good signature.

Great point on the analogue hack. But I think it's a perfect-is-enemy-of-good situation. There is currently no such thing as digitally verifiable media. If such a thing existed, it would at least partially shove the cat back into the bag (maybe people would abuse the cameras with the HSMs in this way, but its one step better than having all images with no verifiability). Whats more, Photoshop has existed for 25 years - and convincing Hollywood SFX for 30+ - so clearly it is deep fakes specifically that are the nascent threat. Doesn't HSM at least help address low effort deep fakes from people without HSM enabled cameras? Also, you could put in a depth range sensor and make the depth reading part of the signed payload.

I just think we’re trying to find a tech solution to a social problem here. We don’t need to trust media, we just need to tie media to the account that uploads it and decide what accounts we trust. The issue is that current social media encourages users to reupload media rather than simply point to existing uploads, breaking the paper trail up. It does this because there aren’t any integrations between social media sites: they’re all centralized and isolated from one another.

> Except you suddenly lose the ability to resize, edit, crop etc without having to have a “trusted editor”.

The algorithms of an image manipulation program form a (very dumb) AI that does the respective operations as requested by the user.

Most cameras use heavy AI inside. you just move the market for AI effects and editing onto camera vendors. It does nothing to solve the underlying problem but generate a monopoly somewhere else by restricting information flow to solve a social problem.

Movie piracy still exists and stronger than ever despite billion spent trying to lock things down.

Basically you are arguing Web Integrity but for Camera.

I did share it in a separate thread, but I developed an adaptation of the logit biasing idea that directly integrates identity proof into the language model output. I think it very directly addresses the challenge of language/diffusion model authenticity.

github.com/HNx1/IdentityLM

This is impossible. You can always take a picture of an image to reset its status.

Reply on news.ycombinator.com