C2PA Cameras Do Not Survive Contact with Reality | Hacker News Reader

C2PA Cameras Do Not Survive Contact with Reality

da.vidbuchanan.co.uk

183 pointsRetr0id143 comments

Aside from the fact that this was obviously never viable and the entire problem is clearly unsolvable if you sit down and really probe it for fifteen minutes, what I find most frustrating about this is that the false promise of preserving photos as reliable evidence is actively harmful.

You will not build a perfect system, or even something near perfect. The best you're going to do is make it so that it's hard to casually present AI photos as real, leaving only the cases where it really matters. In the "best" case, you've just made the public more trusting of photos in general, so that when there's actual money or power on the line that makes jumping through the hoops to fake authenticity worth it, the public is more susceptible.

The best outcome at this point is for everyone to get on the same page that photos have roughly the same probative value now as drawings. Poorly thought out snake oil efforts to prove authenticity are only going to delay that.

Are we entering a world where if I took a picture with a film camera and scanned it, it would be rejected as not real?

This is ridiculous.

It's not a matter of "rejected as not real", it's "There's no way to know if this is real or not".

When 99% of people are posting pics from their big tech approved smartphones with the "200% real no way to fake this" badge it doesn't matter that absence of the badge technically doesn't mean fake, that's what it will be interpreted as.

The defendant submitted a remarkably high quality video of you saying you generated it with Nano Banana.

And their video was C2PA signed.

You can even do it wholly in an analogue process, do a darkroom print, and it could be seen as fake as well - darkroom techniques are quite flexible as well. And a negative itself? That could be a copy of a prepared printed slide.

In my opinion, the benefits for end users are rather minimal since I doubt an average person would ever be checking C2PA provenance data, but there is a commercial incentive for Google and others to promote C2PA, since it makes preparing training material for Machine Learning significantly easier, and perhaps as a smaller benefit justifies hardware attestation that locks users down into proprietary OSes.

> it makes preparing training material for Machine Learning significantly easier

How?

A huge problem with ML training is the ‘ouroboros issue’ - training ML with input from other MLs breaks things in very deep (but difficult to stop/detect when it’s happening) ways due to the way the internal math works (model collapse).

Right now, the Internet training set is becoming more and more contaminated with better and better generative AI images and video.

It makes the models more screwed up, and makes it very difficult for humans to figure out what is original and not too.

If there was some signal that could at least make it easier to identify ‘original’/real images…

"Model collapse" is often overstated, as this paper demonstrates: https://arxiv.org/pdf/2404.01413

The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.

Not when you also including poisoning attacks.

I remember the original paper showing issues with even a couple percent of certain kinds of synthetic data too, not 100%.

What do you mean by poisoning attacks - stuff like Nightshade or Glaze? I was under the impression that those have largely failed to achieve their goals.

Those aren't poisoners. The are attempted protectors.

I think the term "AI incest" would sum this up :)

OK, but any reliance on a fakable signal such as this seems guarateed to invite poisoning.

I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing.

People are concerned that the technology will lend additional credence to the last 0.1%. But anyone who thinks about the technology for 2 minutes will see you can just point the camera at the screen. In cases where it really matters (a court of law, internet arguments between nerds) people will know it's not 100% reliable. Locks can be picked, and signatures can be forged, but that doesn't make them useless.

"C2PA Cameras Do Not Survive Contact With Reality" does not survive contact with reality where very, very few users would even think of rooting their phone so they can create signed fake images.

When I search for "C2PA" on the google play store, there are more AI-watermark-removal apps than there are signing apps. Certain types will jump through ridiculous hoops if they think it will affect their algorithmic reach on social media.

Malicious users don't need to root their own phones. They just need to go to fakemyimage dot com, and someone else's rooted phone in a clickfarm-type setup signs it for them. I am not operating such a service myself because I thought it was unnecessary in making my point, but perhaps I will have to reconsider.

You're arguing that lots of people can spoof this, the other guy is arguing that nobody will know it can be spoofed so it will do more damage. But these are contradictory -- if fakes become common, then they will also become common knowledge. The impact of any given fake is reduced if there are more of them. The technology doesn't need to provide 100% assurance. If it adds even a little friction to the slop mills then that's increasing the signal to noise ratio.

It's actually worse if it is plausibly trustworthy for "99.9%", since that's enough that naive users will get accustomed to believing the verification badge is authentic.

When a motivated malicious user (who doesn't actually need that much resources) will be able to convince people something is authentic because the verification passes when it shouldn't since naive users are primed to believe it by default.

Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not, since if there's really all that much riding on it, people will point out it can be bypassed.

Should we also abolish Pangram, because it's not 100% accurate? Someone might be convinced a text is not AI-generated when it actually is! We should get rid of it rather than fool people into thinking it can be determined accurately. What about antivirus? We should abolish it as well rather than fool people into thinking that their software is ever 100% safe. What about HTTPS? We shouldn't call it "secure" shell because the computer you're connecting to could be compromised! I could go on and on and on.

The median instance of AI image generation isn't evidence in a court case. It's cyberbullying, or deepfakes, or fake news. It's called "slop" because there's a lot of it being churned out at low effort.

You're missing all of the points that there could be by focussing on random people.

While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level.

This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.

It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors.

I like being contrarian as much as the next guy, but "Actually, having security is worse for security" is taking it a little too far.

I am repeating myself, but this is about systems, and not about people.

It is however in the interest of the people to keep the systems running in an untainted way.

As said, on the individual level it's a tragedy, but one that can be absorbed somewhat. Democracy itself failing otoh is kinda hard to absorb.

C2PA is not "having security". It is "having an illusion of security for compliance and CYA reasons, that can be fairly trivially exploited by nation state actors". Banality of evil. Again.

___

Actually, come to think of it, "security" is the wrong term there. Signatures don't secure anything. They attest.

Those are different things. Argh and I ran with your term aah

This is pascal's mugging. Democracy itself might fail! You're just inflating the stakes of your hypothetical bad outcome until it can overwhelm any positive upside.

Not to mention, democracy is under just as much or more threat from "banal" fake news created by citizens. People gonna people. They don't need the DPRK lying to them to fool themselves.

Yeah I think any additional security is good. At worst this could help in a lot of court cases. Someone presents photo evidence - it could be manipulated - it could be not. This happens already. Then someone produces an original higher quality version (like a raw photo - which I take even on my phone at all times now) and experts can verify that as the original.

And I agree on state actors. If a major one is invested in something like this they might have well compromised the signing project itself, the verification process, or even the court system or media. That seems like a rare and extremely high bar to guard against.

Exactly. It's just more information. It doesn't have to be 100% accurate in every case, to be useful.

> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together?

Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with.

This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.

Why is this being framed as 2 types of users, lovable pranksters and fraudsters? There's a whole spectrum between these 2.

Also I'd like to know if a "joke" is likely fake.

I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior.

Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use.

In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.

Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally.

Your argument applies to any imperfect security technology -- aka practically all of them.

> Your idea of a criminal seems to be hypercompetent and think of everything.

No, my idea of a criminal is someone who is motivated to commit crime, and I feel that we've already established in this thread that the approaches we're discussing are motivation gates far more than competence gates.

> Smart, dedicated, technical people can typically make more money legally.

Then who's been running all the botnets, writing cryptolocker malware, and running phishing scams for the past couple of decades?

We've always had script kiddies, and now we have people using AI itself to do malicious things. Technical skill has never been an obstacle for sufficiently motivated scammers.

> Your argument applies to any imperfect security technology -- aka practically all of them.

Ultimately, everything has weaknesses, and with enough effort, most measures can be circumvented. But how much effort is enough varies wildly between solutions.

There's a huge gulf between a "no trespassing" sign, on the one hand, and a concrete wall topped with barbed wire, on the other. The "no trespassing" sign only keeps out people willing to obey it; the concrete wall keeps out anyone who isn't willing and able to accept the time, effort, and risk necessary to climb over it or knock it down.

And the point is that using digital signatures to distinguish AI-generated media from hand-made media is much closer to the "no trespassing" side of things than it is to the wall. Maybe it's analogous to a gate with a latch you can open from the other side if you reach over in just the right spot.

Your no trespassing and concrete wall analogy again indicates the black and white thinking here.

I'm afraid it doesn't. To the contrary, treating effective security in terms of how it influences the attacker's cost-benefit tradeoffs, and evaluating proposed measures on whether the effort threshold they create is enough, is quite literally the opposite of black-and-white thinking.

And in this case we can clearly see that a solution that (a) does not substantially increase their costs -- and in fact, as I've pointed out above, only really filters by motivation, not by time, money or effort, and (b) doesn't target their potential benefits at all, is one that isn't likely to be effective.

You're conflating the effectiveness of the mechanism with its systemic impact.

* Pangram: Yes we should really be discouraging people from putting trust in tools like this because they can't be made totally reliable.

* Antivirus: We should be building application environments with robust security models so that malicious software has a limited blast radius (like we do on mobile, like the Linux ecosystem is trying to do with Flatpak, etc).

* HTTPS: HTTPS is a strict upgrade from HTTP so we should be using it everywhere possible. The UI symbols to indicate to users the security expectations they're getting are good practice.

* ssh: This is just an inappropriate comparison.

The HTTPS comparison would make more sense if actually 0.1% of the time when their browser said they were using HTTPS it was just lying.

Pangram: I'm not arguing against encouraging skepticism; I'm arguing that the technology is not useless. It's good for people to know the limitations, but it's still evidence.

Antivirus: "Actually, we should build this hypothetical better thing" is a cop-out.

HTTPS: C2PA is a strict upgrade from unsigned photographs, so it should be used wherever possible.

SSH: Totally appropriate, the entire point of the discussion is whether it's permissible to the user that they might be more secure.

I agree with "I'm fairly certain this will defeat 99.9% of malicious users".

Also, note that C2PA should have something like Level 3 as well: "The image is mathematically proven to have come from the physical camera sensor."

It's somehow difficult to achieve this, but it's possible (although the attacks will always be possible of course).

Difficult and also solves nothing, since you can just point the camera at a screen.

There is active research addressing this problem, for example https://www.usenix.org/system/files/usenixsecurity25-park.pd...

But yeah, difficult too :)

Have you ever tried pointing a camera at a screen?

The screen is doing all kinds of crazy things that are not apparent to your eyes, but that show up clearly on camera.

Some screens are like that, but it's not an inherent property of all visual displays.

For example, you'll see a sort of barber-pole effect when pointing a video camera at a raster-scan digital display whose refresh rate isn't synced to the camera's frame rate. To avoid that, sync them, or maybe use a colour e-ink display.

Alternatively, print a high-resolution version with a decent photograph printer and take a picture of the print with the C2PA camera.

What percentage of users are malicious, do you think? To me the issue is precisely what the product is trying to solve: propaganda using faked footage passed off as real, which generally has no real difficulty with resources available to boost their message. Giving that any kind of stamp of authenticity is a bad idea, IMO, and the fact that it'll work on 99% of the cases that don't matter makes it even worse.

> I think it's useful even if it can be spoofed. Many people don't even bother to edit visible watermarks out of AI photos/videos. I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing.

Exactly: most people don't bother editing visible watermarks out of AI-generated media, because they have little or no incentive to bother doing so. This will "defeat" the 99.9% of users who are not actually trying to do anything malicious, but will be a minor annoyance to the 0.1% of users who are actively engaging in fraud, fabrication of evidence, etc.

The upshot is that not only us this not useful for its intended purpose, it will lure people into a false sense of security by creating expectations that AI-generated media will always be easily identifiable as such, and reduce the level of scrutiny that gets applied to the stuff that actually is malicious.

> I'm fairly certain this will defeat 99.9% of malicious users, many of whom won't even know it exists until someone points out it's missing.

Everyone realizing that photos don't prove anything would defeat 100% of malicious users. I don't understand what people incorrectly trusting photos is supposed to achieve at this point, in your view.

Good luck with that. People are basically going to believe photos if they're aligned with what they already believe. They're not going to make strong decisions based on the C2PA tag. The idea that people will revise their entire worldview or submit to fraud based on a C2PA tag is typical HN thinking. People don't trust technology that much.

Almost everyone seems to be overindexing on this second-order effect. But second-order effects don't typically negate or surpass the original effect, because they depend on the original effect.

I don’t think there is a technical solution to this problem but I think there is a legal one.

Make it a legal requirement to mark AI generated photos and enforce penalties for posting unmarked AI generations. Social media should also mark the country of origin for each post, with the knowledge that posts from your own country are covered by these laws.

The EU did this, Claude and Microsoft complied and HN was practically rioting about it literally yesterday: https://news.ycombinator.com/item?id=49421158

No, this would be literally you (the user) checking "this is AI generated" before you post something publicly. Platforms can auto mark it if they detect a watermark like SynthID but they can also leave the choice up to you.

The point is making it easier to go after bot accounts that spam generated videos to influence politics. They need to disclose that its AI and lose their power or lie and get criminally prosecuted.

It's not the same thing as model providers adding a mandatory watermark to everything, and even worse one you can't verify yourself and have to trust that Anthropic is telling everyone the truth. People should have the option to use undetectable AI tools in private, even if it's illegal to post the outputs on social media.

I think public key cryptography offers a technical solution that is nearly as ideal here as for its existing uses for securing communication between physically remote actors -- please see my comment here for details: https://news.ycombinator.com/item?id=49444227

I say "nearly", because as soon as you need to keep a private key secure from someone with direct physical access to the device, you're entering dangerous territory. TTBOMK there's no way to make a "perfect black box", so it becomes an arms race between defensive "obfuscation" and tamper detection mechanisms in the one hand and stealth scanning techniques on the other. But this is already the case for TPMs -- that is, the situation is no worse than for an already widely accepted technology.

> Enforce penalties

Really, everyone should know by now it is not as simple as that. We already have lots of laws that are not enforced (fully or at all) because there is a cost to enforcing the law - time and money. Some random person uploads an AI image to Facebook and you really expect the police to expend a few thousand $ prosecuting them for what? To Server as a warning?

Some people will be untraceable. Some will claim innocence or mitigation or some disability that excuses them. Even those that might be prosecuted will be fined and won't pay which costs even more to follow up.

We should know that the legal system does not create an obedient society.

That hasn't done anything to stop scamming, so why would it apply to AI? People located outside of areas with these laws won't have to follow them, and this reduces building a immune system to such actions, making people more likely to fall for it when done by those not bound by the laws.

In a real like political misinformation, this will have the effect of making people trust non-watermarked images more, which will then be used by foreign actors to pass off propaganda as legitimate.

Also, if you don't hold people responsible for spreading an image they know is fake, bad actors can take advantage of this even within the US (they purposefully spread a image they have reason to think is fake but lacking a watermark), but holding people responsible for a strict liability crime for spreading AI without knowing it is AI seems an even worse route.

I'm not sure a law even makes the issue better in a 'don't let perfect be the enemy of good' sort of way.

What makes you so certain that this valuable research showing weaknesses in today's systems will render the C2PA concept useless forever?

Yes, software LPEs are a risk -- as they are in every nontrivial computer system. New ones will appear, and old ones will be closed in time, as TFA acknowledges.

Re hardware attacks: The (neat!) glitch injection attack the author describes in the linked "lighter" page only raises the implementation cost of doing image certification properly. For example, if the camera module presented only an interface that dumped raw RGB or JPEG-encoded data plus a digital signature that used a private key known only to the manufacturer, then all that would be required to verify a "downstream" image would be to keep a copy of those original bytes inside the final (potentially cropped, filtered, AI-ed, etc.) image, in the worst case roughly doubling its size on disk (though certainly more efficient schemes could be designed). Any interested third party could then compare the original and final images by eye and decide for themselves whether or not the subsequent processing materially changed the image's "meaning".

Finally: Does the existence of lock picks or bolt cutters render padlocks pointless today? Does it corrode society by encouraging people to mistakenly believe that anything they put behind a $5 padlock will be safe forever? No, and no.

This is yet another of these absolutely caustic takes that come with a veneer of intellectualism, but actually just ignorantly deconstruct reality.

Each of them weaponizing the rules of the platform that (for sensible reasons) demand you engage with the strongest interpretation of the message/argument you see.

The asymmetry of effort there is unsustainable, and that's exactly the point.

No idea how that could be solved. Maybe a meta comment like this one helps.

I mean if you think about it, it shouldn't be possible for some anon account to drop this and sound like it's a worthy contribution to a debate against some real person with a real name, a track record and multiple thousand dollars of bricked hardware leading up to that assessment. (Nor would it make sense for a non-anon but equally empty account)

It makes no sense, and the guarantees regarding protection of speech and all do not apply to these topics, because it's not an opinion that would get your real name in jail.

What can we do about these social exploits. Someone tell me please. It's driving me up the walls

There is no content in your response. At all. I honestly don't think I've come across a post this devoid of content on this site before.

All I can perceive in it is a generalised hate towards some broader thing that you feel certain I'm consciously aligned with somehow, and that you think is inherently and obviously evil. Because... I don't use my real name on here? (Is your real name "hypfer"?) Because I don't have bricked hardware to back up my opinions?

If you want to convince me or others that something I wrote is wrong, stop hyperventilating and engage with at least one of the specific claims in my post. For your own sake, I'd also suggest editing or outright deleting your original post.

It is correct that I have completely side-stepped your frame and not even attempted to engage with it. This is on purpose, because wanting people to engage with that specifically engineered frame is exactly the bad faith mechanism I've described.

What is interesting is what happens when these tactics are called out in a meta comment like I did, because it's actually a common thing that people start flailing like this once you side-step the script that was supposed to be followed.

Someone acting in good faith would not counter with an attack and a double-down like this, but with intent to resolve the situation (+ probably feeling a bit bad about an exchange having failed and being misunderstood).

I also like the "hyperventilating" claim. Easiest way to get out of a situation is to invalidate the source, and easiest way to do that is to claim emotions.

And the fake concern (for what, even?) of course.

> For your own sake, I'd also suggest editing or outright deleting your original post.

___

Anyway. I think the convincing people is actually working quite well here. Though not in the sense you'd think.

I'm just pointing a spotlight at what I believe is violating the spirit of the rules (while staying within the letter of the rules).

What other people make out of that is their decision to make.

What on earth is happening here? I seriously have no idea what you're even talking about at this point. You seem to be five levels of meta deep and talking in circles about yourself?

Very bizarre.

That's a very long-ass way of saying "I disagree with this comment, I can't be bothered explaining why (too much work), so instead I think this user should be banned". I extracted nothing useful from your comment, which bears no relation to its parents that I can discern, am irritated to have spent time reading it and composing this reply, and also find your attitude pretty rich when you're also an anon account, and with less karma too. If you want to "call someone out" you've gotta be on damn solid ground and bring receipts. As it is you've just decreased the signal/noise ratio of the thread and contributed to a toxic atmosphere. I would just flag and move on but in this case I think feedback is important, and unlike you I can be bothered to explain what it is that's got me riled up.

This is what I'd call the "enabler" archetype that gets unhappy about friction (rightfully!) but takes some.. uh.. mental shortcuts when it comes to tracing back the source of it.

I'm sorry man. The problem is that this exact described dynamic (see also the other reply by me which you seem to have conveniently missed), is that it affects _all_ internet spaces and makes them utterly miserable.

I agree though that this meta stuff doesn't exactly improve SNR, and I am open for better solutions. But those need to actually solve the problem, instead of just silencing the immune response.

___

Interesting side-thought

> with less karma too

I think this might point at (one of the) root cause(s) of the dysfunction I'm pointing at here.

Internet points ceased to be a metric documenting value quite a while ago. Something something Goodhart

Gonna make this real short and simple: it's totally unclear what you're on about. You've described nothing, merely complained, infuriatingly, in the vaguest possible terms.

Don't take this as dismissive or a dunk or anything like that, but maybe an LLM can help with unpacking?

At least I know that they understand my writing and that they can help me understand other writing I don't.

I can assure you that there is a point. Whether it is worth bothering is of course your decision. I'd wager probably not, but I guess that depends. No hard feelings either way.

The comparison to padlocks and bolt cutters is not relevant, they are quite different shapes of problems. A certificate that is weak enough that it should not be trusted in any case where it matters is actively harmful, not just mildly less helpful.

(This is true in general: if you add a trust signal that is mainly just a bit of effort to broadcast, you'll find that scammers and fraudsters will show that signal much more reliably than honest actors. For example, every email spammer has DMARC and DKIM set up absolutely perfectly)

I guess you're making the argument that, while padlocks sometimes protect things of low value, thus justifying the use of cheap, easily broken locks in those cases, the times where authenticity of an image is important are nearly always cases of high importance, meaning that there's no case where an assertion that's actually fairly cheap to forge (assuming a similar "lighter" glitch attack works on phones, less than $1000 for a modern smartphone that may get bricked, a soldering iron and an afternoon's work) makes sense? Perhaps so. It's certainly easy to think of images or video clips of very high importance, and outside of kids uploading AI clips to Reddit and pretending they're real, I can't think of any similarly low-stakes cases for image authentication.

What do you think of my digital signature idea?

> Does the existence of lock picks or bolt cutters render padlocks pointless today

A padlock shouldn't be the only component of the bigger physical security picture.

A random person carrying/using bolt cutters or trying to pick a lock looks suspicious, and should be noticed by on-site staff or whoever is monitoring the security cameras.

If the padlock is decent, there will also be an inherent delay involved in bypassing it using those tools, which should increase the likelihood of the person being noticed.

If the padlock is used in an unattended/unmonitored location (i.e. a remote vacation house with no neighbours and no security cameras), then the padlock is probably only good for keeping honest people honest.

A lot of physical security => information security analogies are misleading for the same reason. In information security, it's very possible for an adversary to have effectively unlimited time to perform their attack (or to develop the means to perform the attack quickly).

Imagine a scenario where any determined person could e.g. spend a month in their home workshop and develop a pair of gloves containing transducers that would vibrate any padlock open in seconds. They could mimic the action of using the key or entering the combination to avoid looking suspicious. Also, the gloves have a button that instantly creates another pair of the same gloves at no cost. How much value would a padlock have in that scenario? That's the kind of asymmetric playing field one has to consider in information security contexts.

People got on fine until 200 years ago with the only means of rendering a picture being to draw it, as in Hogarth's calumnous image of the in fact really quite civilised Gin Lane (https://www.theguardian.com/artanddesign/picture/2012/sep/12...)

People didn't have the ability to distribute forged images to millions to form a mob in seconds though.

> and the entire problem is clearly unsolvable

The problem is not identifying AI generated media

The problem is identifying real media. That it can actually do.

Worse than not working, this will likely be used as yet another excuse to attack computing freedom.

It's compliance for advertising. Your client doesn't want AI in their project you show them the audit trail and if it turns out to be faked you point to the supplier who faked it. Our agency signed a insurance not only because of clients who don't want to use AI in their artwork but also because of the EU AI act. They c2pa to get their money back from a cheating supplier if they are not compliant with the AI act.

I'm very surprised Google put in so much effort to implement an approach that is basically the equivalent of client-side verification of passwords. Did no one designing it mention that it could be defeated by any rooted device?

Not any rooted device, it must be rooted via an exploit. Still pretty bad, though

Actually I think this approach is very forward looking! Attestation is on the cusp of becoming a very powerful technique. We just need to figure out how to build 100% bug-free and 100% secure hardware and software, and then it's gonna work great.

Wait a minute. I think you might have forgotten a /s, I can spot this kinda thing.

Well, all one has to do is look at the bigger picture of how rooted devices are being shuffled into 3rd rate/totally blocked experiences and the overall direction of things starts to take very clear shape.

At over a decade old, still prescient as ever: https://www.youtube.com/watch?v=HUEvRyemKSg

Someone better figure out how to make computing devices at home from everyday parts because the only way I see this (shockingly rapid) arms race end is legally mandated, cryptographically locked down hardware and software (or even thin clients) everywhere.

RMS must be having daily nightmares at this point.

P.S.: Fantastic talk you linked there.

I think it might tell us something about the culture there by now.

Doesn't sound like it's engineering-driven, even though they still do have a lot of capable engineers sitting there and atrophying.

I also wouldn't rule out that the less capable ones actually believed that the systems they've built are unrootable or something like that.

I always got the impression that C2PA is a way to say "this photo came from the BBC (for example) and they've only signed it because they've verified the supplied edit chain". It's always been obvious that one could point a camera at a screen, I don't think anyone involved with C2PA has claimed otherwise.

It seems like there's a big disconnect between what C2PA says it's for and what certain journalists think it's for.

Why not just have BBC sign stuff with their own creds then? You can originate the chain of custody anywhere and relying on the camera is kind of silly when the reputation of the original publisher in a more meaningful backstop.

It's useful to have all your tools automatically apply metadata in a standard way instead of having to keep track of it manually. Most cameras already add metadata that says what camera and lens were used, but you lose that as soon as you import it into Photoshop and export as a jpeg.

C2PA is a bit nebulous as a concept, which is part of the problem. It is both of these things. The BBC type use case where a publisher signs their own content with their own keys seems reasonable to me, or at least, not obviously broken.

However I question the value-add when e.g. the BBC website is already authenticated by nature of being served over HTTPS, and anyone who redistributes BBC content can and should link back to the source.

> It's always been obvious that one could point a camera at a screen, I don't think anyone involved with C2PA has claimed otherwise.

They haven't claimed otherwise exactly, but some have implied it's a solvable problem. Here's where the "learn more" link goes, for when Youtube annotates a video as having C2PA metadata: https://support.google.com/youtube/answer/15446725 (Google is a C2PA Steering Committee member)

> The metadata that leads to a 'Captured with a camera' disclosure is made by a third party (for example, a camera manufacturer). This means that there is some risk that someone could take a photo of another screen showing synthetic content. Because the other screen shows an image that has been modified, it wouldn't be eligible for the 'Captured with a camera' disclosure. This issue is called 'air-gapping'. Camera manufacturers will continue to develop detection measures to prevent 'air-gapping', but the sophistication of those detection measures may vary in the near term.

Interestingly they do not mention any of the other known limitations. Their phrasing is highly weasel-wordy, but the implication is clearly that they imagine picture-of-screen detection to become robust (somehow) in the medium-to-long term.

> However I question the value-add when e.g. the BBC website is already authenticated by nature of being served over HTTPS, and anyone who redistributes BBC content can and should link back to the source.

The value would be in images reposted to social media where the website an show a badge that says it came from a certain source.

Even at the hardware level, if it was a separate chip that the camera data passed through or something, that's not really good enough either, people have broken TPMs before. It'd have to be baked into the camera sensor. Even then, you could attack it from the next level up, with some fancy optics and a display, or something like that.

I don't think completely solving this sort of problem is even possible.

And I'm not sure it's even useful to solve. The presence/absence of a digital signature will never be the deciding factor in whether people accept/reject an image as authentic.

Yeah this is what I don't get why are people even spending time on this.

A desperate attempt to preempt regulation.

A desperate attempt to establish their version of regulatory capture and not have to pay licensing fees to the other guy

This is a clear explanation, thank you

Is this picture real or AI is a real problem it is worth money to solve.

Why? An image should never be proof of anything, by itself.

What should be sufficient proof for you that AI wont fake trivially?

What I'm getting at is that metadata that claims that a photo is "real" won't necessarily convince people that it is, and the lack of that metadata doesn't mean anything at all. About the only real use I've seen for C2PA is to confirm that an image bearing that metadata is AI-generated - and that marker is easily lost through editing or retransmission.

Yeah we live in the era of alternative facts and this is just more, at best, "fact checking"

I'm not saying it's an easy problem or that this is the right solution. I'm saying it's a valuable problem and so they are trying this solution.

And it's very real problem to workaround (aka, hack the device into faking the signature).

It is (not sure I agree, but I can see how one would think this) but you're not going to solve it like this

And in 2026, I don't think it's too big of a stretch to imagine that there are going to be people in power that can add + remove the metadata to whatever image they want, at will, to tell whatever story they want to create. Sadly.

There were similar fears about the webtrust CA system, but AFAIK there's no known incidents where a government strongarmed a CA into misissuing a MITM certificate, and then it was used in MITM attacks. The closest is some misissued certificates seemingly due to incompetence but weren't used in attacks.

And there are unicorns living at the end of the rainbow.

> Even then, you could attack it from the next level up, with some fancy optics and a display, or something like that.

The analog hole is alive and well:)

And at that level, it’s a matter of definition anyway. What is “AI generated”? A photo of a screen showing an AI picture is still a photo. If that’s “AI” then what about a photo of an AI generated billboard? Or a photo of a bus with an AI graphic on the side?

I can break it with zero skills. Tripod, camera, clear monitor in a dark room ... just take a real photo of a fake photo.

That might be detectable if they signed content contains focal-length metadata... but even then, some foresight and a collection of lenses would hide it.

Wouldn’t the introduction of the lidar signals embedded in the photo (say used with apple’s faceid system) help here?

I'd always thought the usefulness of C2PA was limited to verified devices in custody by trusted actors. Like a security camera with a tamper evident enclosure, or an organisation being able to attest that they recorded the imagery.

The idea that it could be used to attest the authenticity of any random person or device surely wasn't a thing serious people expected was it?

> was limited to verified devices in custody by trusted actors. Like a security camera with a tamper evident enclosure, or an organisation being able to attest that they recorded the imagery.

But that is also not the case, because whatever keys are in those devices may have been duplicated in the factory or somewhere along the supply chain. Or the stuff is cloud connected and an exploit can be executed via that.

Or, as written in the blog post you're commenting on, software exploits.

The whole idea is that the concept works for no one.

Well, you can say the same thing about HSMs, or the CPU in your device.

If there's no trust afforded to the device holder, or it's manufacturer, there's no trust in anything.

Always to degrees, and never fully, but trust can still be had.

I got a good laugh out of the "unblur to verify" first image. I dont know what I was expecting to see.

As far as AI-generated images go, that was a good one.

C2PA was never going to work to prevent abuse, but what it will do it give false confidence.

Most people don't read HN, or understand tech, and so if the device says "captured with camera" then they will believe it since Google says it's true.

Accusing someone of a crime, with fake but verified photographic or video evidence will be trivial, and claiming it's fake just makes someone tap the "Google says it's true" sign.

For the time being, I wish more workflows revolved around RAWs. You can pretty much prove the origins of an image sans cryptography by just possessing the image sensor data that you used to produce the final. It's not foolproof but AI image models can't really generate a convincing bayer raw and it's not something anyone is going to spend significant time training.

Until it matters. Which you are proposing happening.

Exactly

I fear that the long-term result might be a small cartel of "trusted" companies that holds the private keys which can attest and lead to the death (or at least limiting widespread adopting) of open-source operating systems.

It won't stop fraud by governments and/or determined/well-resourced attackers, but it'll make it difficult enough for 99.99% of the public. And as usage drops over time, it becomes more socially acceptable to justify further limitations.

Governments and corporations (e.g. banks) will require that you use a "trusted" (meaning locked-down) devices to interact with their services. We're already seeing some companies block GrapheneOS/LineageOS.

All these funny jabs at Android's security meanwhile Pixels are literally one of the most secure devices in the industry and Google invest tons into security.

Hardware attestation stands unbroken to my knowledge, only software attestation can be faked, and even then it's a constant cat and mouse game which Google continues playing until they are done with Pixel 3 generation.

C2PA is not immune to the analogue hole, sure, but dark room + photo of a photo approach falls apart the moment you bake in depth data into the image, which is already done.

David is upset that Pixel is not patching fast enough.

Pixel 2 and later had hardware attestation, not the Pixel 4 and later. Devices launched with Android 8 or later were required to support it.

Android hardware attestation using root-based verification is highly insecure. It depends on the weakest links in the overall ecosystem. There are tons of leaked keys from insecure TEE implementations. People can often even downgrade to an ancient TEE implementation to exploit it when anti-rollback wasn't used for updates. Devices never updated since launch can be used to get keys via known vulnerabilities. Google doesn't revoke all the known leaked keys because it would break compatibility across many devices. Originally, private keys were provisioned to 100k or more devices in batches. They've moved to a system where each can get a unique key and then the ones used by apps are dynamically rotated but it's only the Pixel 7 and later using it in practice for Pixels and other devices took far longer to adopt it. That only got forced in the past year or so for other devices.

Android hardware attestation using pinning is highly secure on devices with a good implementation but that doesn't work for this use case. It could work for giving out phones to people and then verifying those are still genuine, not tampered with and have continued applying updates on an ongoing basis.

I would be interested in a note on whether Sony / Leica / Olympus “content credentials” do any better with their hardware to ensure a signature is assigned to data straight off the sensor.

Unfortunately they're a little outside of my tinkering budget, but if anyone wants to send me some I'll do my best to pwn them. Can't be any harder than a Google flagship, one would imagine.

I have ordered a faulty Sony A7 IV motherboard, but due to its faulty-ness and the lack of the rest of the camera, I'm not sure how far I'll be able to get with it.

Could you make use of a Sony a6000?

My bet is they do considerably worse. Digital cameras are not designed with security in mind. Arbitrary code execution has been achieved on many DSLRs and there's even been open-source firmware projects for some.

Why would someone paying for an expensive camera to damage the pixels of their images with "invisible" watermarks?

The signature doesn't touch the image data in any way. It's just a piece of metadata attached to the image, like any other metadata tag.

I sat in on a number of tedious C2PA meetings about three years ago. It was clear that no-one wanted to hear the hard truths, and one was certainly not rewarded for introducing them. I'm guessing there were a lot of consultancies to maintain, and a lot of snouts in the trough.

This was a technology that required international governmental prescription, not voluntary or viral adoption. Either way, it was horrifically flawed from the outset.

And never mind should someone want to edit¹ the image that comes off the camera. Now it cannot remain cryptographically verified, which means it's now in the pool of "dunno" images, where AI fakes can thrive alongside it.

¹in a non-AI, traditional way, that doesn't mislead the viewer and this entire footnote should reveal how subjective and intractable this problem is

Surely the easiest thing to target is photos taken by journalists and modifications, down sampling etc when shared to twitter?

This reads a bit like "Door locks do not survive contact with reality."

The point is not to prevent state-sponsored actors to produce fake media, but to add friction and avoid shady marketing agency and photographs from claiming their pictures are genuine.

Yeah, the target is not the cryptographic "safe forever", but a real world "safer than not having it".

If we're trying to decide whether a high profile politician has committed a crime, then yeah, C2PA on the footage isn't fully trustworthy. However, every bit of footage you get that corroborates the story raises the threshold of the attack.

If we're trying to decide whether Joe Everyman has crossed a double solid while driving his truck? C2PA is probably good enough. The chances of that footage being faked by a malicious party would be low even without C2PA, but C2PA makes them even lower.

Isn't film photography verifiable with a physical negative? i.e. you could verify a digital reproduction of the analog photo by comparing the digital image with the negative. Tedious, but sound?

You can expose a negative with whatever the fuck you want. It doesn't have to be a real scene.

Does that prove you didn't photograph a display showing the digital image?

Transferring electronic images to film is a long-solved problem.

https://en.wikipedia.org/wiki/Film_recorder

I do find it somewhat ironic that it's implemented in Pixels first, considering the amount of AI processing phone cameras do to get a pleasant image.

If this could be made to work reliably, it would imply that cameras would have the ability to doxx users.

You could actually make this secure, by having firmware in the camera module itself do the signature before handing it off to the rest of the system. The key is to prevent ingress of control, as long as the module only emits signed photographs or video, and has tightly controlled input channels, for zoom, aperture, etc... it seems a quite reasonable project.

No, you can't make this secure. You're just tweaking who can pull that off and how.

And, given that the main threat here is disinformation by nation state actors, they can also attack the camera module (or its supply chain) instead.

>the main threat here is disinformation by nation state actors

When did that happen? I thought we were just talking about people being able to prove they took a photo, and it wasn't photoshopped after the fact.

I expect the only plausible chance (and it is a stretch) will be at-the-censor marking. Quantum bla bla magic pixie dust or unicorn farts something. The chance of a trustworthy (including from nation-state tampering a la Stalin et al) means of verification of digital anything is as good as dead imho.

I have a feeling Apple is going to knock it out of the park on this when they get around to it. They have a great foundation for doing image provenance well. The device attestation workflows are already there. And the same attacks that work against Android won't be as easy or effective because of Secure Enclave. Apple could run the whole signing process inside SEP.

And, Apple could choose to integrate a LiDAR depth map into the signed photo as a mitigation against the analog attacks (eg. pictures of screens).

Apple isn’t going to touch this with a 10-foot pole.

The provenance “proof” these approaches provide is very tenuous and nowhere near the “this is a real photo of a real world event taken by a real camera and not an AI image” proof that marketing types like to push.

Apple doesn’t want a PR disaster where some crazy image is totally fake but becomes world news because it is “cryptographically signed as being from a real iPhone so it must be real!”

Apple is working on their own system, which is expected to launch with IOS 27: https://www.macrumors.com/2026/08/10/ios-27-apple-reference-...

>Images captured with an opt-in Reference mode can be authenticated to confirm they were taken with an iPhone. Authenticating is done by tapping the Reference badge on the image, which sends the raw image, sensor signatures, capture time frame, and the unique hardware identifiers of the sensor to Apple's Private Cloud Compute (PCC) servers. PCC uses the information to determine whether the camera captured the photo, gives it a unique ID, and then returns an authenticated version to the user's device.

Yeah, it might never ship, or it might not ship as described, or that might be exactly what they do - I love being wrong.

If it does ship like that, it’s hard to not imagine a situation as I described earlier - an “iPhone Reference Image” being used to propagate fake news, at which point the credibility of the feature goes to 0 (and Apple’s takes a severe hit).

Wait & see.

The proof itself relies on asymmetric crypto, and that part is not tenuous (well, until Q Day, but that's a different story...).

But:

- You cannot prove that no one has ever been able to break a Secure Enclave or a YubiKey or another secure element. That's okay, these companies focus on making it so expensive that it's not worth doing.

- The analog "attack vector" is real, but that was always true with analog cameras as well. Anyone could have taken an analog picture of a screen, or even painted a hyper realistic image of something that never happened.

I think a realistic goal for provenance is to at least get to parity with analog cameras ("this is a picture directly from a sensor"), not to make the perfect system for proving that a photo is of a real world event.

Reply on news.ycombinator.com