There was never a time when authenticated photo and video could be trusted without knowing the source and circumstances.
Everything has always happened, so who cares? We need to go deeper than that. Many things that are perfectly a-okay today are only so because we do it on a small enough scale. Many perfectly fine things, if we scale them up, destroy the world. Literally.
Even something as simple as pirating, which I support, would melt all world economies if everyone did it with everything.
What is happening now will raise awareness of it and of course make it a several magnitudes bigger problem.
I am sure there are large efforts ongoing to train AI to spot AI photo, video, written production.
A system like printer tracking dots¹ may already be in widespread use. Who would take the enormous amount of time to figure out if somesuch is hiding somewhere in an llm, or related code.
Lying on a small scale is no big deal. Lying on a big scale burns the world down. Me pirating Super Mario 64 means nothing. Everyone pirating everything burns the economy down. Me shooting a glass coke bottle is not note worthy. Nuclear warheads threaten humanity's existence.
Yes, AI fabrication is a huge problem that we have never experienced before, precisely because of how it can scale.
You should be able to follow a chain of evidence towards an unprocessed raw image potentially.
[1]: https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat...
[2]: https://leica-camera.com/en-int/photography/content-credenti...
Edit: sister comments have links.
Both manufacturer's keys got extracted from their cameras, rendering the feature moot. Now, a more robust iteration is probably coming, possibly based on secure enclaves.
I'd love to have the feature and use it to verify that my images are not corrupted on disk.
Mid-term plan is to build an UNRAID array with patrolling. Will probably do backups with Borg on top of it, and keep another "working copy", so I have multiple layers of redundancy with some auto-remediation.
UNRAID will keep disk level consistency in good shape. Patrolling will do checksum tests, Borg can check its own backups against bitrot and can repair them in most cases. Working copy will contain everything and refill if something unthinkable happens.
I don't do that at home. I have 600GB of Nikon raws. I keep one copy on my Mac. I have a time machine backup in the house (in another room) on an Samsung T7 shield and I have an off site backup which is a quarterly rsync job to another Samsung T7 shield.
I put a lot of effort into thinking about what could be proper trustable on photos and videos [0], but short of performing continuous modulation on the light sources the light of which is eventually captured by the cameras' sensors, there's no other way, and even that I'm not sure how would work with light actually doing the things its supposed to (reflection, refraction, scatter, polarization, etc.). And that's not even mentioning how the primary light source everyone understandably relies on, the Sun, will never emit such a modulated signal.
So what will happen instead I'm sure, is people deciding that they should not let perfection be the enemy of good, and moving on with this stuff anyways. Except this decision will not result in the necessary corresponding disclaimers reaching consumers, and so will begin the age of sham proofs and sham disprovals, and further radicalization. Here's to hope I'm wrong, and that these efforts will receive appropriate disclaimers when implemented and rolled out.
[0] related in a perhaps unexpected way: https://youtube.com/shorts/u-h5fHOcS88
Signing in hardware is nice, but you then still need to trust the company making the hardware to make it tamper proof and not tampered with by them. Additionally, such devices would have to be locked down to disallow things like manual setting of the time and date. It's a rabbit hole with more and more inconveniences and fewer benefits as you go down it.
Better to just go for reputation, and webs and chains of trust in people based approaches.
A good example: https://imgur.com/iqtFHHg
I didn't mention this in my previous comment, but after having thought about it even further, I've eventually arrived at the idea that there can be basically an arbitrary amount of meaning embedded onto a photo or a video, and it being "verified" to me as a human really just means "it is what I think I'm looking at" - that's when I realized this task is likely unsolvable in absolute terms, and just how much more difficulty lies beyond the already hopelessly difficult crackpot idea of light source signal modulation that I came up with prior. But yeah, as I suggested, these are just what I could think of, and perfection should indeed not be the enemy of good.
What a load of nonsense. A little bit of humility and a basic understanding of history should quickly make you realize that.
OPs point is far more interesting and deserves more discussion
It's immensely frustrating to dress up a natural language sentence with enough precision to try and account for all and every bit of nuance, so you should anticipate and actively consider that I have missed or implied some.
Despite this, you clearly did not, and instead went into attack mode on the assumption(!) that I did miss them intentionally.
I'd recommend you take your own advice on intellectual humility before offering it to others.
The idea of "cryptographically signed photos coming out of cameras"? It's been discussed to death and is essentially a hope for magic technology to solve our social problems.
Also it won't work, technically: it's like asking for perfect a DRM implementation to be universally implemented everywhere.
You do realize that would still not provide perfect proof that what was recorded by the camera was real, right? It does seem like an obsolete idea you may not have fully reconsidered in a while.
But considering that same old idea that dates from prior to the current state, I would also not be surprised if you imagined clandestinely including all kind of other things in this cryptographic signature like location, time and day, etc.; all of which can also be spoofed and is a tyrannical system’s wet dream.
You don’t think that would be immediately abused, as it was in other similar ways like through all the on device image scanning that was injected as counter-CSAM appeals to save the children…of course?
It doesn't do a whole lot for something entirely fictional, unless it becomes so ubiquitous that anything unsigned is assumed to be fake rather than just made on a "normal" device. And even if you did manage to sign every photo, who's managing those keys? It's the difference between TLS telling you what you see is what the server sent and trusting the server to send the truth in the first place.
Presumably if you were discovered you would then "burn" the device as its local key would be known then to be used by bad actors, but now you need to be checking all photos against a blacklist. Which also means if you buy a second hand device, you might be buying a device with "untrusted" output.
The problem of Trust is a human problem, and throwing technology at it just makes it worse.
This particular idea has so many glaring problems that one might almost wonder if the motivation is less about "preventing misinformation" or "protecting democracy" or "thinking of the children" or whatever, and more about making it easier to prove you took the photo as you sue someone for using it without permission. But any technology promoted by Adobe couldn't be about DRM, so that's just crazy talk!
You’re forced to trust the source and “read between the lines” or you’re reading something politically motivated.
Nothing new. I hope folks start trusting the source more.
You can take a photograph of the image with an old Polaroid and, if the resolution is high enough, the watermark would still be there.
Let's now let's say your AI deepfake image is watermarked/hashed upon generation. Difficulty: Intermediate to hard. You might be able to get it done, but not without real effort, and removing all traces of the watermark without leaving artefacts might be so difficult as to be nearly impossible.
...So it doesn't eliminate the possibility of fakes, but it hugely raises the cost and effort bar. Only the most motivated would bother.
I tried some "AI watermark removers" on a Shutterstock image, and they did a pretty good job, but they left traces that would be obvious to any investigator.
The problem then is repeated compressing causing that to occur inadvertently
However, the real breaking point in my view was when shills and semi-automated bots become so prevalent that they could fool people into believing some consensus had changed. Faking photos doesn't add much to that in my view.
What we don’t know is whether we’ll be worse or better off when the technology of forgery is available to random broke assholes as easily as it is to governments and companies. More slop seems bad, but if our immunity against bullshit improves, people might redevelop critical thinking skills and capitalism could end.
Does anyone other than me notice this common tendency on HN:
1. Blockhain use case mentioned? Someone must say "blockchain doesn't solve any problems" no matter what, and always ignoring any actual substance of what's being done until pressed.
2. AI issue mentioned? Someone must say "nothing at all has changed, this problem has always existed" downplaying the vast leaps forward in scale, and quality of output, ignoring any details until pressed.
It's like when people feel the need to preface "there is nothing wrong with capitalism, but" before critiquing capitalism. You will not criticize the profit.
It's not really a shibboleth. What's the name for this type of thing, groupthink?
EDIT: while oversimplification is essentially always a problem, nuance and persuasion are usually at odds. So, it's especially noticeable in contexts where people are trying to persuade each other. The best parts of HN are not that, but rather where they try to inform each other.
At least made up citations are quick and easy to denounce.
https://www.independent.co.uk/news/science/archaeology/europ...
"There was a brief window where photography and videos became widespread so events could be documented and evidenced."
Photos are video have in themself not been evidence. You need trust the photographer or publisher too, since the camera was invented.
Moving the cost from big $$$ level to my neighbour level makes scams more personalized, sure.
Localised fires are common in nature, a massive wildfire is just a fire, at a different scale and degree. Lunatics raving about conspiracies were very common in public squares, in front of metro stations, anywhere there was a large-ish flow of people, now they are very common in social media, reaching millions in an instant, different scale and degree.
Just sheer scale and degree makes an issue a completely other issue, decreasing the effort required to fake a video to the point where a naïve/layperson cannot distinguish it from a real one is a massive difference. Before you needed to be technically proficient with a number of tools, put a lot of work, and get a somewhat satisfying result to fake something, now you just throw a magical incantation of words and it spits out a video you can deceive someone, it's a completely different issue.
Which just goes to show that one of the core tenets of techno-optimism is a lie.
The Amish (as I understand them) actually have the right idea: instead of wantonly adopting every technology that comes along, they assess how each one affects their values and culture, and pick the ones that help and reject the ones that harm.
Not something that can be said of most people. Worse, the number of affinity groups with long term coherence collapses into niche religions and regional cults.
There’s no free lunch
If you want structural stability then you’re gonna have to give up individuality for the group vector, if you want individuality you’re not going be able to take advantage of group benefits.
Humans are never going to be able to figure out this balance because we don’t have any kind of foundational coherent Universal epistemological grounding that can be universally accepted.
Good luck trying to get the world to even agree on the age of the earth