Especially for public figures with lots training data available.
Especially for public figures with lots training data available.
This stuff even has the catchy name "deep fake".
Initially, every Tom, Dick & Harry script kiddie will try to blackmail people whilst the public and justice system is not really aware of the technology possible.
When eventually enough of the public is made aware to not trust photos and videos anymore then these types of blackmail attempts would be less effective. But there will still be less informed gullible targets.
The blackmail angle I expect will work for a while. People falsely accused of sexual crimes can have their lives ruined by the accusation.
But as you say the media (real and social), political propaganda, etc will exploit the uninformed masses for all it is worth, so blackmail will still work. For a while.
The only way I see it working is if the key is "burnt-in" to in the camera hardware and any applications cannot MitM it.
> the app itself could take a deepfake video and have the OS sign it
Note how I said "it asks the user via the OS to approve the result". I would expect a modal OS dialog to let the user review and approve the content before being signed and passed back to the app.
Thinking about it, there's actually nothing stopping this from happening on today's hardware using just application sandboxing. Substitute "OS" above with "Signing App" that does the same thing (accepts media signature requests from other apps, and opens dialog to request approval from user with a preview).
There's been r/deepfakes where around 30% of the content was porn with swapped faces. It was banned though.
Fake celebrity porn has been on the internet since 1996 at the very least. It's always been crummy; but porn in general requires a thick suspension of disbelief and an intense focus in a partial object of desire (what Lacan calls the objet petit a) that blurs everything else.
That's not all you would need for verification, but it is a big help.