Ctrl Shift Face: A growing ‘deepfakes’ YouTube channel
theregister.co.uk
theregister.co.uk
Upload your content, request a key you can sign with, the OA checks against previously registered content in it's database for originality (I can only assume this would be as buggy as Youtube's algorithm is today).
In order to deepfake it, a neural net would have to understand the crypto algo in such a way it could modify its deepfake input to collide with the original input's hash. Basically it would need to reverse the crypto to generate all frames that evaluate to a certain hash, and then find one that is mostly similar to the intended input.
I don't think it will be realistic at all, if even theoretically possible.
And that's assuming it wouldn't be somewhat trivial to dump keys from one or more cameras on the market to create a an easy tool or resignvideo.com... which is a massive assumption.
As the movie industry has already seen, cryptography doesn't help you when you push out the keys to everyone. It'd be about as effective as DRM is at stopping video piracy.
If you wanted to do the cheap version of this you could probably set up a screen with proper lighting such that it'd line up well enough that screen and sensor pixels matched closely making it near if not completely impossible to tell a screen was involved.
Worse, such a signature on a fake video may give people false confidence in it being real, when really it still needs scrutiny especially if a possible state level or dedicated attacker is involved.
You could make things more difficult by using multiple cameras (and perhaps some other sensors) to detect a 3D scene rather than just a 2D representation, but I agree that it's all ultimately fakeable.
The problem isn't that creators can make fake videos (which would be analogous to DRM). Let them. There is no harm in it. The problem is that users have no way of verifying true videos. This can be solved with cryptography, because both parties involved (creators and viewers) would want this to work.
In other words, DRM is about forcing everyone to do a thing. Verification is about allowing willing participants to check a thing. Different scenarios.
Companies could make tamper-proof sensor modules that do the signing right where it matters. The important thing is to make sure they aren't astronomically priced and don't compromise privacy.
As far as I'm concerned, the need to use a real camera to capture real light is sufficient to thwart creation of truly convincing fakes.
Yes. It used to be the industry standard method for recording TV programmes prior to the invention of videotape recorders. If you know what you're doing and use the right equipment, the generational quality loss is very minor.
Back when everything was analog and low-resolution, had noise and artifacts anyway, sure. Do you have a video that compares modern digital 1080p with its best "Kinescoped" version? You have to deal with parallax, mismatched frame frequencies, pixels and differences between natural light and whatever spectrum the source monitor produces. I have never seen a recording of a screen that looked anywhere close in quality to the original in 1080p.
If it gets it to the point where it takes years of experience with this to figure out what is and isn't fake... I doubt that's any better than the situation without such a video verification system at all. In fact, I'd argue it's worse due to the fake seal of approval being given to the video.
As an informal comparison, check out a downloaded "cam version" of any recent movie compared with a DVD rip. If you ignore the occasional noise or interference from the crowd, the quality can be quite good.
Deepfake makes it less expensive to create fake video of someone, special signing cameras could make it expensive again in the face of that.
Let's say I want to create and circulate a fake sex tape featuring a senior politician. In a world where signed sensors are common but not mandatory, the absence of a signature says nothing at all about the integrity of my video - there are still millions of cameras in the world that produce unsigned files. In a world where they are mandatory, I can simply fake the video in software and then re-record it using a high-resolution display and a signed sensor. A digital video expert might be able to prove that such a re-recording process took place, but I highly doubt that they'd be able to say so with any degree of confidence if there's a bunch of sensor noise and a couple of rounds of low-bitrate transcoding.
Signed sensors won't do anything to undermine the credibility of fake videos, but they may have the opposite effect if they're hacked - I can "prove" that my deepfake is genuine, because I've extracted a signing key or injected a bitstream into a camera module.
https://www.forbes.com/sites/kalevleetaru/2018/09/09/why-dig...
If we're at the point of trusting videos only from select providers, well, that's not really much of an improvement over just finding the video on that provider's website over HTTPS.
Because the future us will be conditioned to do so by our cryptographically integrated devices. There'll be so much fake video out there that we'll get accustomed to seeing it on our devices with the big huge "fake" sign flashing on it.
(Or, again, even worse, it will not have the "real" sign flashing on it that the video from validated publishers and cameras will have.)
We can be conditioned to do this extremely quickly. Again, ActiveX is what made all the plugin security warnings start popping up everywhere. But after a while, plugins like flash and unity even fell to the reality of us having been conditioned to click "Do Not Run Plugin" whenever we saw that warning. Are we better off without flash and unity? Probably, but that's beside the point. The point is that masses can be conditioned to implement their own control when you set up a system where only data meeting a certain standard is accepted. And it's a hardware standard. Consider very carefully the position over us that such a system would put hardware makers and those with power over hardware makers in.
Now again, I'm not arguing that fakes are not a problem. They are indeed a problem. Just saying we should go about solving the problem with extreme care and caution.
That's my point.
Going down the hardware cryptography path certainly does solve the problem. I'm not here to try and claim that it doesn't. The issue is, 10 or 20 years down the road, how have humans been conditioned because of the fact that only certain cameras can be validated to be real?
If cryptography takes over, 20 years from now, no one is going to convict any guy of rape because they see him on a non-signed camera video, or a non-signed CCTV feed. (And don't get me wrong, they shouldn't convict the guy if that's the so called evidence the prosecutor has.)
What I'm asking is that everyone take a deep breath, slow down for a minute, and just think about the implications of that.
And I was being generous in the scenario I laid out here. In reality, I don't believe it would take 20 years to condition us to dismiss non-signed video out of hand.
I think that's the end goal of all the "deep fake" hysteria we're being primed with. In the future if some damning video of a politician or other elite member of society pops up the defense will simply be "well you can't rule out that it's a fake, look, it's not signed and it's not in our database of trusted videos!"
related: It's telling that "factchecking" isn't a dialogue between various authorities, but instead just a bunch of weird private judicial systems with no appeals courts, selected and assigned to specific stories by internet aggregation intermediaries. Eventually, this can't help but shake out as stories being divided into two categories: factual and banned from distribution.
I sadly feel that we're about 10 years away from licensing journalists with a security clearance-type process that must be sponsored by an approved outlet. The PropOrNot at the Post shook me a bit.
Audio and video were sort of aberrant in fact that they could still generally be accepted at face value (as compared to quotes and attributions) and we're very quickly losing that stability.
ie - A world where so much fake image data has flooded into the zeitgeist that people only believe the feeds that are announced to be "real" by their cryptographically secure devices. They can play the fake video, even play video that may not be fake, but was not cryptographically validated. Their devices will just be flashing a huge warning sign right next to each such image feed. (Or even worse, it will only flash a "real" sign next to the image feeds it and the central have concerted to cryptographically validate.) It'd be kind of like the huge warning dialogs that always popped up at a certain point when people went somewhere with an ActiveX plugin in the page. After a while, it wasn't just ActiveX, it was everything. People just got conditioned to believing that every plugin, no matter the source, was horribly bad for them. (Every plugin probably was, but that's a separate issue.)
Only this time, the powers that be will be the only ones capable of delivering data that is guaranteed to be pure. Everything else will be either, not validated by their cryptography standard. Transcoded, (and consequently the masses will dismiss it out of hand). Or out and out fake, (which, yes, the masses should dismiss out of hand. I get that).
But I'm just saying that people should think a little further ahead here when they are trying to solve this problem. If we want to use cryptography to solve this problem, that's perfectly reasonable. (And yes, it would work. It would work only too well, and that's the entire problem.) I'm only saying that we should do what we can to make sure that cryptographically validated image feeds are not uniquely identifiable, for instance. Because when you go down this road you need to realize that with hardware cryptography, the hardware makers and those with power over them become de facto arbiters. And they can potentially use distribution to control the masses. Because they have the only pure sources. They could potentially even use attribution to control the masses. (You sure you want to video tape that police beating and mail it to the media "anonymously" with one of these devices?)
I'm just saying. Stop and think it through a bit.
And having a plan based on, "Oh... We're smart enough to get around it!" Is a very foolish plan for two reasons. First, because we won't be able to get around it, and second because it's the same as having no plan at all.
We already live in the word where a video is not a 100% proof that something happened. Given enough time and money you can use video editing, actors and CGI do show almost anything. This does not, however, completely invalidate all videos as evidence, because the cost of faking something is fairly high.
Deep fakes can potentially decrease the cost of impersonation of someone in any video to almost nothing.
To mitigate that we can introduce additional technologies that will bring the cost of video impersonation back up, effectively restoring the current status quo.
Like most security, it's all about the costs involved in attacks and mitigations. I remember the time where most of HN visitors understood this very well.
Can you explain what you mean as the distinction between "content creators" and "some arbitrary video online" if the same technology is used to "authenticate" both?
> Like most security, it's all about the costs involved in attacks and mitigations.
Ignoring the fact that this could likely be done fairly cheaply, unlike other security improvements this actually weakens the next layer down substantially, faked content that does beat the system introduces a new seal of approval on that content. I don't want to see people get convicted because footage that's later found out to be faked has a green checkmark when the jury watches it.
Please don't respond to swipes with more swipes. Your comment would be just fine without that bit.
How do you verify it's real? You can't - because the camera of the guy that published it says it's not modified, but it could be just as well a camera recording a display with the fake playing.
Seeing footage literally taped by a phone off a NVR display is pretty common...
Would you please edit swipes like that (and "you must live in a parallel universe, to put it politely") out of your comments here? They break the guidelines, lower the signal/noise ratio, and invite more of the same from others. Your comments would be just fine without those bits.
Just film a screen displaying the fake video. As soon as it's possible for a bad actor to create signed fake content it invalidates the entire protocol.
Oh, ok, I see what you mean now. The viewer already trusts the creators, so the threat model is not about an attacker creating fake videos. But if you already trust the creator why do you need further proof? Identity is already solved. The problem at stake here is videos released to the general public where the source is unknown, or not trusted by all stakeholders, i.e, during a trial.
This is fundamentally just another MITM attack.
https://inis.iaea.org/collection/NCLCollectionStore/_Public/...
I remember visiting the MIT media lab when I was in college, they described the audio version of "invisible watermarking" - manipulating the source audio with a generated acoustic effect, similar to a doppler shift. The effect is not detected by the human ear, but signal analysis can translate it into binary metadata, such as a DRM signature.
If you're saying it'd be added by legitimate camera sensors, well it wouldn't beat the possibilities I suggested already of simply feeding tampered data to the sensor.
You have to get all video editing software or displays to inject such a watermark while keeping their keys private. Getting direct LVDS access to a panel and being able to write your own firmware pretty much bypasses most display options, and well... software based options are even worse I'd say, software is open to such a wide array of attacks and the mere existence of open source video editing tools, including FaceSwap itself makes this seem... questionable at best.
When I watch a video-clip of, let's say a press-conference, it would be nice to have a way to verify that the video-clip is indeed the one recorded by whatever journalist was at the press-conference.
It's not about camera hardware, it's about making it easy to check if the video someone is presenting to me is what they claim it is.
What you describe has always been possible, but it seems rare.
And as pointed out by another comment, you'd get a modified camera and feed it into the sensor, or modify the firmware. And some guy in China will do a clean-room RE and make little video signers.
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything.
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything. Also, most of us complain about code signing when it affects us; why would this be different? Fred the Blogger can't get an "official video signing certificate", which is probably what we would end up with. Or better yet, get the government involved and create a Bureau of Trustworthy Video Sources!
For a generally positive example of this, see a good chunk of the history of Silicon Valley with smaller tech companies displacing larger companies like IBM, because there was no regulation stopping it. For an example generally less loved around here, consider Uber making a good pass at eating the entire Taxi industry by virtue of dodging regulations.
Maybe that would have some influence about how seriously they take takedown requests.
"Speaker Nancy Pelosi on Wednesday strongly rebuked Facebook, saying the company’s refusal to take down altered videos of her demonstrated how the social network contributed to misinformation"
From the New Yorker article I read:
"Truepic, a startup in San Diego, aims at producing a new kind of photograph—a verifiable digital original. Photographs taken with its smartphone app are uploaded to its servers, where they enter a kind of cryptographic lockbox. “We make sure the image hasn’t been manipulated in transit,” Jeffrey McGregor, the company’s C.E.O., explained. “We look at geolocation data, at the nearby cell towers, at the barometric-pressure sensor on the phone, and verify that everything matches. We run the photo through a bunch of computer-vision tests.” If the image passes muster, it’s entered into the Bitcoin and Ethereum blockchain. From then on, it can be shared on a special Web page that verifies its authenticity. Today, Truepic’s biggest clients are insurance companies, which allow policyholders to take verified photographs of their flooded basements or broken windshields. The software has also been used by N.G.O.s to document human-rights violations, and by workers at a construction company in Kazakhstan, who take “verified selfies” as a means of clocking in and out. “Our goal is to expand into industries where there’s a ‘trust gap,’ ” McGregor said: property rentals, online dating. Eventually, he hopes to integrate his software into camera components, so that “verification can begin the moment photons enter the lens.” [1]
[1] https://www.newyorker.com/magazine/2018/11/12/in-the-age-of-...
Occasionally a security minded computer savy person might pull one over on the insurance company - but most clients aren't going to have the expertise or dedication to correctness. To contrast that - the pool of people who would try and produce a deep fake, especially in the political realm, have already decided to invest a good amount of investigation and effort and are likely rather technically minded.
So this would stop the script-kiddie equivalent actor in the deep fake realm, but I am more concerned about dedicated nefarious actors.
This is leading society to a very dangerous place where the truth is becoming a matter of opinion.
The Bard is back again!
"What if the government can't determine if a 100 dollar bill is fake?"
Answer to both questions: those who have a vested interest in detecting fakes spend more money on fake-detection mechanisms.
Yes, this is utter BS, sorry.
I am curious to know more about this space because it's all very new to me. What failed solutions were tried in the past? Which issues does Truepic fail to address? And is it even possible to address these failures?
I genuinely thought "wow, when he squints his eyes he really looks like Arnold". The transitions between Bill's own face and Arnold's were totally seemless to me.
I finally realized what was going on when I saw the channel's name, and it spooked me that I had fallen for it.
my experience, exactly
Just take a sample of your close friends or relatives. How many of them can tell 48fps video apart from 24fps? How many people leave Smooth Motion turned on on their TV's without even noticing it?
Hell, I remember people telling me that they could not tell the difference between Blu-ray and DVD when Blu-ray came out. Even in this forum, a lot of people have claimed that there's little difference between a 1080p monitor and a 4k monitor scaled at 200% (Retina). It's 2019. 1080p screens are completely unsuitable for looking at text, and yet many people can barely notice the difference.
That's what we have to worry about. The tech is good enough to weaponize right now. And I see no way out of it, except with education. Perhaps high schools should eventually include a visual and aural literacy course as part of the curriculum.
When we have deepfake video _and_ deepfake audio we'll be in trouble. At least for moving images.
These authors worked/work at baidu.
https://www.vice.com/en_us/article/3k7mgn/baidu-deep-voice-s...
- Secure Data at the Source -
Public Created - I have seen some demo's of phone apps that put video and pictures from phone directly on the blockchain with gps, time stamps etc.. Also have heard of dongles being added to phones
Professionally Created at Device - have read about hardware getting processors to secure data directly on the blockchain at the device.
Created at Editing Software - This could be connected to the identity of the original on the blockchain and then the edited version has notes with it and added to the blockchain as an edited version of the first.
Security cameras for Customs and Border Patrol are getting blockchain capabilities added per this quote - " Factom (blockchain) has integrated their technology into two brands of cameras used by CBP at border locations to ensure data collected from these cameras is tamper proof" - http://www.activecyber.net/dhs-st-lays-out-a-broad-yet-innov...
IoT Devices - This micro processor can be added to an IoT (raspberry pi and the like) device and secure data the source - https://iot-sas.tech/
Professionals (media) and citizens can then have proof of what and when they created the video and potential a certification can be added in manor and tool for others to check the video is in fact the original.
-- Identify Fakes -- Two articles from universities working on it https://jsis.washington.edu/news/deep-fakes-fake-news-and-wh...
https://medium.com/jsk-class-of-2019/six-lessons-from-my-dee...
None of the above completely solves the issue but with implementing these and having standards the public can start to trust what they are watching is authentic.
1. If the image quality is good and if you pause the video, the artifacts are pretty obvious.
2. Making meme videos is not a problem. You have to think about serious scenarios and ask yourself whether they were impossible before. In a lot of cases they were possible before without using AI. People edited interviews and staged recorded events since forever.
That doesn't mean everybody is.
See the problem?