Hacker News Reader

Comment by fredley | Hacker News Reader

By the time the next US election comes around it's very likely this technology, combined with Lyrebird etc. will allow anyone to make [politician] convincingly say or do anything they like.

I can't make my mind up if this will make the current fake news / filter bubble situation worse, or if it will make it so bad that people learn to distrust any kind of digital media entirely.

I'm still not sure, given that the use of these tools for political gain seems inevitable at this point how we will actually verify a video is real in the near future.

View on HN

I find the more concerning thing is not that people will believe that a fake is real, it's more that they won't believe something real is real anymore.

Already nowadays the whole thing about fake news and alternative facts has segments of the population flat out not believing the truth. This may very well be greatly exacerbated in the future where, regardless of the amount of evidence you bring, people will just disregard it as fake.

If it can be faked, is it really evidence? I mean, you're worrying about people not believing assertions at that point. This is already something that can be faked just not by the masses yet and not easily. You should always look for multiple sources of anything and not just believe one source that shows you any evidence of something being true.

There was this essay posted on HN some time back, where the main thesis was that teaching people to be critical about media strengthened the positions of antivaxers, flatearthers and so on, because they did not rely on traditional media so they were then sort of considered more trustworthy. And now deepfakes etc throws just more fuel in the fire that burns trust, people end up more and more believing things that happen to make them feel good.

Believing 'the truth' is a huge can of worms here.

Verifying the legitimacy and accuracy of 'news' claims and representations has always been very difficult for the lay person. We are now well into an era when 'the truth' is even harder to find.

'Believe nothing of what you hear and only half of what you see' (Edgar Allen Poe) has now become 'believe nothing of what you see' too, at least until you have verified to your level of satisfaction...

We need to start thinking about what criminal justice will look like when digital recordings are all considered hearsay. That's bound to happen.

And perhaps it should... if average people have the technology now, how long has it been in the hands of police departments, newspapers, militaries, and intelligence agencies?

We tend to focus on fakes with high-profile people like politicians and celebrities, but it can clearly be done with everyday people as well. The author was able to produce a fake video segment of a well-lit, animated conversation involving his wife. She's directly in front of the camera, speaking and making hand motions for several seconds. How much easier would it be to insert a face into grainy security camera footage, or a shaky clip of a large crowd at a political rally?

The presence of one source can be easily dismissed, but if you have 500 independent sources that produce video of the same event from their own angles and timespans in an interconsistent way, it's far more likely that the recorded event happened as opposed to all of these different sources lying.

Really? How much harder is it to run the OP's script with 500 input videos as opposed to only one? Especially if they're very similar? How many people have the expertise needed to identify a faked video, and how much training would they need?

The great part is that using a GAN means we cannot automatically distinguish a fake from a real... because that's the whole point of the GAN.

At that point there is no such thing as identifying faked videos. Instead you'd rely on the abundance of independent videos and see how consistent they are. Generating a lot of them doesn't do you any good unless you can insert them into the black boxes of the 3 planes and 40 cars in the area, the databases of all the drone operators, the internal storage of all the nearby smartphones, etc.

Ahh you are undervaluing the power of automation.

With a few sources you can build an accurate single model of an event. You then edit that event as you see fit and then generate as many "independent" sources from different viewing locations as you want. Then upload them to different social media sites at different times.

I am sure you can do that. The hard part would be building consistent histories to back up those uploads. An upload from my facebook account, verified by other incidental data like GPS tracking on my personal phone, would more merit treatment as an independent corroboration, whereas a random reddit upload from a throwaway account would not.

And the real verification would come from ultra-high-res captures from drone cams, self-driving car cams, traffic cams, etc that are constantly running and recording everything, so that you can't realistically fake soemthing without simultaneously compromising uber, amazon, walmart, etc.

> but if you have 500 independent sources that produce video of the same event

This only works in large events. What if you film a politician lying at a small event, but his image team makes 20 fake videos showing the event in a different light? I guess you're the liar then.

Firstly, people already know politicians lie. Nobody is going to care about your video in the first place.

Secondly, your analogy is wrong. The new hypothetical you present is equivalent to the old model where you are all giving verbal accounts and his staffers lied. If you are going to compare to an old-tech scenario where you one-up them with a more detailed recording than they have, you need to do the same with new technology to keep the comparison accurate.

Thirdly, why aren't you the liar?

Fourthly, you can always go one level up. Show the footage of them constructing doctored footage.

The difference is with testimony, the witness can be as moral and honest as possible but still miss report what they heard. With faking audio, it is either correct or someone purposely tried to fake evidence. That means that if the source of a recording could be verified, we know that it is very close to true. With witnesses, there is always doubt.

You're right to be skeptical of witnesses, but recordings are also not either/or. Even if you have some sort of digital signature scheme such that a camera can somehow sign its footage and the time and location it was recorded (a scheme that currently is fairly uncommon), how could it encode the circumstance in which the footage was recorded? Who is responsible for maintaining the signature scheme, and why can we trust them? In the absence of any aspect of that scheme, why shouldn't we closely scrutinize video and audio evidence as much as we scrutinize witness testimony?

The thing is that we have to believe someone when trying to understand what happened. A person, no matter how honest, can make mistakes. With a recording all that is necessary is to make sure that it is honest.

> if average people have the technology now, how long has it been in the hands of police departments, newspapers, militaries, and intelligence agencies?

Most of those people are average. Intelligence agencies can be assumed to have more advanced technologies in a number of areas but the gap between what they have and what's out in the more public world has narrowed.

We've already had image tampering technologies for a long time but there's not been a significant problem with fakes in the press or as evidence.

It is an important issue anyway and figuring out ways to being able to differentiate the genuine from the fake, with cryptographic signing in cameras for example, are worthwhile.

Ultimately, though, don't you have the same problem with things like contracts? Especially with digital signatures, but there is not actually a reliable way to determine whether a signature is authentic. And countless people are going to jail because of complete junk science like bite mark analysis. This seems like the least of our problem.s

Not much will change. People have been able to convincingly fake still photos almost since cameras were invented, yet courts accept some photos as evidence. What matters is the provenance, chain of evidence, and testimony of witnesses about the validity of the photo. Videos will work the same way.

> And perhaps it should... if average people have the technology now, how long has it been in the hands of police departments, newspapers, militaries, and intelligence agencies?

To be honest with you, when I was younger I bought into this.

Nowadays....it kinda seems like with the open source community, consumers are getting stuff first. Not always, of course, but often enough.

And police departments are one category that I can be quite certain are lagging behind, they are definitely not getting stuff before consumers have it.

You're right, lumping police departments and newspapers in with intelligence agencies is pretty silly.

The longer time goes on, the more you realize that what companies can get to the public in a year for $1000, takes 10 years and a multi-billion dollar contract for any government (other then priority bureaus like NSA) to obtain. Like good luck getting anything through government bureaucracy and shitty subcontractors.

Yes. As a grad student in ML, on a few occasions I worked with sponsors under the US military trying to help them use some code from projects I'd open sourced, and it was always a struggle.

Frankly, the suggestion that government agencies -- let alone local police departments -- have sophistication in AI/ML years ahead of publicly known work from academia, industry, or even side projects on reddit, will verge on comical to anyone who's actually worked with these folks.

You're not talking about a monolith. Sure, if you've met 20 cops, it's unlikely that you've met any cops who are technological geniuses. Most LEOs haven't had a reason compelling enough to figure out how to fake a video. Can we be sure that no LEOs have? If we can be sure of that, can we also be sure that no intelligence agencies have? That seems unlikely.

Or to say it another way.

Only a few cops will have the ability to edit videos, the problem is almost all cops in his department will lie for that officer on the stand.

Then defendant could make his/her point by doing the same but with the face of the judge or prosecutor.

It's time for cryptographic signatures in cameras (phones and otherwise) so that you can prove something is unadulterated.

Apple signing key -> your phone's key -> signature over hash of the media (edit: for video, embed signatures every so often to you can verify a shorter clip out of the whole). Embed Apple's signature and your pubkey in the metadata of the image, and TwitYouFace will verify on upload that it's authentic.

Of course, you can't fix stupid unless you talk about a complete overhaul of public education... And even then, people are people.

That doesn’t prove anything. Make your fake video, point your phone camera at it, record.

Maybe a second layer of AI that cleans up that video-of-video look?

Philosophically though, I think it's a good thing that many are learning not to trust information from self-proclaimed authorities :-)

So what do you propose? At some point a system of trust must be established or are you going to travel around the world, attend every event, see everything with your own very eyes, run the analysis, crunch the data etc... to be convinced of the information?

It's possible to simply believe less. Not only will you be less likely to get screwed over, you'll also be happier, when you no longer have a big handle on your nose that the bastards can use to lead you around.

The "just sign the photos" line of thought fails in exactly the same way DRM does, yet its brought up every time this issue is raised on HN...

Hasn't ios DRM continued to get more effective over time?

If there are no known exploits, a signed video, marked as having a network based timestamp, is at a minimum incrementally more reliable than a video without such metadata.

The phone has enough metadata to make that easy to expose, at least in the simple case. There's GPS location, time, and even your Face ID verification (on iPhone X) to prove that it was really you holding the camera.

If you make your fake video after the event, the location and time will be obviously off.

Which are all values written by the camera. My camera will believe it's whatever time, location, and face I tell it.

The point by clarkmoody was that Apple would be signing the photos with their key to ensure those values are correct.

It would be basically the macOS Developer ID software model applied to photos.

You can set the time on your phone, and GPS spoofing is a technology that already exists -- or you could disable location services.

https://en.wikipedia.org/wiki/Spoofing_attack#GPS_spoofing

Cryptographic timestamps are a thing

Try it. Seriously, try and record a realistically looking recording off the screen. Movie pirates have been trying to perfect it for decades, and even their best attempts look obvious.

Unless you build DRM right into camera sensor (which is, admittedly, not that hard), you can use an FPGA chip to receive your source video in HDMI format, convert it to CSI interface and send it to the phone SoC over the camera flex cable.

If you can make a porn video of someone look like a porn video of someone else, I’m sure the technology to solve this problem is not far off.

That's fine, if you publish that video it will be signed by ggg9990 and you will have to decide if you want to be associated with it. Watchers will then have to decide if ggg9990 is a trustworthy source.

It boils down to trust, not whether the original video was real or not.

To put it another way, reputable news organizations already go to great lengths to ensure that their stories are well sourced. At the end of the day, they are putting their reputation on the line.

What if all legally acceptable cameras have to use multiple sensors?

In other words, 2 main stereoscopic sensors, each recording their cryptostream to their storage. In addition to that, you have 2 UV sensors doing the same thing, and 2 IR sensors as well?

The idea being that it's much, much harder to fake something if you have to take IR and UV into account.

Isn't this also where something like a blockchain is useful? Where data is dumped into it realtime by this mythical secure camera, with all its associated metadata.

Then you can ask: "does the time the data was added agree with the time the camera says the data was recorded?" "Does the blockchain agree that the data hasn't been modified after it arrived?"

Add to that the possibility that there are multiple cameras all recording this incredibly detailed video from different angles, and the evil counterfeiters will (hopefully) have a dickens of a time faking all this.

Although with all this wonderful processing, networking and storage power to enable this secure camera, you can probably create a faking system too.

This is how things have to proceed for sure.

It's time for cryptographic signatures in cameras (phones and otherwise) so that you can prove something is unadulterated.

The otherwise is already covered by DSLRs. Canon has it, and I'm sure Nikon does. Law enforcement has been using this stuff for years.

Is there a way to enable a keysigning in my Canon SLR? How do I install my private key?

No, it's a hardware module that you connect to the camera.

http://www.canon.co.jp/imaging/osk/osk-e3/index.html

Apparently it was broken in 2010 (found while googling for this).

This is at the same time a great solution, and a useless one. Sure, you could verify that a video came from device X by carefully comparing the signature of the video was signed by the public key of the phone, but who on earth is going to do that for a video their aunt posted on Facebook?

It would absolutely have to become more consumer and lay-user friendly.

Of course, depending on how that's implemented it could just be another vulnerability in easy-to-use proofing.

But if there was some way to add an abstraction on top of the raw keys, keeping them still accessible if the user is keen, then that might be a step in the right direction.

You're missing the point.

Facebook / Twitter / etc verify the crypto signatures on media ingress. Then they can give proper attribution and show a checkmark or something that attests the media hasn't been altered.

So in your feed, there will be some media with checkmarks and some without and you can choose which to believe more.

Alternatively, a competitor could come along and reject all media that doesn't pass cryptographic verification.

That’s a big challenge: for big news stories it’d be useful if it meant that TruePatriot420 has to explain how their phone was used to record a candidate’s private conversation in a different state but I think many of the most damaging things for many victims won’t hit that level of scrutiny. If some horrible person tries to smear their ex, most of the damage will come simply from circulation.

What might be worth trying would be some sort of verified badge if this became common enough that players could meaningfully highlight its absence, similar to the way browsers are now showing non-HTTPS sites as insecure. If Facebook badged the video with a big “this iPhone video doesn’t have the usual signature” message that might slow people down.

For the deepfake porn or fake news problem I also wonder whether there’s a useful fuzzy ID service where a player could display a “this is substantially similar to <source video> and may be a copy” warning which wouldn’t be so inaccurate that nobody would heed it. That could be really important if we started seeing political fakes where someone is shown accurately but their words are faked.

Your fears are greatly overblown. Haven’t we been able to “Photoshop“ people convincingly for a very long time? People are less sceptible to this than you think.

Even when scandals do break, it’s not a picture or video that’s proof, it’s a picture or video that produces witnesses.

Video has been pretty much immune to "Photoshopping" AFAIK. The new technology also makes it extremely cheap.

nein.

or there was that time a german variety show 'false flagged' the germans by doctoring a video of yanis varoufakis to have him 'stick the middle finger to germany' during a talk he gave (https://www.youtube.com/watch?v=Vx-1LQu6mAE). There were witnesses; but they didn't stop the confusion and outrage that ensued due to the effects of viral media. Would this pass a sophisticated technical analysis of the video? No - is there time and public understanding for the results of said sophisticated technical analysis to prevent the damage before it is done? no.

The notion that witnesses or forensic analysis can combat the speed of information, especially across borders, is a bit quaint.

Its something entirely different to fake a face instead of a finger in a video. Then again, the deep fakes I have seen so far arent really that convincing either. But one can only wonder how much you can improve such a thing. With politicians of whom a wide array of video material exists and were you have the option to create an incident, instead of altering existing ones, the possibilities are really interesting.

did we really post that link at the same time, or did you edit your other response to match?

I posted the Böhmermann incident 2 minutes prior and see it more as an example for the state of the art in video manipulation. I think its hard to see the response back then as an example of future interaction with the topic of video fakes. Which when it comes to altering faces in videos is still in the future.

I edited the post you replied to with "in a video." in the first sentence to clearify

His response was generated by a GAN.

Could you elaborate? I dont understand what you are talking about

It's ironic that on a discussion about copying images of parts of someone's body over to images of another person's platz would ask "did we really post that link at the same time, or did you edit your other response to match?" i.e. copying over parts of one person's comment to another person's.

(And the GAN is the neural network model which does these transformations)

"Falsehood flies, and the Truth comes limping after it."

—Jonathan Swift

https://quoteinvestigator.com/2014/07/13/truth/

It's of course technically possible but it's also much more difficult. You basically have to photoshop a long series of frames and make sure that it looks natural and consistent in sequence. You have to take into account changes in lighting, focus, motion blur, things passing in front and behind the object etc...

In your example they modified a very clean source video, image is perfectly stable, very little motion and they edited a very small portion of it for a very short duration. TFA is about replacing one person by an other while they're talking and moving on a potentially lengthy video clip. Doing this convincingly by hand would be hard work.

I think a good example of the difficulty is with the infamous "moustache removal" for reshoots of Superman in the Justice League. It cost a fortune and the results were not exactly seamless. Also the re-created Princess Leia in that Star Wars movie, not bad but definitely still in the uncanny valley.

We've always had AfterEffects and CGI. Some viral videos have been proven fake , but for the most part a lot of those faked videos were low res to make it more forgiving to fake them.

Captain disillusion on youtube has a great channel covering all of this. Here's an example of a fake lightning video https://www.youtube.com/watch?v=zhPRtCW5sRk

Deepfakes has made it so the average user, with no knowledge of Aftereffects / composition, to basically run a python script so long as they have high res images + several angles / expressions of the target user. That same folder set of images can be replicated in multiple scenes with just the cost being GPU rendering time

Deepfake videos can also have a much higher resolution than "viral videos" with aftereffects to fake effects.

Deepfaking also leads to a lot of legality issues as well.

TL;DR deepfakes does make faking videos much cheaper, higher resolution, and easier now

My point was that they don’t lead to legality issues. This has been the case since the invention of the digital camera. No matter how convincing digital images (now video) has seemed, they aren’t “proof” in the eyes of the court because they can be faked. There’s nothing new here.

How do you think cgi works?

There was even a rather convincing incident 2 years ago where a Jan Böhmermann may or may not have faked Varoufakis "giving Germany the finger"

https://www.theguardian.com/world/video/2015/mar/17/yanis-va...

https://youtu.be/Vx-1LQu6mAE?t=127

Youtube Video with subtitles with Böhmermann.

The joke was that the version with the middle finger was the real one, and they faked the "original". This created a lot of confusion :)

This example below of citizen journalism (written about in south african mainstream media here) is an everyday occurrence.

https://www.sapeople.com/2015/06/19/fake-images-claim-photos...

Someone finds very crude photoshop work that undermines the 'official' version of a news story.

> or if it will make it so bad that people learn to distrust any kind of digital media entirely.

There are already fake news sites out there that are "so bad" you'd think nobody would believe them, and yet...

Still, it's important we inform people this is possible, so that at least the more reasonable among them can view future videos through a skeptical lens. I'm sure by now many HN readers are not surprised by this technology, but many outside of HN would be blown away.

Herd immunity? Perhaps naive people believe those spam sites because smarter people believe the better spam sites that share the same narrative...

However, distrusting digital media isn't the solution either - there are plenty of trash magazines publishing conspiracy theories and other absurdities. Plus, digital media has revealed huge segments of America are becoming culturally isolated and undemocratic. A return to print media might shift funds back to deep professional investigative journalism that covers those issues... or just give owners better returns.

Next election the "grab her by the p*" soundbite would have been "fake sound".

It's really two problems. One, making people say things they didn't say. And two, allowing people to deny things they really did say.

The second probably being much more damaging.

> Next election the "grab her by the p*" soundbite would have been "fake sound".

You mean the last election right? A lot of people believed it was faked.

Well those people cannot be helped, he admitted it plainly.

No disagreement on that, but even he allegedly started saying it is faked recently.

source ?

http://www.cnn.com/2017/11/27/politics/donald-trump-access-h...

https://www.nytimes.com/2017/11/28/us/politics/trump-access-...

Putting aside the US or first-world elections/media..

It's worrying to think of how this technology could be used in parts of the world where there is much less widespread understanding of this level of digital manipulation.

It's basically the same situation as we had with print before television.

It's all about which sources you trust. In fact, it has always been that way.

Deepfakes just makes it more obvious. Which is maybe a good thing.

I don't think this will be a big issue. The same could already be done with still images years ago, if not decades. And with enough time and resources, the same can be applied reasonably well to videos, manually frame by frame or using CGI.

It takes a lot more to make it undetectable as fake though, and I guess the same applies to deepfakes. Once people get used to the fact such face swaps can be done, they'll be more sceptical and demand realness verification in case of published sensitive material.

Most people barely question the validity or truth of news they see on TV, and that's real footage..

You really think those same people will actually question if a video was real in the first place, particularly if it reinforces their world-view?

I'm not so much concerned about people making politicians and other personalities say something they haven't, rather I worry that these personalities will now have plausible deniability for any footage of them doing or saying something embarrassing. It's always harder to prove a negative and in this case you'd have to prove that it can't be a fake. Now imagine figuring out which video is real amid a flurry of fakes.

How will we actually verify a video is real in the near future? I’ve been thinking about this a lot lately, for many of the same reasons.

I think one answer can be found in device-level signing of images and A/V feeds. A recorder can integrate its cryptographic signing and keystore on the same chip as its CCD/audio controllers, like Apple’s T2 chip does, so that it has the ability to sign direct feeds from the hardware as being such. This might be more complicated for A/V than still images, since a large enough recording would have to be written to disk. Perhaps some rolling scheme could be devised.

For an added measure of verification, the SoC could upload them (optionally encrypted) to a distributed public store like a blockchain or IPFS, proving that they were created at/around a certain time.

This seems like it would be useful in concert with other tools for news agencies (who should be leading the way in providing as much documented research and evidence for things as they can, as a matter of course, while protecting their sources), law enforcement (who is losing the public’s trust when it comes to the chain of custody and testimony), etc.

We jumped very far ahead both in how much importance we place on digital documents and in how easily we can forge them, so we need to catch up fast on finding robust mechanisms for verification.

People should never have been trusting media in the first place, digital or otherwise. And for the most part, they don't have to. Humans believe what they want to believe or are told to believe. Evidence has little to do with it.

It will make things way worse, since politicians from now on can and will plausibly deny any video evidence about them. Candidates will literally rewrite history instead of merely suffering from amnesia and any real evidence will end up in battles between one TV show 'expert' against another.

On a bright side, young aspiring actors can make porn movies to get some cash early in their career without necessarily having to fear it will ruin their career many years later - those movies will exist anyway.

Just like there is a social system in place to sift fact from rumors (via watercooler chatter, snopes, networks of trust), at least to a degree where society doesn't collapse into chaos, in the same way will we be able dampen any damage that this new technology might cause. I think this whole "fake videos of politicians" will become a way smaller issue than what doomsayers predict it will be.

It's likely that we'll soon enter a new era of technological cat-and-mouse as tools are developed to automatically detect artifacts caused by digital processing and counter-techniques are developed to minimize the appearance of those artifacts.

Something that comes up a lot on the deepfakes sub is the notion that this should make people more skeptical. They use the phrase "If anything can be real, nothing is real" and I think it makes a lot of sense.

That's not necessarily a good thing. Sometimes it's necessary for people to believe something factual. Blanket skepticism can be very destructive in these cases.

I wrote some thoughts on this:

https://github.com/pjlsergeant/multimedia-trust-and-certific...

Kerry was getting photoshopped into propaganda photos in 2004. The new thing isn't fake documents. The new thing is a highly scalable social media infrastructure to share false documents widely.

You can't prevent someone from making a fake, but one could at least use cryptographic signatures to prove they made a statement on video/audio.

Information will need to be verified by consensus on a decentralized blockchain.

Hopefully, this makes celebrity untenable.

There will be no next election in the US.

I'm surprised nobody build this Trump watersports video yet. The tool is probably still to complicated to use. There is a business opportunity.

Maybe we're seeing so much coverage of deepfakes just so there is plausible deniability if the watersports video leaks?

Nice twist.

What is "Trump watersports"? I haven't heard of it and that is a searchterm I am not super interested in getting google NSFW results from. Is it what I think it is, I haven't heard about this at all?

There is a conspiracy theory about trump urinating on a prostitute on his visit to Russia some years ago. The theory states that Putin has a tape of it.

Probably a reference to the Trump pee tape:

http://www.newsweek.com/fox-news-host-jesse-watters-bizarre-...

Yep, that's what I expected. I read the article and while it hasn't been proven, if this exists and was released it would be pretty appalling.

Methinks it's only a matter of time when someone creates a phony video using a software like deepfakes. What a bizarre age we live in...

The rumour that Russian intelligence have video of Trump getting pissed on by prostitutes, the #goldenshower hashstag is used I believe. No idea of its veracity.

The claim is not that he was getting pissed on, it is that he paid prostitutes to pee on a bed that the Obama's previously used.

Reply on news.ycombinator.com