US House of Representatives Hearing on the Dangers of Deepfakes and AI
lionbridge.ai
lionbridge.ai
It's great the developers and researchers are taking pause - this is great tech that could be used positively and creatively in so many areas - and with any new tech comes pain from misuse.
If history is anything to go by, legislation is likely catch up slightly too late to stop the cultural impact, be overly broad, drag in a lot of relatively harmless outliers, undermine the positive use-cases and won't deter dedicated high-stakes bad actors.
"Warning: this video is unsigned and claims to originate from NYTimes.com."
There are two issues I do see, though, and they're kind of the same issue. Right now, we have this concept of a central store of public certificates. It makes it easy for you to get a certificate for a particular entity, but it also makes the central store a target. If you can compromise a central store (or a machine that is attempting to access said central store), you probably have the resources to at least redirect the user to your own site and leave them none-the-wiser, and you probably have the resources to man-in-the-middle their connection entirely and just snoop your heart out. So central stores of trust are a bit of an issue, and the ways around that are non-trivial to set up. A good example is probably KeyBase, who allow you to certify your various online presences with your private key. So if someone wants to replace your information on KeyBase with their own one, and they have the resources to do so, now they also have to compromise all the places you've distributed that key to. Or, they have to compromise one of those centralized stores of trust....
The big issue with centralized stores of trust is that they build blind trust. That's the big issue with humans in general, though. We don't want to question what we're watching. And we probably don't want to be bothered with validating that the "trusted source" of the certificate used to sign this content is actually _trusted_. It's just too much mental overhead. We want it to be automatic. We want central stores of trust, because it's just _easier_. The work is going to be convincing people that _easier_ is dangerous, in this case. Or, it's going to be to convince software companies to build in inconvenient technology and not make it trivial to turn off.
To be fair, the point of society is trust. It’s a way to trust information and ensure the species is safe.
The whole point of using markets and capitalism is because they generate more trust worthy results than top down driven systems.
Until this mess, which makes it seem like a central authority will be better than a system that leaves nodes on the three open for manipulation.
Essentially, we had a distributed decision making society. Now we found a hack to break that society structure. The cost for such a society to manage verification is absurdly high - every person will have to spend non trivial effort to verify that they are not being manipulated.
In contrast central decision making societies like China will just avoid that cost and be more competitive, beating out western democratic systems.
People still believe grandma's easily Snope-able hoax E-mail forwards. We've had Photoshop for decades, yet people believe (and share/spread) doctored photos likely more today than ever. Pros can already make convincing doctored videos. I guarantee, in the 2024 election (maybe even as early as the 2020 one), we'll have professional-quality, flawless "video evidence" of one or more candidates saying or doing things they never said or did, produced by amateurs. Despite knowledge that this technology exists, most people will fall for it.
Yes they will. That's the one part, which is why there is a problem here in the first place. The second part is that people will stop believing media, which might actually be a good thing.
The real problem with deepfakes is when the latency to modify something goes from hours to minutes to real-time, and even a live stream can no longer be trusted to be genuine. We're pretty close to that point, and for some use cases have passed it already.
Why is that a good thing?
Besides that, people are just too plain gullible when it comes to media. It used to be said that if it is printed it must be true, now it is 'if it is on the internet, it must be true' leading to vast numbers of people being critically mis-informed about important issues.
More likely: They won't trust media not in line with what they are already thinking. This won't help reduce confirmation bias. Thus already existing problems will likely be aggravated even more.
> But there's no reasonable way to stop it from being written and used.
Can't stop encryption, can't stop P2P file sharing, can't stop deep fakes either.
Cellebrite, etc.
> Near as I can tell, it's still quite a problem for law enforcement.
They complain about it because they want everyone's data all the time, likely for convenient fishing expeditions. But as far as we know (based on public testimony and documents at least) they have been able to get access when they actually need it for criminal and terrorist cases, or at worst they have been able to work around the lack of access in other ways. This all came out back when Comey was first complaining about it.
> And it's a false equivalency: deep fakes mess with something far more essential to a working society than secrecy: truth.
The truth left the public sphere long ago, leaving us with a false consensus propped up by the media. The "fake news" hysteria is just the public waking up to this condition, coupled with the media's growing fear over the loss of their ability to influence public opinion. It remains to be seen whether we will actually find a way to identify and hold onto the "real" truths, many of them highly unpleasant, or whether we will slip back into another false consensus led by the ruling class and their courtiers.
https://www.chicagotribune.com/opinion/commentary/ct-perspec...
https://www.aarp.org/money/scams-fraud/info-2018/grandparent...
https://www.consumer.ftc.gov/articles/0204-family-emergency-...
The dramatization was written as a news report and didn’t identify itself as fictional until well into the broadcast. Angry callers reported mobs taking to the streets, but the scale of the reaction has been debated.
0: https://en.m.wikipedia.org/wiki/The_War_of_the_Worlds_(radio...
There's always more to an event being a trigger for change than a simple binary is this true/false. Humans are not simple rational actors.
You can trigger action with small things, concentrated on the right people, or you can have major news go under the radar.
Of course, if reliable sources were present at the occasion, they could jointly sign a recording, but under other circumstances it could be fake.
Another potential problem is producing fake confessions in e.g. criminal cases. The person who allegedly confesses would not sign the recordings, an the police's signatures are worthless if they are fabricated.
To look at it from the dystopian, draconian point-of-view, deep-fakes created and used by the local justice system[s] could be signed by a police officer (who's intent is "solving the case, no matter what").
No matter how many witnesses and experts that you throw at it, the court will almost always believe the officer (because of unjustly doted infallibility) and that deep-fake could cost someone their life.
Just as other tools have been used for good and bad, on both sides of law enforcement, the probability lends to this equally be abused; especially, in areas where "south of the Mason-Dixon line" and "the south will rise again" are still parts of every-day parlance.
Existing libel laws would cover this, no?
Not sure what is gained here.
Why won't it be the same this time? It's much easier to fool a human than to fool tools designed to detect manipulation, and that's limited fake photos' impact severely. As long as we have ways to detect fakes, and media responsible enough to do the cursory investigation required to verify authenticity, I don't see why manipulated videos poses any greater risk than manipulated photos.
As the cost of producing fakes approaches zero, the noise-to-signal ratio increases until people distrust everything they see and hear. I'd argue we are already on that path, and have already begun feeling the consequences. (Read about the Russian "Internet Research Agency" and their role in tampering the US 2016 election.) Deep fakes accelerates that trend.
> As long as we have ways to detect fakes, and media responsible enough to do the cursory investigation required to verify authenticity...
Media is becoming less prone to cursory investigation (fragmentation, understaffing/lack of revenue). A GAN is specifically designed to fool detection, and they're only getting stronger.
This is a bold face lie. The cost of producing the fake is nearly irrelevant, what matters is the reputation of the broadcaster. This has been so simce time immemorial and did not change with Photoshop.
Many journalistic revelations stand, even though they could be trivially faked. The Snowden documents would cost trivial amounts of money to fake (it's a bunch of PowerPoints), but people believe it, because se US government issued a warrant for his capture. The metadata is much harder to fake.
- Have a piece of hardware in-frame that can show hashes that is hooked to the operating cameras.
- In each frame show the hash of the previous frame (low latency would be required, as would the transmission of the full-quality original).
- Publish the original seed and the final hash (and, since the original is broadcast, the whole chain of hashes can be verified).
And how do you validate what is displayed on TV? Who will validate most videos they see?
If you want to do something more without hurting free-speech improve existing slander and defamation laws to specifically target deep fakes.
Hold the platforms accountable the same way copyright has, its worked wonders.
I haven't seen much evidence of this. Now we've got a significant portion of the population who calls every news report that conflicts with their view of the world fake news and then turn around and consume media which has been measurably shown to be among the most inaccurate available. All of this behavior is supported by the President of the United States. When will people wisen up?
Deepfakes wont be any more challenging to deal with than the struggle for accurate information in the pre-TV, or even pre-print days. People will learn to look for reputable sources, and propaganda will keep trying to trick people in new ways.
The internet brought a few things into play that have made the issue more complicated. Primarily that information and false information can be made by anyone and spread at the speed of light. Secondarily, that more people can present themselves as experts because all you need is a website and not third party's validation.
In only a few years we went from a researcher showing how he could manipulate a video to make it look like someone said something different than the actual transcript to the ability to fabricate entire scenes with deep fakes. Most people in the world don't even know this technology exists.
If you think only some back woods Trump supporters will fall for this then you're definitely mistaken.
Fake news isn't binary - it is a continuum. For many, fake news is merely an extension of misleading news which has existed for a long time (recent examples are things like the crowd level at the toppling of Saddam's statue).
And to be frank, many of the alarmist 90's email chains (warnings about fake scams, etc) fall into the category of fake news. They're written like news articles and passed around, and are totally fabricated. It's just that today they have fancy looking web sites - but the concept is the same.
Misleading news is as old as news itself. When I was a news junkie, I knew the effort it took to distinguish real news from misleading news - both coming from the same established news agency/company (be it NY Times or CNN or Fox News). The only difference now is that fake news sites tend to publish only fake stuff - the difference is merely a matter of degree.
As much as it's annoying that people are calling everything fake news, I don't want to encourage people to go back to easily trusting traditional news sources - NY Times, etc. It's naive to think that people calling everything "fake news" is an artifact of Russia or Trump. There's been a very legitimate distrust of news organization for a long time, and all we're seeing is the lack of nuance in this distrust. Trump/Russia merely tapped an existing vein. Had they not, this level of distrust would have arisen soon anyway.
It has?! Citation needed.
You mean by creating a selectively enforced corporate hellscape?
Because it’s clear, that there’s no private solution to essentially an evolutionary war between carnivore (malicious humans) and herbivore(people who don’t want to be manipulated).
Right now, the Chinese total control model is the model that’s working, and while on HN we may find it abhorrent, people in high places are being forced to make pragmatic choices. For them a Chinese styled information system will win out.
To prevent this, I think it’s increasingly time to re-examine our norms on information and expression.
In general, taking a step back - human society (markets, media, reporters, books, news) are effectively a giant solution to finding out what is “true” and what is “something else”.
We are excellent at solving these problems, we build families (parents are decision makers and know what information and implications make sense), clans (what is ideal for this group of people with similar genes I can trust), businesses (contracted people with relatively aligned interests), and more. You get the drift.
So the question resolves to: How, do we organize ourselves, to verify information, to at least keep parity with the verification rates before the internet era.
The key difference, is the mass production of information/content. The rate of content generation outstrips the ability to verify.
This last part will not change. We will always lose as an information society, if we get into a pure verification war with computer generated information.
This leads to the first conclusion:
1) clear measures to control the rate of generation of fake data.
to be blunt: this means jail. Punitive and clear response to stop this behavior, across borders.
Here’s where a decent chunk of HN will be aghast. This is what I meant by our values coming into conflict with the necessities of the solution.
Which brings us to problem 2/ addendum to solution 1.
2) who watches the watchers?
If govt has the power to punish people for “fake news” how do we know that’s not misused for “inconvenient news”.
Well, the weak solution that presents itself is a fire walled agency, with guaranteed ability to be funded, in order to seek out and identify manipulation and spread of faked information.
Hopefully, this agency won’t crumble on day 1, on the inherent contradictions of its role and responsibilities.
Some structure of this sort, gives us a pathway through the near future to deal with this issue.
Hopefully, it can be used to buy the time to solve the issue we are facing.
The irony, of advocating for a ministry of truth, in order to save the truth, is not lost on me.
But unless action is taken to stop propaganda, mass produced information creation, then it is a guarantee that our old human ways of assessing information will fail.
The only option which will be left on the table is dystopia, or some bizarre world where nothing and anything is true.
Good luck punishing "fake data" from China/Russia.
You write as if web-of-trust isn't a thing.
I think any government agency has too many inherent biases to ever be trustworthy. I’d believe 100 self-asserting bloggers before I believe Pravda.
in that case how do you reconcile fake news with the needs of a complex society?