NSA, FBI, and CISA Release Cybersecurity Information Sheet on Deepfake Threats
cisa.gov
cisa.gov
Original story: https://www.nytimes.com/2005/02/02/world/africa/rebels-say-t... Confirmation of hoax: https://www.nbcnews.com/id/wbna6894934
It's pretty easy to see that the photo is a hoax, but many news outlets didn't notice and ran the story anyway.
I have little faith in our ability to detect deepfakes using these recommendations. It seems we'll have to assume something is fake unless we have a way to cryptographically verify its provenience.
More likely: many news outlets didn't CARE and ran the story anyway.
ps- also the "dumping babies on the ground" fake too IIR, but maybe only a few times on that one. Congressional testimony on camera by a daughter of a State Department official IIR to justify the Kuwait invasion by Bush I
Keep listening to them if you want. I used to be an avid listener, just realize that they've lost a TON of loyal listeners due to their heavily biased reporting in the last few years.
Here's a fun one for ya. They backtracked on it after not getting away with it but there are many many more where no one notices and they just get away with it.
https://news.yahoo.com/npr-claims-limited-scientific-evidenc...
you see, the issue is not NPR being untruthful. the issue is NPR not even having a chance to check. member how "logic"/"rationality" works on faked premises? so smooth and yummy.
I think that's still much better than a news source that just outright lies to your face whenever it's convenient. As long as you remember that the bias is there, you can at least trust what they say is the truth. When a news org tells a mix of truths and lies you don't even have a foundation of trustworthy fact to start from.
Even then it's probably hard to prove that insufficient checking was performed at publication time
In theory, subscribers will leave outlets that lie to them for outlets that are more honest and fact-based. Or, if I'm pessimistic, subscribers will give money to whoever does rage-bait the best...
You could I suppose allow adverts where the information is strictly for entertainment, but given that was Fox News defence... I think it probably wouldn't work.
1. https://en.wikipedia.org/wiki/Luther_Blissett_(pseudonym)#Li...
How to make someone feel old. All the way back to 2005, eh?
"The authenticity of the militants' claim Tuesday could not be immediately verified. Defense officials at the Pentagon in Washington said that the U.S. military was investigating the incident but that had no indication any of its soldiers were missing in Iraq."
I know all this can be done, I'm just surprised it's reached the maturity where an attacker would choose to impersonate someone the call recipient presumably knew vs just being a vague "Bob from IT".
Although to be fair the article does say the employee was suspicious so maybe there was a delay which (if you were looking for it) you would spot.
The approach seems to be unnecessarily risky vs just phoning up pretending to be someone they didn't know is my point.
You can also use text macros to type the response faster. Here they were trying to get MFA access, so you could map longer phrases that will come up often like "Okta multi factor authentication" to numpad 1. Company name to numpad 2. IT supervisor name to numpad 3.
If you know the target of the conversation you can tailor what you pre generate. I like to mess with scam callers when I get one, and I've noticed some are using some kind of soundboard with a woman's voice (I'm pretty positive it is real and not AI) and they have a planned flow / script. If you try to deviate from the script they have some options to bring you back into it. If you ask them to repeat something you can notice it's the exact same audio snippet as before. If you accuse them of being a bot they have a few samples of the woman being shocked and mildly embarrassed. "Oh my goodness, do I really sound like a bot? No it's just been a long work day for me. I'm sorry about that."
Live transcribing in realtime has been a thing for, forever, so there’s no reason for me to think this couldn’t all be glued together into a “voice changer” like the typical super deep “I have your son give me a million dollars” boxes, except instead of doing frequency modulation it is pipes to a model trained on someone’s voice, and applies it. Transcribing to text probably isn’t even needed because why would it be for machine to machine modification. It only needs to go to text for human consumption.
Raw pcm bits from audio in -> AI model trained on victims voice -> line out to phone or voip app.
We totally have the compute to do that. Probably with our phones.
The real solution is to carefully select a variety of experts who you trust as sources of information, and rely on the consensus of those experts to determine what is true, while maintaining a healthy level of skepticism.
Before internet and telephone, we still took things on faith from people we met, be it travellers or politicians or newspapers. But newspapers and politicians were more local, and didn't buy into whatever they read like scripture the way many seem to do these days.
I really don't think the situation today is as bad as people say it is. You can still go to the news pages (not the opinion pages) of the New York Times, Washington Post, Wall Street Journal, Financial Times, and the Economist, and the vast majority of what they publish is reliable information. We can debate over the partisan nature of what stories they choose to emphasize and editorialized headlines, but it's not like they are regularly publishing lies and nonsense.
We live in special times. Until 100 years ago we had just news from other people, mouth to mouth. So we had to decide what is wright and what just BS. Last 100 years we had newspapers, radio and TV. News was mostly true, we could trust what we saw on pictures, on screen. But now again .. pictures can so easy be just fake. So we have to decide again, what is true and what BS:
100 years ago was the time of yellow journalism, where lies were commonplace.
The problem with deepfakes is the instant amplification they’ll get from media and well-intentioned responses from people of good faith.
So there will be the deepfake of (insert your favorite politician) saying something unsavory and it’ll be taken at face value and a bunch of people will cancel their social media and bank accounts and protest in front of their houses etc etc.
The solution is super simple but unpalatable in today’s culture - waiting periods.
What if corporations were required by law to wait 30 days after a public spectacle before taking action? This would provide time for the victim to defend themselves and the truth to come out.
Similarly I have always thought that passing laws in the heat of the moment results in people’s rights being tramples. What if legislators were required to wait one year before passing laws in response to a public event of some sort? This would give time for passions to cool and alternatives to be considered and impacts to be weighed.
But this is all a pipe dream, let’s just cancel them immediately like we do now.
Someone discovers a way to bypass regulations and legislature and easily acquire the material needed to create a dirty bomb and sets one off. Now we wait 365 days to fix the loophole.
Pretty obviously an absurd extremity for that example, but I sure as shit don't want there to be that level of lag time on important things like that, and I imagine most people would also want to be able to move faster in situations where it is dangerous not to. How do we determine where that line is, what the exceptions are, etc.? What happens when we need an exception and didn't realize it?
What happens in real life is that in emergencies, the legislature passes some “sentencing enhancement” that adds on a decade or three of prison time in the circumstance. Or they create a new bureaucracy that in the end just makes life worse for everyone, like the DHS and all the airport suckage that happened after 9/11 - if you’re too young to remember, flying on airplanes did not always suck.
You're talking about something that has been ruled by two courts as a massive first amendment violation, and is heading to the Supreme Court (if they even want it), as if it were a conspiracy theory.
And what political point are you even talking about, where do you learn to use this tone about serious issues, and why the fixation on celebrities like Musk?
https://www.scotusblog.com/2023/08/once-suspended-twitter-us...
This case?
Where did _you_ learn this tone of victimization when discussion issues with no references? I assume Musk was brought up as he has framed this issue in the same way the GP commenter did and he is the current owner of the company in question. You can dismiss that as celebrity fixation but it only undermines your own comment.
https://arstechnica.com/tech-policy/2023/09/supreme-court-co...
> The 5th Circuit appeals court saw things differently, finding that Biden administration "officials made express threats and, at the very least, leaned into the inherent authority of the President's office. The officials made inflammatory accusations, such as saying that the platforms were 'poison[ing]' the public, and 'killing people.' The platforms were told they needed to take greater responsibility and action. Then, they followed their statements with threats of 'fundamental reforms' like regulatory changes and increased enforcement actions that would ensure the platforms were 'held accountable.'... Given all of the above, we are left only with the conclusion that the officials' statements were coercive."
https://news.ycombinator.com/newsguidelines.html
Check them out when you have a chance. I _trusted_ ScotusBlog and the previous comments gave no references to _verify_.
We've all sat through various crappy powerpoints on ethics. That is plain and simple coercion. That's a first amendment issue and prohibited.
California here
Could you point me to that information?
> President Biden, press secretary Jen Psaki and Surgeon General Vivek Murthy later publicly vowed to hold the platforms accountable if they didn’t heighten censorship.
I don't want a government or DOJ in place that allows me to have my 1st amendment rights "most of the time" except the times they don't like me having it. Are you serious with this comment?
> These emails establish a clear pattern: Mr. Flaherty, representing the White House, expresses anger at the companies’ failure to censor Covid-related content to his satisfaction. The companies change their policies to address his demands.
From a nonpartisan org: https://nclalegal.org/2023/01/the-white-house-covid-censorsh...
That Biden set up a "Disinformation Governance Board" is wild. 10 years ago I would have laughed and called it Orwellian, and then it happened
"beware of false information" said the people who keep the biggest secrets
I find your distinction between altering and creating to be ... a sidetrack. But okay, let's pretend I said "Deepfakes is just the hype word for digitally altering or creating photos/videos".
Large institutions/etc have definitely been able to create novel "photos" from scratch for a long time now. It was just a lot of work, which is why it was restricted to large groups of people with lots of capital. Now even individuals can do it. And that's good. The idea that prior to deepfakes that photos could not be created is not supported by my lived experience since the 1980s. Just look at any hollywood movie.
There's still something about the scale of effort that seems to change this, though. For example, even if a government could hire people to produce a fake photograph, and even do so in time to use the photograph for some political manipulation, I'm not so convinced they could do so with a video. Even considering that large organizations could already do this technically, the ease of function that this allows is pretty worrying, though I do otherwise think it is a good thing that this technology is "out of the bag", so to speak.
Unless you're reversing your prior point and now talking exclusively about only deepfakes that alter existing video (like face replacement) and not about novel created from scratch video. The face replacement stuff can almost look real.
Think of telephone scams, and now apply deepfake versions of a family member to it. Imagine your 70yo aunt having to deal with knowing if the whatsapp audio or zoom call that sounds/looks like you is really you.
I don't think this was possible before, feel free to prove me wrong with some evidence.
In the past to find the right person to imitate someone elses voice you'd have to do a lot of casting trials/etc that would cost a lot of money. So it was restricted to only large corps, governments, institutions, etc. But now any human person can do it. And that's great.
I can think of several ways that these types of technologies being easily accessible to anyone who wants it directly leads to bad things.
There's good things, too. But come on... Not bad in any form or manner?
I really enjoy generative AI and think that we're better off with it being in the open than controlled by specific corporations.
I also think this is an absurd comment. People are already making deepfakes of classmates, coworkers, etc. and sharing them around. I'll ignore the ethics of generating them for your own personal entertainment, but I can't think of any argument that you could make that when they start getting shared that this isn't "bad in any manner" - obviously this can cause emotional distress and damage to someone.
Yesterday I was reading this article about the day the USA almost accidentally nuked itself:
https://en.wikipedia.org/wiki/1961_Goldsboro_B-52_crash
> The Pentagon claimed at the time that there was no chance of an explosion and that two arming mechanisms had not activated.
> A United States Department of Defense spokesperson stated that the bomb was unarmed and could not explode.
> In 2013, information released as a result of a Freedom of Information Act request confirmed that a single switch out of four (not six) prevented detonation.
Nobody should ever trust a single thing these people say.
At this point, giving up entirely on any form of media seems appealing.
I mean, what is new to this problem. Other than it is somewhat cheaper to pair the fake statement with the person responsible reading it out laud.
Department press release -> Reuters -> News paper -> reader
No signing required. The reader can verify the press release of he wants to.
One could argue that this will be a good thing because deep fakes will be so prevalent (e.g. kids making videos of their parents saying and doing funny things) that the default assumption is that everything is fake until proven not fake.
Thanks to a businessman/politician who turned "news about me I don't like" into "fake news", we got a jump start on that.
This is what it's like living under an authoritarian government. "Of course the government is lying", "Of course the politician is lying", "Of course my neighbor is lying", "Of course the company is providing me with a fraudulent product"
This eventually turns into a kind of learned helplessness and is how you create a crapsack nation/world. "Everything is bad, so there is no reason I should do anything good"
I can promise you that you won't enjoy this world we're creating if you don't live in an authoritarian shithole already.
Also, official channels get hacked too.
* Text: You have little (definitive) clue who wrote what. You essentially have to ask the (apparent) writer.
* Photo: You used to have high confidence that a picture shows who appears to be shown. Not 100%, sure, but it's high.
* Video & Audio: You used to have very high confidence that the video including its audio are genuine. It was very difficult to replace video and/or audio.
Nowadays, none is trustworthy by default anymore. You can say: Well, just trust the company or Reuters. Sure, but I don't think anyone cares about this case. It's not controversial. But how will they be able to verify controversial sources?
If they get sent a video claiming to be about Ukrainins killing civilians, and outfits & speech matching that, how can Reuters be sure about anything now? Trust can't be given to the source, nor to the video, nor to the audio, nor to the metadata.
If a CFO makes a statement and that is on the company's website we can have reasonable confidence that the CFO made that statement and we can act on it.
Reporting on a video of unknown (possibly unknowable) provenance is a different kettle of fish.
Example - Dictator A says terrible things on video/audio. Of course it's not going to be shown on their nations' broadcast website or in media.
How can the rest of the world make sure the video that was recorded is trustworthy?
A network of trusted sources, reporters and newspapers that the public trusts. Eye witness accounts, a preponderance of evidence.
Basically it boils down to the reputation of the individuals involved in the chain of trust.
Personally I'm excited by the prospect that we might get mainstream investigative journalism back in some form.
I don’t agree. Many important photos don’t show what we think they do.
The Soviet flag on The Reichstag. When it was taken and what it showed are different to the impression you get looking at the photo. It was taken after the event and the signs of looting were removed. https://en.m.wikipedia.org/wiki/Raising_a_Flag_over_the_Reic...
The flag raising on Iwo Jima was the second flag raised that day. It’s not quite the same once you know that. https://en.m.wikipedia.org/wiki/Raising_the_Flag_on_Iwo_Jima
The famous ‘Falling Soldier’ photo from the Spanish Civil War is now thought to have been faked. https://en.m.wikipedia.org/wiki/The_Falling_Soldier
There are bound to be loads more, and the faking goes way back. The US Civil War has examples where bodies were dragged around and made more dramatic. Added cannon balls in Crimean War photos etc.
Perhaps that is a good thing. Maybe this is a good excuse to stop and consider multiple news outlets, even if it conflicts with our own opinions, for our news sources.
The thing about FoF is it gets people to stop believing anything which is very effective for bad actors and far less useful for good actors.
The physical artifacts yes, but not the narrative they were portraying. The “news” media has been spinning fictional narratives with physically authentic video and audio for a long time.
Think back to the Nixon watergate scandal. When the reporters were going to press about that, they made damn sure it was 100% real first. By interviewing varying sources, human trust, etc.
All that really changes is they can't take video and audio evidence as fact anymore. So they have to, in essence, audit the video/audio trail, so they will want to talk to the person that filmed it, make sure the story holds up, etc.
Some technology changes can help with authenticity here, but it's not really a technical problem, it's a human trust problem.
There will be learning curves and maybe one or two of the currently well known and trusted news sources totally burn their brand because they didn't do their homework. Nothing really new though.
Don't roll your own authentication.
I guess that is not a new problem either with lying newspapers.
Which it won’t. Eventually might be relevant when in a context where someone actually stops and spends time looking at evidence (civil and criminal court cases perhaps?) but those already use chain of custody for evidence because evidence has already been easy to fake for… well forever.
Still should be done IMO though, as it’s cheap and easy and will hopefully make it a little harder (or easier to detect) to do mass faking in the ‘middle’ - like fake IDs for online services, fake blackmail photos, etc.
For example, when watching a video on YouTube containing a speech by the president (provided on an official channel like the White House's), there should be a clear indication that the video has a digital signature and the option to verify it on an independent government website.
Adding signature after recording works to certain degree, but it still does not guarantee that the content is what the camera saw.
Cameras can record screens.
Perhaps adding a signed channel for depth and/or non-visible light would be the next step.
You're pushing a (bad) technical solution to a social problem.
Cameras that cryptographically sign their output will not solve anything. The idea has more flaws than it's possible to list, but here's a big one: do you really think a technological gimmick like that would stand up to a nation state? Do you really think the CIA, NSA, FSB, Chinese Ministry of State Security, etc. will not be able to sign whatever the hell image they want with a camera's signature?
Or you could just smear some Vaseline on the lens and tell people the lens got dirty. It hurts the credibility for anyone who knows about these cameras but I doubt the public would think about it that much.
Also, how to protect a chip from reverse engineering even from all except "top actors"? I remember the price for reverse engineering of certain ICs was between 5 and 7 figures of USD. Don't know about modern IC processes, but it may be affordable for many even for those?
I don't know audio well enough how it happens there. But potentially it can be signed in chunks as well.
Of course, one needs to consider risks if editing can make content appear different than originally intended, when the video as "whole" is not signed. But for that, different entity can be used again.
It would require completely changing how software currently handles video editing in short.
At the beginning, camera manufacturers might need to provide their own editors, to make editing possible. How much we can trust the camera holders, if the editor software even allows using the key from the camera for better editing in certain limits?
If you just cut away to the original clip and didn't have any modifications like motion graphics over the top of it you could in theory pass through the original video with the same compression and signing without too much drama but any modifications over that or presenting it as picture in picture would be a big difference as now you need to have both the original frames with the added graphics on top.
Does that really matter? You could sign keyframes, and then also sign the differences frame by frame til you get to the next keyframe.
The answer is that it doesn't. Some might try to make it work by using proprietary video editing software that signs a ledger of what edit operations were performed, or something like that, but that doesn't work. The signing keys will eventually be extracted from the video editor or the camera, or the video editor or camera will be hacked to sign something it shouldn't. You might say that this at least stops low-skilled attackers, but the misinformation that is most dangerous to humanity, that created by governments to start wars, won't be impeded by any of these schemes. The whole cryptographic signature proposal is worse than useless.
I wonder if you could use the camera to record deepfaked video and in effect bless a lie. Even just filming a TV set might be enough for low grade blackmail and much more complicated methods are available.
But even hardware can be hacked/bypassed if the effort is worth it.
For example the whitehouse is known to revise the text of the president’s speeches when he says the wrong thing. If we only have officially released videos where the gaffes and fables are left out, how is anyone to know what he actually said?
The biggest risk IMO is that key becomes immediately one of the most important secrets to keep since it holds the promise of validating anything you want to lie about.
The loss of truth is a serious thing for a society. (Yes, back in the Walter Cronkite days we had less truth than we thought we did. We had more agreed-upon truth that matched reality than we do today, though, and I think the difference matters.)