South Park creators have new political satire series with AI-generated deepfakes
theregister.com
theregister.com
I've been thinking about this video a lot since I first saw it. It's genuinely very unsettling.
0: https://www.nytimes.com/2020/10/29/arts/television/sassy-jus...
I don't think he's doing the voice for Sassy Justice, though. That sounds more like Trey Parker to me. But they probably brought him in to do Michael Caine.
That said, this stuff will improve over time. But before you label it the end of the world you really have to think about how much has always been possible with impersonators and makeup/prosthetics.
“But when Parker got to see himself digitally altered to look like Al Gore, he said, “It was the first time I had laughed at myself in a long time.”
Parker added: “I always hate watching myself. Even with ‘South Park,’ I have a perfect image of what it’s going to look like in my head all the time. But on this, there were moments where we felt like kids in our basement again.”
To Parker and Stone, the experience also reminded them of “The Spirit of Christmas,” their 1995 homemade short film that became a viral sensation in a more primitive age of the internet and paved the way for “South Park.””
[0]: https://www.nytimes.com/2020/10/29/arts/television/sassy-jus...
It’s made me stop and think about why that is, and the best I can come up with is that very little is surprising anymore. When you get past 30, everything feels like something you’ve seen done before. Deep fake comedy is so cutting edge, that it couldn’t help but be fresh.
Or maybe I’m just a boring old man. The last thing that made me laugh this hard was this Reddit submission, and again, it succeeds because I didn’t see it coming: https://i.reddit.com/r/ContagiousLaughter/comments/gzdja1/on...
Matt, Trey and a guy named Robert Lopez wrote the theatre production Book of Mormon, which was well received and won Tony awards (Trey studied Musical Theatre in College, which explains why the South Park Movie was a musical).
Trey's daughter was cast as Jared Kushner in Sassy Justice, which was hilarious, and the lead in Sassy Justice is played by Peter Serafinowicz [0], who is a British comedian!
I hope they make more of these, not just because I'm a fan, but because it massively disseminates the power of deepfakes!
And a big part is that you have prior knowledge. I'll be honest, I didn't realize it was Trump at first. Nor did a friend that I sent the video to that didn't have the prior that all characters were fake. Took him a good minute. That's a meaningful difference.
From the NY Times article:
> The “Sassy Justice” creators said they had spent “millions” of dollars to make the video, including the initial investments to produce the halted movie and set up the Deep Voodoo studio, though they declined to specify the exact cost. “It’s probably the single most expensive YouTube video ever made,” Parker said.
Would make those kind of artifacts much harder to identify
Most likely not, and the parent comment seems to agree with you on that. I think they were just trying to point out, in general, that the arrival of commonplace audio deepfakes might be way more disruptive than video deepfakes, despite a lot of people (including myself) who used to counter-intuitively think that video deepfakes would be more disruptive.
Here's an exploration of a deepfaked Jay-Z reading/rapping the Navy Seal copypasta: https://www.youtube.com/watch?v=UZzYoOdIXoQ
Also, it's one thing to say the world hasn't ended, but that's to potentially downplay at a minimum the idea that commercial and widespread use of photoshop hasn't had widespread effects on body and self-image, creating and interacting with arguably culture-bound psychological issues such as anorexia, bulimia, unnecessary surgery, self- harm, suicide, etc. Or to take examples from non Anglo culture, eyelid removal, skin whitening, nose surgery, etc.
it's true that the world hasn't ended, but that's a thought terminating cliche. there's a lot of evidence it's creating and created significant harm and significant effects.
https://boingboing.net/2008/07/10/iran-you-suck-at-pho.html
(and of course, you certainly have missed the fakes that didn't get noticed!)
Focusing first on 'changing the meaning in a believable way'. Believability is a subtlety bordering on a truism. If it's believable, we would often deem it to have "not sufficiently changed the meaning". If it "sufficiently changes the meaning", does that make it no longer believable?
Second, it's arguable that the whole point of being a good photographer/photo-editor is to make an altered image look "believable" or "impactful" without triggering the brains uncanny valley or rejection response. That's fundamentally why we photo-edit.
For a start, there's actually lots of explicit examples from history:
- Lincoln/Calhoun composite
- Stalin editing opponents during purges
- John Kerry + Jane Fonda in US election
- George bush holding book upside down reading to school child
-etc etc etc.
Those examples are just the obvious things that a general man on the street would accept as photoshopped, and the ones that I know are 'shopped. There's a whole bunch of photography techniques involving telephoto lenses into crowds doing the rounds worldwide to make people look more numerous and close together (to spark social distancing outrage), framing and cropping like of trump's inauguration to make it look more crowded than it really was, and arguably "Hunter biden finger-lake tattoo child-molesting" stuff currently doing the rounds on the internet's seedier sites, but I don't know how/whether they're photoshopped, and you get the photographer's dilemma of discussing the lay person's belief that effects done "in camera" are "real" but effects done "out of camera" are "'shopped": a distinction which is often effectively meaningless.
Then we have the general "mass photoshopping" phenomenon, where practically every commercial image one sees has been explicitly edited to create some effect and giving the idea that the subverted-reality is just 'normal' and not in anyway actively subversive: be it skin that doesn't look like actual skin; eyes, hair and teeth standards and colours that are practically biologically impossible; general biological-hair removal; slimming, shaving and exaggerating the respective female body parts; slimming and expanding male body parts.
And practically everyone's doing it: take Kamela harris' officially supplied photo: https://upload.wikimedia.org/wikipedia/commons/d/d9/Kamala_H...
You can zoom in on the eyebrow and hairline to see such poor use of the blur tool that they barely even bother hiding it any more.
Now, as per my previous point you can argue "oh yeah, sure all our pictures and images are messed with and touched up, but it's not harmful and that's not really changing the photo!".
To that I repeat my initial observation: no, it does indeed seem to have a harmful affect in terms of body-loathing, a drive to self-hatred and consumption, and general reality distortion. It can be picked up upon and observed via the way it expresses itself differently in different societies and different ethnicities by observing the peculiar culture-bound mental illnesses and aesthetic phenomenon that appears as each is exposed to different photo-editing ideals and cultural mores.
And then there's the second point that these deep-fakes will actually be the next level. If we see, and know, that there are problems with our photoshopped media while the common man has folk-beliefs about the objectivity of the photographic media, imagine what's possible once we see that disconnect begins to apply to video + sound as well. It's probably going to get worse. Much much worse.
And remember, with doubtful photos, we have the belief in the authority of sound + video recordings to fall back on. Once we remove the authority of sound + video, we no longer have any effective authority to determine the factual nature of the media we're viewing. The deepfakes aren't generally quite there yet, but in 5 - 10 years, they'll be better. And they only have to get to a baseline where a significant number of people are fooled by them, or at least create enough of a reasonable doubt to allow them to dismiss or accept positions they disagree/agree with, before it becomes a real problem.
Yesterday there was a fake Trump tweet with a zillion upvotes on Reddit, and that’s just text. Text is trivial to fake, so maybe the chain of trust we see in text is what we’ll see with everything else moving forward. “An anonymous source within the White House provided this footage,” being countered with, “you can’t trust anonymous sources!” It will come down to who you trust, just like it always has.
I often feel deep existential dread about deepfakes and the lag between when they are viable (now) and when everyone stops trusting video that doesn't have some kind of rock-solid signed/encrypted/testified provenance (years from now or never?).
What better way to educate the public about deepfake technology than with over-the top satire?
I really don't think this is true. I hear stories about deepfakes on NPR pretty regularly. It would be very strange if a subject that has managed to gain some mainstream media traction was never internalized by a large percentage of the general public.
Also, people believe what they see, on an instinctual level. Hell, people believe what they hear, mostly. It will take a lot of education and time to untrain that.
It will introduce a healthy dose of skepticism to that media.
In the - now famous - film below, Kuleshov edited a shot of the expressionless face of an actor, which was alternated with various other shots (a bowl of soup, a girl in a coffin, a woman on a divan).
- https://www.youtube.com/watch?v=_gGl3LJ7vHc
When the clip was shown to an audience, they believed that the expression on the protagonist's face was different each time he appeared, depending on whether he was "looking at" the bowl of soup, a girl in the coffin, or the woman on the divan, showing an expression of hunger, grief, or desire, respectively.
Of course, the footage of actor Ivan Mosjoukine was actually the same shot each time.-
The audience even went on to rave about the actors "performance" ... "the heavy pensiveness of his mood over the forgotten soup, [they] were touched and moved by the deep sorrow with which he looked on the dead child, and noted the lust with which he observed the woman".-
The point is that, given the audiovisual medium, a certain degree of manipulation, or "intent" on the part of those creating the work is always to be expected ...
... and, also, that audiences bring their own bagage when they view something.-
I'm the author of https://vo.codes, and my goal has been to make deep fakes as accessible as possible so people become familiar with them.
You know what people use vo.codes for? Memes. That's it.
A number of journalists have decided the technology is entirely confusing and deceitful. While there are new risks posed by deep fakes, I believe the potential for good far outweighs the bad. Today I see it being used mostly for artistic purposes and for humor. The real threat is social media and the soapbox amplification and attention algorithm, not deep fakes.
Deep fakes enable creatives to make this kind of stuff (most of these are made with vocodes) :
https://www.youtube.com/watch?v=dSgd4PoQofQ
https://www.youtube.com/watch?v=iBpqJF5LXX4
https://www.youtube.com/watch?v=3qR8I5zlMHs
https://twitter.com/RodanSpeedwagon/status/13172304947156213...
https://twitter.com/twpl_logan/status/1306542488694460416
vtubers are using it to give themselves new voices, which is amazing:
https://www.youtube.com/watch?v=LCKcrSrPYcw
As the technology improves, it will become disruptive and enable average people to join the ranks of Hollywood actors, directors, music producers, and vocalists. The future isn't A-list celebrities and grammy winners, it's people like you and me.
The people stoking the anti-deepfake fire are the media. They claim that this is the end of trust and authenticity, despite the fact that classical methods of trickery and deceit are already way more common. Where are their doomsday articles about catfishing, phishing, and social engineering? There aren't any, because it isn't exciting.
This technology doesn't really move the needle for deception if you can slow a video of Nancy Pelosi to half speed and claim that she's drunk.
Deepfakes are just the next wave of photoshop. People don't use photoshop to steal elections and win court cases. They use it to make memes. It's the same deal with deepfakes.
The technology is going to improve. And I'm sure the hysteria will increase in volume too.
Real time voice conversion and video generation is going to be cited as "scary", but it'll mostly be a hit with gamers and vtubers. It's not going to be a "Mission Impossible"-style espionage tool. It's going to be used in good humor and to good effect.
These tools are going to bring about a new media renaissance. They'll let the small players compete with the giants. That's not scary - it's exciting.
That's also what I'm working on for my startup: memes today, hollywood / old media disruption tomorrow.
I've already got insane growth (millions of requests a week, and our videos have hundreds of thousands of YouTube views). I'm wondering when to pull the trigger and start hiring people. (My users are already working with me to build more!)
I'm super excited by this field, and you should be too. The media is screaming fire, but I'm running at it with full speed. I see the magic and the amazing opportunity.
It can be used to destabilize small countries rapidly to provoke a ethnic or religious conflict. In conjunction with other tactics and a prepared military effort.
Imagine if ISIS did this to Assad just before their main offensive? Or if Hezbollah did it to the Prime Minister or Lebanon before a renewed military offensive? Or if China did it to the mayor of Hong Kong just before sending in troops.
It has a lot of potential when used together with other tactics.
People definitely attempt to do both of these things.
While we definitely can't (and certainly shouldn't) stop this tech from existing, there is still an important civic and academic need to address deep-fakes in conversations about media literacy. People should understand how to analyze the reputability of media regardless of the type. It doesn't matter if they're watching a video or reading a book.
― Upton Sinclair
If it's at me, I could just as well be building anything else. I find this technology fascinating, and I see an almost magical future ahead where we can tweak sensory input and play it like an instrument.
It's the closest we've come to building our dreams. The possibility of the The Matrix made more real, and bent to our own desires.
This stuff is going to sink Hollywood and replace it with an improvement at least an order of magnitude more imaginative.
So it's not that I'm letting personal interest or profit motives cloud my judgment. I think this is truly revolutionary, and I don't understand why others don't see the same glittering and fantastical future.
They're too afraid of the demons to build the cool thing.
Not really. Browsing the dark corners on 4chan and reddit, you'll often find people posting pictures of real people and asking if someone can put them into a deepfake.
It's not the journalists I'm worried about. It's the advertisers.
> Deepfakes are just the next wave of photoshop. People don't use photoshop to steal elections and win court cases. They use it to make memes. It's the same deal with deepfakes.
You say that now. By 2024 we'll be getting served political ads depicting "Person who looks like my cousin" in a riot, "Child that looks so much like mine" being shoved into the backdoor of a pizza parlor, or "Sad sack that looks like me" standing in an unemployment line. Fairly certain we all signed away the permission to use our likenesses in the various TOS.
It's also going to be a whole new vein for bullying, e.g. "Goofus hates Gallant. Most kids hate Gallant. Goofus posts low-grade deepfakes of Gallant dying and committing acts of self-harm. The bodies and the hair don't match at all since the source GIFs are from movies and tv, but it's definitely Gallant's face. Goofus gets a short term dopamine burst from his fake internet points as his peers pluslike and cross-post. One day Gallant decides, 'maybe they're right'"
I know it sounds like panicky, theatric, Black Mirror script stuff, but there are no missing pieces to keep either of these from being a button click away. It just might not be quite cheap enough yet.
But you're right, there's always money to be made in making the Internet an even dumber place than it is already. Have fun!
How many leaps away are we from 10 year olds making their own Star Wars movies? Not many, I posit. And I think that many of them can and will do better than George Lucas.
This technology is going to give so many more people the ability to create. As we begin to automate the tedious jobs and industries, it's important we have something fulfilling and engaging for people to move to. The creative field is rewarding and leads to self-growth and entrepreneurism.
The future is going to be a Cambrian explosion of creativity and expression. Look at YouTube, TikTok, and Patreon. Imagine what more tooling will do for these folks. Brains are teeming with ideas and imagination, but they often don't have the resources to breathe life into things imagined - with this next round of tech, we're going to change that.
Conversely, the concentration of wealth and production value at the top (entities like SpongeBob and Cardi B) will erode once everyone has the ability to generate character designs, animation, music, lyrics. More money will pump into the system, and it'll spread more evenly.
This is the Internet / Smart Phone revolution all over again.
TikTok is exactly the kind of thing I'm talking about -- you just want to create another vector for even more soulless, time-wasting popularity contests.
What you're describing to me sounds like a technological way for adult children to play brand-promoting dress-up inside the already-worn-out shell of pop culture, a way to infinitely recombine the old without actually creating anything new or interesting.
Call it a narrow-minded take if you like. I'm sure you'll be laughing all the way to the bank, so who cares what I think? It'll be a hit on 4chan.
Just my two cents, but acting is not as simple as wearing someone's face. If deepfakes make me look like an A-list actor, that alone will not get me a lead role in a big budget movie. I would still require acting skills.
And after that, the computer will make the stories and we will watch, at which point the Drake Equation takes over.
I don't think Disney will keep pace. This trend will cannibalize their IP and this level of tech competence isn't in their DNA.
There is a sizable group of people who do not know that The Colbert Report was satire. The problem with satire is that there will always be a group of people who think it's real, it's like a corollary to Poe's Law.
Source: Research: Conservatives believe Colbert isn't joking https://www.cnet.com/news/research-conservatives-believe-col...
Maybe we'll have to go back to sworn witness statements. Worked for thousands of years.
Sure it has the side (main!) benefit of empowering targeted advertising.
But fundamentally creating traceability between content/comment and a singular identity enables elimination of the most egregious abuses (mass-disinformation/ganging).
I'm as 90s internet-is-for-anonymous as anyone, but I have to admit traceability has merit in the larger social ecosystems.
https://www.microsoft.com/en/security/business/identity/own-...
Rather than let mega corps control identities, which should be building open source tools to allow developers to include decentralized identities within their platforms, preferably systems that interoperate with Ethereum.
It gets worse when businesses and adverts are included in the mix. Who is Bob's News Agency, really? Can you click on every advert to find who paid for it? No.
If someone is repeatedly toxic (which should be algorithmically identifiable), you can take steps to balance that.
You cannot do the same if 50% of your userbase lacks a stable / historical identity. At least without attempting to recreate identity on the basis of metadata (IP, patterns, etc).
What we need are institutions that are trusted to verify this data, like journalism. Except it is being allowed to be perverted for profit or ideology.
And isn't a journalist going to bring their own side of the story to it? Hardly independent.
A journalist that has a reputation for truth would hopefully bring that integrity to the process. Every party will have some kind of perspective or bias, we should pick the ones who are incentivized towards truth vs lying for profit/sensation.
I don't think the parent comment was using journalists as a specific group of people who should be the ones in charge of verifying video data. I took it more as "we need a dedicated group of people for verifying video data, similarly to how we had journalists verifying all other sorts of data for their articles since the dawn of journalism".
Goes without saying, but, also, the parent comment was clearly alluding to the "verification by journalists" in a traditional sense of proper investigative journalism, not "journalism" that is based on pulling random tweets without context from random no-names.
1. Physical evidence can at least by analyzed by a third party, and
2. Misleading deepfakes aren't created accidentally by honest people.
Where they fail, especially compared to evidence and video, is details like what did the noise sound like, what did the person look like, what were they wearing, etc
I also agree on video provenance/signing. Perhaps signed video will be something devices do by default in the future. If there are multiple devices recording, we can probably find a way to cross-check. Actually, that probably also exists. ;)
But they still should be spottable as Fake, if examined.
Related, a MIT project to build a KI to spot deepfakes (also ways to spot deepfake as human):
More practically, PGP.
Probably never going to work with cameraphone journalism, but if we're talking about newsroom & press conference footage you could make it happen.
How do you know if a signed recording is from the actual event, or from a camera being pointed at a screen in a pitch black room recording the playback of some malicious video trying to show said event?
Not really any different from other encryption. Your bank could be defrauding you on the back end despite the verified https session. But you trust they aren't.
[0] https://www.bbc.co.uk/mediacentre/latestnews/2020/trusted-ne...
[1] https://www.bbc.co.uk/blogs/aboutthebbc/entries/46f5eb33-b7b...
I think this only works if you're signing something currently much harder to fake, like a full light-field or something, and even then it won't stay hard to fake forever.
I think Apple could possibly start providing this for iPhone videos if they decided to as the hardware is all there already both the lidar sensor as well as the Secure Enclave etc
I'm curious what the equivalent of "3D" to "2D" will be for deepfaked voice would be.
It's possible to record and replay signals from gps satellites with different delays, so maybe that could be used to spoof the location, but if the camera also has an internal clock, perhaps it could detect if there was too much discrepancy there? But I don't know how much that could constrain the location. Also gps only has limited precision, especially indoors? Uh, maybe using cell towers instead of, or in addition to, gps would be better? Hm.
Edit: This would of course be predicated on PKI being safe and easy for most people to use which decades has proven otherwise though.
I'm more concerned with the manipulation of grainy low-res video. Police body cameras are an incredibly tool for police departments to fight misinformation (well, only when people watch the whole thing). Is editing these types of videos as obvious has high resolution video, or is it more easy manipulable?
It's probably going to be a cat and mouse game. Think about how far along green screen technology has come. You watch a movie in the 80s and 90s today and the green screening is super obvious to our 2020 trained eyes. Imagine how someone from the 80s and 90s would perceive the greenscreening from 2020?
I imagine there will always be a cutting edge of deepfakes that will travel halfway around the world before the truth comes out and won't spread to all the people that believed the fake.
I can't help but to think that as a society, we're entering the "schizophrenic" phase of popular reality.
Read these quora answers on what it's like to have schizophrenia and think "What would a society collectively afflicted in the same way behave?"
https://www.quora.com/What-is-it-like-to-have-schizophrenia-...
It's going to be weird to see society have to operate in a state of constant disbelief of so many things that were previously accepted as fact.
Basically, the ability to fake any video means that other data is needed to prove that the person/events in the video was in fact the person/events that that video is claimed to depict.
Other proof, other measurements of the same events, etc.
Deep fakes mean people will know they can't trust their eyes, they need to think critically. That's been true for a while, but now they'll know it.
Which I presume is why Parker/Stone are doing this now, other than the obvious (make money). They're going to force people to recognize this is possible and how perfect the fake can be.
But I guess this is wishful thinking given that we don't even generally sign written comments yet.
https://www.digitalartsonline.co.uk/news/creative-lifestyle/...
>In Canon's second version of its ODD system, the HMAC code is 256 bits. The code is the same for all cameras of the same model. Knowing the HMAC code for one particular model allows the ODD to be forged for any camera within that model range, Sklyarov wrote
To show that a video was created after a particular time, a person can include a hash from any public server in the video. The easiest way is to have somebody hold a phone in view of the camera, running an app that shows a QR code of a recent hash, updating in realtime. They could also read the hash digits. And the phone app could play the hash as audio tones.
Afterward, anyone who has the video can use software to extract the hash, query public servers for the hash's position in the immutable historical hash mesh, and get the associated timestamp. The existence of the hash in the video proves that the video was created (or edited) after the timestamp.
To show that a video was created before a particular time, use software to send the file's hash to a public server for inclusion in the mesh. A good recording app can publish repeated hashes of the file as it is recorded and gets longer. When the user stops recording, the app adds an "end of recording" mark to the file and publishes a final hash.
Afterward, anyone with good enough software can determine the time interval between the hash embedded in the video and the published hash. This is the interval when the video was produced. A malicious actor would have that much time to edit the video. With a good enough network, this could be 2 seconds, insufficient time for a human to make any editing decisions.
Hashes apply only to the original file. Anyone distributing a file with hashes must provide the original file that was used to create the hashes. Low-res resampled files can't be verified. Shortened or spliced files can't be verified. Only the alleged original file from the camera can be verified.
To fake a video, malicious software would need to insert a hash QR code into the video, process the video so it looks like it was recorded from a camera, and publish the hashes, all with a few seconds of delay. Fakers could also use a physical camera to re-record a video on a monitor. Either technique should be detectable. Fakery software will get better and may produce undetectable fakes. I hope that people will invent new physical anti-fake techniques to thwart fakery software.
I'm thinking of calling the system "Livestamp". It would work for any type of file or recording, not just videos.
https://petertodd.org/2016/opentimestamps-announcement#how-o...
If someone has confirmation bias, they're still going to find ways to call it fake with or without deep fake tech.
People don't need deep fake videos to believe lies, they do it with little to no supporting evidence anyways.
So you could easily take existing footage and change who it appears to be. But,
1) doing that to frame or unframe someone will be pretty niche 2) the original correct footage might still be findable 3) with access to the video files(which is going to be required in any legal situation), I'm sure there will be (exists for photos) algorytms that detect the editing the deepfake did to video file.
Of course, knowing all that is not really necessary for someone to shrug off truthful incriminating footage as “fake news”.
[0] One caveat is that deepfaking works better with higher quality footage. To produce a complicated scene with altered actor faces while maintaining realistic “phone video” look, it would make sense to film with good lighting and high-quality gear into log or raw format, and then imitate the look in post-production after deepfaking is applied.
Thus, one method of detecting such fakes could be by checking for traces of VFX, artificial noise, signature lens properties, signature behavior of phone video recording “magic” (such as noise reduction and stabilization), etc. Enough of producer’s dedication could make that tricky, but IMO it could be easier than applying automated deepfake detection straight up—it’d be buried early enough in post-production workflow, with a lot of noise introduced by subsequent “phone look” VFX.
This was treated (along with the ubiquity of CCTV in London) by the 2019 BBC show The Capture[0].
They glossed over a bunch of technical issues, but got the idea across pretty well.
Of course, they made the Americans look like the worse bad guys (no one comes out looking good, even the nominal "victim.") but that's to be expected, given the darkness the show attempts to purvey.
It's not great, but it absolutely provides context for the eventual advent of reasonable quality deep-fakes and the potential for abuse of the technology.
I'd expect that we'll see lots more of this sort of story telling, especially since any sort of video production (not including raw video footage) is never seen the way things actually are. As such, the "faking" (minus the deep-fakes with faces, etc.) is the key to the exercise.
Go watch the taping/filming of any television or movie and that will become immediately clear.
[0] https://en.wikipedia.org/wiki/The_Capture_(TV_series)
Edit: Added the missing link.
I could easily see a deep fake viral video go around in a sub-Saharan country, with a leader claiming he is gay or something else socially unforgivable in that country, and starting an internal ethnic conflict that cannot be undone.
Like for most of my life, this is the goal I keep hearing about. Now it's here, suddenly it's too real.
I dunno just seems a bit funny to me. I've never been one of those graphics people myself, but it seems like a case of getting what you asked for, but being upset because it's different to how you expected it was going to be.
It'd be likely better, if that decision is made based on character - the pattern of thoughts of the person, rather than the charisma - attractiveness or charm.
If it is possible to curb charisma/charm/cults of personality with realistic fakes, overall the effect might be advantageous.
I also don't think we truly understood how susceptible so many people are to made up nonsense until recent years, so this particular kind of made up nonsense suddenly looks like a serious threat to social stability and cohesion.
That's not true, that's been the basis behind marketing and advertising since it existed. It's been fairly well known for a long time people easily believe nonsense.
The article consists of 5 different small articles.
Does anybody know where the new show by Trey Parker and Matt Stone called Sassy Justice is airing? Is it a free Youtube series?
It’s going to get to the point where you can’t even say in a court room “well we have video of him doing this”. The fact that deepfakes will exist will erode confidence in even those things that are true. At the same time it will add additional fake situations to the conversation.
Worst part is that eventually even the AI-powered counter measures are going to fail eventually. The moment a computer knows what gives away a hint that it is a deepfakes and not real, a computer can solve to not present that give away. The “good guys” and “bad guys” will iterate with each other until it is perfected.
>The model for the discriminator is usually more complex than the generator (more filters and more layers) and a good discriminator gives quality information. In many GAN applications, we may run into bottlenecks where increasing generator capacity shows no quality improvement. Until we identify the bottlenecks and resolve them, increasing generator capacity does not seem to be a priority for many partitioners. [1]
Put another way: GAN training ends when the discriminator can no longer meaningfully distinguish real from fake. By definition then, the best generator will have no useful discriminator that can distinguish its output from real data. (conversely, if you did have such a discriminator, you could use it to train a better generator)
[0] https://developers.google.com/machine-learning/gan/generator
[1] https://towardsdatascience.com/gan-ways-to-improve-gan-perfo...
How far ahead do they need to be?
Suppose that it's cat and mouse, at least initially. Every six months someone comes up with a new way to detect the best known deepfakes, then six months after that there is a new way to evade that means of detection as well.
Someone drops a deepfake five weeks before an election.
You just described a GAN (generative adversarial network)!
I think the first question asked of a video is going to be "is this a deep fake?" just like we currently ask "is this picture photoshopped?".
It doesn't mean that we no longer use pictures, or that we don't alter pictures, just that we are more critical of them.
That said, there is just more realism in a series of moving pictures, but I don't see why the situation for a series of fake pictures has to be wildly different from the situation of a single fake picture.
By being so open about it Stone and Parker will just increase awareness of how easy it is to do, so people will know how realistic these things can look and not blindly believe what they see.
So, again, what these guys are doing is a true public service.
One thing that worries me is that everything is becoming immutable, saved forever, and digitally signed. Hard to claim you didn't post something if your account is super secure. You can't go anywhere without cameras taking your picture. And if someone steals your bitcoin or your smart contract has a mistake, your money is gone. You can't argue with an algorithm. And the internet never forgets. (Well, unless it is inconvenient for you, then Murphy's law applies ;-))
There are many such phenomena but I'd say they are all related: They mean you have less wiggle room for mistakes, or social deviance. If you are in a situation where you have to break a law, or if you are just having an affair, chances are someone or something is going to see you. There is no anonymity in the crowd anymore, on the contrary.
But this deep fake technology, if it really evolves to be undetectable, can be liberating, if it erodes the trust in pictures. At least the social control mechanisms based on cameras are going to stop working.
We were living in a very specific phase of human history, where we learned how to produce pictures from real scenes, but haven't learned yet to easily fake them. I just think this is going to come to an end, and we'll have to adapt. (I hope we'll adapt socially, and not just by cramming DRM into our technology like we tend do to; but that is a different topic.)
Audio and video evidence isn't admissible because it's audio or video (and may be inadmissible nonetheless). It's admissible because someone testifies under oath that they have personal knowledge of its provenance. The burden is on the party introducing the evidence to show that it's reliable, and the question of whether or not it is indeed reliable is a factual one for a judge or jury to answer. It's not assumed to be "true" or to accurately reflect reality just because it's a purported photo, video, or audio recording.
Ultimately you end up back at the beginning: holding people, and not pieces of information, accountable for their actions. That's an issue with civil society, not social media.
Plus, deepfakes have the potential to create much more noxious videos than real life. Real life videos depict real life people, flaws and complexities all included; deepfakes will be constructed to depict representations of the target that're most likely to generate inchoate rage of the mob. It's rare that the former happens to be exactly the latter.
Why not blame the person who confirmed your husband's death at this point if accepting comfortable illusions of someone easy to punish is what we are doing? Without them you would still have some remote hope of survival!
The first step of solving a problem is recognizing the actual problem - what we find comforting is only a distraction for rationalization purposes. Better to recognize that microscopic organisms caused the crop failure instead of burning an old woman as a witch.
Politicians don't have a particular right to remain in office despite inciting violence, even if it gives them the sads if they face repercussions.
If the video weren't fake, would that somehow make the murder of Rohingya acceptable?
Just one example: A radio station (RTLM) in Rwanda laid the groundwork of genocide by dehumanizing Tutsis, among other things by referring to them as "cockroaches" repeatedly. The station was deemed instrumental in the resulting murders of at least half a million people.
It's not fear mongering to look at the facts of history and see that propaganda is a primary way that violence scales. Deep fakes are another tool which will certainly be used towards these same ends. What propagandist would not want to use such a tool?
Propaganda is a primary way that human action and organization, whether violent or not, for good or ill, scales.
Does it though?
Suppose there is a theft at a company. The police go to the company's security team and get the surveillance footage. Presumably admissible.
If it was an inside job, the surveillance footage could be a deepfake showing someone else committing the crime. Or maybe it's real surveillance footage. Without some way to distinguish the two, how do you know?
To rip from current headlines: https://www.justice.gov/usao-wdpa/pr/erie-man-charged-arson-...
(Leaving aside the merits of the case / opinions, and just using it as an example)
".. the [Facebook Live & coffee shop] videos depict a male – with distinctive hair to the middle of his back wearing a white mask, white shirt, light blue jean jacket, black pants with a red and white striped pattern down the side and red shoes - setting a fire inside of Ember + Forge."
"A review of additional Facebook public video footage from the area of State Street near City Hall in Erie on the evening of May 30, 2020, shows the same individual without the mask but wearing identical clothing and shoes. The subject’s face is fully visible in this video footage."
And that's a federal arson charge (min 5 years, max 20 years prison).
You cannot underestimate the global political ramifications of this. If you think the amount of video manipulation was bad enough this cycle in the 2020 presidential election? Imagine a few years from now when video's are being put out by political operatives and people with nefarious agenda's with impunity all over social media.
This sort of thing absolutely terrifies me since you can start to twist reality to whatever you want and influence people in ways we never thought possible. I really feel like the genie is out of the bottle and this has the potential to become a very dangerous tool for people with bad intentions.
You are assuming people will have never heard of these deepfakes. Sure there are still a few grannies today who have never heard of photoshop. But your average twitter user?
Every piece of controversial video content will have a group loudly screaming "TRUTH!" at the same time as another group screaming "FAKE!", and it will just amplify the social polarization we're already experiencing.
Well, something really bad is definitely ahead with fake videos (as if real videos, with significantly meaning-changing omissions made with the plain old cutting process weren't bad enough) and people with high visibility creating awareness by doing it in the open is the closest thing to a defense that we have. It's a hopelessly weak defense but better than nothing.
A weird thing about this thread, and others like it, is that there seems to be this broadly shared implicit premise within the thread context that the feared ill effects of new technologies like this have not already been prevalent in our societies and information ecosystems for decades. In a thread about bias, fake news, propaganda, etc, people seem to have no problem realizing and acknowledging that we already have a very serious problem (often only visible in one's personal outgroup, but that's better than nothing) - but when the specific topic of conversation is a new technology, the majority of the comments seem to be written as if we don't really have any significant issues currently. It seems as if there's some sort of a phenomenon whereby the logical methodology for evaluation of the situation changes according to the topic, as opposed to there being a consistent methodology that at all times has an explicit awareness of the ever-present bigger picture.
Here's [1] an 8 minute video on Presidential debates. This fairly well demonstrates how this aspect of our political system is largely pure theatre...and yet, intelligent people often speak (again, depending on the specific(!) topic of discussion) as if this charade is highly legitimate process, within a larger political (and journalistic) process that is also highly legitimate.
The way I view the ecosystem is that the vast majority of things are to a very large extent ~fake (in whole or in part). Cranking up the absurdity to 11 in classic South Park style, making a complete mockery of both the politicians as well as those who can't consistently(!) conceptualize the true nature of our system, seems like an excellent response to a situation that has been sorely in need of some good old fashioned satirical mocking for decades. Western society & politics lost the right to be taken seriously ages ago - admitting to ourselves that there's a problem seems to me like a prerequisite first step in fixing it.
[1] Winning the Presidency: Debating
ex:
Trump access hollywood tape -> fake Soros conspiracy to save the pedophiles
I believe the optimum approach is to be concerned with all risks, and weigh the magnitude of each in a state of careful self-monitoring of one's potential biases (and ideally, have your conclusions reviewed by others, preferably from a diversity of ideologies and perspectives in an attempt to minimize the well known affects of groupthink). Noteworthy to me is that a significant number of people (if not the majority, depending on which community you are in) are easily able to see the epistemic errors in their outgroups thinking, but has more difficulty in doing the same within their ingroup.
For example, in your comment it seems that you have noticed shortcomings when it comes to politicians of one general ideology, but I wonder if you are of the belief that this phenomenon does not occur across all ideologies?
Huh? Don't deepfakes do the exact opposite - make plausible deniability in audio and video a much stronger argument?
Before if you denied a video of you was genuine that would not be plausible. Now it would be plausible.
https://thehill.com/homenews/media/523045-cable-news-october...
Tucker Carlson spent the month of October pushing Russian disinfo on Hunter Biden, all for it to culminate in "losing" his documents, re-finding them and then suddenly, inexplicably backing down.
https://www.thedailybeast.com/tucker-carlson-suddenly-says-i...
Good. We should dispense with retributive justice, and replace it with restorative, transformative systems which are a noop on the wrongly accused.
That's good. Courts should not have high confidence in any single bit of evidence. It can all be manufactured. And long before now. It's just (too) hard to manufacture, not be detected, and get larger number of people required to "go along" with it, when you have manufacture multiple collaborating pieces.
I can imagine lifelike movies that render characters to the users preference in real-time.
Talk about “representation”!
I can imagine as well dialects, language, etc being rendered in real time to adapt to what the user prefers. You and another person could watch the same film and talk about the same story but have totally different experiences on what the characters looked like, talked like, and even said within some parameters.
Making a movie with humans will be a prestige event like riding horses today or driving an ICE in the future. They don’t be able to compete with rendered film at a mass scale due to cost. Rendered films could be built and distributed cheaply and cost a fraction of a real movie to watch.
What would be interesting would be the techniques used. Are they like animators or roleplayers focused on a single character to give them emotional touches in added details, quirks, and improved line changes or "greenscreened" such that what they actually look like is utterly irrelevant to their job?
The reality is that 'we' were too trusting of what is presented on screens as true. Its the main piece that is used to manage and govern us.
So, I welcome distrust on what we see on screens - that trust was always misplaced, and all about manipulation rather than information.
Only people that know you on, at least, a last name basis, should have political power over you.
Democracy has accountability by virtue of everyone having a vote. It’s a small power that everyone can use to hold their leaders to account. But it still allows people to hold arbitrary power over each other. If you can convince enough people, you can apply your morals and beliefs on others. For instance, lots of discrimination is a function of democracy and codifying oppression of certain people.
We sit here furiously debating who can use a bathroom, who gets preferential treatment, who you have to interact with and a million other things. “Both sides” are bent on forcing people to behave a certain way and they use the power of the vote everyone has to accumulate power and make things “how they ought to be”.
I think this could be done democratically but everyone has different interests. How do we find a common, singular interest and then optimize around that?
Or maybe democracy is the “best bad system” and we just have to make do. I do believe with the hyper connected world we have today, cryptography, and resource abundance that we could transcend the modern system and discover liberation from each other to be ourselves and pursue truly enriching lives at a mass scale within local communities. And this means a different thing to different people. But just about everything in our modern system would need to be disposed of and recast.
Any time I see a "this is peak technology" comment, I'm always reminded of the PC gaming magazine cover showing the first Unreal game's graphics ("Yes, that's an actual PC screenshot!").
It looks awful now, but in the nineties, it blew us all away.
https://i.pinimg.com/originals/d1/5b/d9/d15bd9f544e6fd4bf740...
Because he's not the cartoon supervillain you think he is?
Would be used as evidence of how untrustworthy the Dems are.
This video of two opponents endorsing each other cropped up last year.
-------
I played maybe a hundred games of among us (they can be very short). The game is about one or more imposters trying to murder the rest of the crew but you'll have to be discreet and get/find people alone so you don't get voted off. When a body is found a meeting happens and you can lie (text chat)
One problem is you don't want to accuse someone when you're an imposter because you immediately become suspicious. Most games will tell you if you voted off an imposter or not so they'll know you're lieing right away once game tells them they voted a non imposter. Most of the time you want to accuse noone, play dumb and act like you're everyeone else and saw nothing.
I lost count when a guy doesn't accuse anyone for 20+seconds, get accused then claims the guy who found the body is an imposter and all these things he did that are suspicious. (why didn't you say it right away?!). Like 90+% of the time the guy being accused is the imposter who waited so he can feel the situation out. It's extremely obvious but maybe 70% of the time literally every player but me and the guy reporting the body is fooled. Which is far too many players at a far too high fool rate. It's so painful because it's so obvious. 90+% of the time in that scenario the reporting guy is telling the truth.
I think making a studio that specializes in this type of work is genius. This technology could really change the industry in interesting ways. Faster iteration, less re-shoots, less extras.
I guess you could even "storyboard" with rough deepfakes.
Pretty interesting stuff. I wonder what else people will come up with if this stuff becomes a shrink wrapped industry standard.
Why do we need to wait for the media to tell us it's interesting?
It's the first attempt at mainstream commercialisation of Deep Fakes I think? It seems like a big deal. It's meta still, they are cheating a lot.
But it's a working example of whether deep fakes might or might not work in the near future.
Trump wrecked South Park the same way he wrecked the Daily Show. These people are past their time, their satire is no longer relevant; they too are missing the point.
Didn't take long for the MAGA brigade to find this post.
I could see clips from this show making the rounds on lower-information parts of social media like Facebook that are already prone to being hoodwinked by conservative fake news like Alex Jones and the like. Professionally-produced deepfakes becoming the exact attack on democracy that other people in this thread are proposing will "innoculate" us.
And yes, all politics and public figures warrant criticism and satire. Cast aside your sacred cows.
I fully expect the outcome to be increased tribalism, when faith in your tribe's leadership and information sources is the only source of comfort, confidence, and "truth".
I don't think it will be the opposite where everything is questioned and critical thinking suddenly becomes prolific as you seem to imply.
Are you imagining if Tucker Carlson were punched in the face at his local coffee shop, he'd shut up? :rolleyes: His following would only grow larger and more supportive as he milked the controversy for all its worth.