Deepfake used to attack activist couple shows new disinformation frontier
reuters.com
reuters.com
I'm stuck in a loop with my father for a while now where he forwards me a certain sort of propaganda. I feel like I've struggled to break through on a key point: when I say something smells like propaganda, it doesn't mean I think it is false--it means I think it's part of an organized campaign to build narratives and consensus.
In my case, the topic is highly charged. Reapproaching it with something low-stakes like this is at least worth a try.
This piece might encourage someone else to research a different story and thus it amplifies the goals of the original propagandist. Many folks in a psyop/propaganda campaign are unaware of any sort of coercion. If the receiver understands it as propaganda the effort has failed.
It is an exciting time for sociologists, the disinformation war that is occuring right now is global in scope. If the cold war was WW3, we are currently in WW4.
The very best propaganda doesn't lie, it simply is very selective about what truth it shares. By carefully choosing your facts, using loaded language, and by properly framing your statements, you can make almost any argument without lying.
Intuitively it'd seem that the greater dissemination of information with social media and messaging apps would make people smarter and promote critical thinking, but in reality they seem to have the opposite effect. I still keep wondering if it's the environment or his age that was the primary factor. Do we become more gullible and less discerning as we grow older?. Perhaps I'll be much worse when I'll be as old as he was.
I naively believed this too. It's wrong. The Internet and the various apps, sites, and social media things running on it aren't an information dissemination system, they are a data dissemination system. They take bytes and get them in front of many humans. The system itself cares not one whit as to whether the data it transmits has any connection to reality.
The result is that the Internet functions as a mostly indiscriminate amplifier of its inputs. Sadly, it is much easier to input lies than truth. First of all, creating lies takes much less effort since no measurement or fact checking is required. I could tell you a hundred lies about what the neighbor next door is doing in the time it took you to go over there and ring the doorbell. Worse, my lies can be crafted to take full advantage of all of the human biases and flaws in the social media systems. I can make lies that are emotionally loaded, play into stereotypical narratives, or have evocative photos. Meanwhile, you foolish truth-teller are restricted to only the narratives that actually happened and the imagery that you actually took of the event.
When you think of London in the Industrial Revolution, do you picture all of the bright shiny products of industrialization? The newly affordable textiles? Or is it more an image of a city covered filthy coal soot? It feels like we're in the latter state right now: buried under the pollution of disinformation.
We need to think of "information pollution" and need an "information environmentalism" movement if we're going to survive the Information Age.
I think my father is still sharp and have several competing quotidian explanations (projections?) of what's going on.
But, it's impossible to silence the nagging worry that there's a chemical/mental imbalance brewing, or that this will be one of those things I'll look back on some day as an early sign of something.
Maybe "infomorph"?
Also it reminds me of Animorphs!
Reuters and other media will fetch deepfake to mean more things, and it will become a category of related frauds.
I have to wonder when the meaning change occurred. It's hard to tell if they got it from an interview, or a misunderstanding introduced it into the article.
>A generative adversarial network is the name given to dueling computer programs that run through a process of trial and error, according to Hao Li, chief executive and cofounder of Pinscreen, a startup that builds AI avatars.
>One program, the generator, sequentially fires out millions of attempts at a face; the second program, the discriminator, tries to sniff out whether the first program’s face is a fake. If the discriminator can’t tell, Li said, a deepfake is produced.
https://graphics.reuters.com/CYBER-DEEPFAKE/ACTIVIST/nmovajg...
To expand on the parent comment's point, this is most likely using some variant of StyleGan2 which has plenty of endpoints on the Internet to make it easy for someone to just download a face for free.
However, they all have this uncanny StyleGAN glare. And there are still artifacts. Quality's improving though. Scrubbing it of EXIF data doesn't really deter detection.
Go ahead, try posting some StyleGAN avatars to Facebook or something. Your account will get flagged immediately for doing so. There's already a robust amount of interest in "deepfake" detection, and there are methods to fingerprint or adversarial attacking your machine learning models with data poisoning so you can reasonably determine a dead giveaway. Kind of like putting a dye pack in your bills.
What is NOT helping, is how media outlets can drum up some cheap panic over the "end of truth" or insinuating how the barrier for disinformation campaigns is lowering. Both statements can be valid but I think we are teaching the wrong lesson having people told they can't trust anything rather than to establish what trust means to them in the first place.
Probably something as simple, and catchy, as deepfakes (which was ultimately linked to the initial user from github, if I remember correctly).
Surprised that publications don't do more research as anyone can create a social media profile that looks legit these days.
Trolls have been here for years and they'll continue to find new ways to exploit weaknesses.
This, on the other hand, is a case of a fake online profile being used to say terrible things about the activist couple. The only notable thing is the profile photo was computer generated.
As similar to several created on twitter to spread misinformation
Deepfakes create an "adjacent identity" to trick people into aligning their opinions with someone who is in the same group. But, if we were aware of this, and removed the identity to strictly review the information, it might change the way that information is received.
This would only work if a majority did this, so it would never work, nevermind.
There are differing views on how much to showcase authors (or reporters) names, and some organs (The Economist, Hacker News) intentionally hide or de-emphasize these.
Almost all questions of identity actally revolve around trust in one of its various guises: trust, credit, accountability, entitlement, and making or receiving of payment. In creation or relating (mediating) information, the persistence of authorship attribution provides a bundling handle for trust. An author might file numerous stories per year, month, week, day, etc. Assessing and assigning trust to each story is expensive. Bylines, editors, and publications accrue positive or negative associations with time. No, trust is not fully consistent or transative, but as an efficiency heuristic, rolling up and bundling reputation offers powerful gains, and numerous checks.
Even a pseudonymous source (you're reading one now) can accrue a certain reputation.
Yes, the crowd on HN is more likely to do this, but I suspect it is very overwhelming for the average news reader to correlate so many data points.
and:
> Artist Mario Klingemann, who regularly uses deepfakes in his work, said the photo “has all the hallmarks.”
As an amateur photographer who only* knows enough Photoshop to quickly enhance images, I find these two statements completely unconvincing. This type of he-said-she-said news article makes me uncomfortable. TTBD: Truth To Be Determined.
* Edit: I know nothing about deep fakes but I expect "fake faces" to have "hallmarks" in the face.
At best Photoshop can play a role in covering tracks of evidence or artifacts of this much more sophisticated approach to faking identifies.
It’s not to say the developers and scientists at Adobe are lesser, it’s that it’s not the same tool or problem that’s being solved.
Put crudely Photoshop can let you draw a mustache on someone’s photo. This is about inventing a photo that never existed before.
They weren't expressing expertise in Photoshop. They were saying that basic use of it is the extent of their expertise.
> What do you think is unconvincing about them
Them being the 'experts' - they did nothing to convince other than say "I'm an expert in this thing."
adobe research scientists are crazy-strong in the area of deep/neural graphics[1]. Perhaps we should disentangle adobe research from photoshop?
In an article in U.S. Jewish newspaper The Algemeiner, Taylor had accused Masri and his wife, Palestinian rights campaigner Ryvka Barnard, of being “known terrorist sympathizers.”
This appears to be a "thispersondoesnotexist.com" image. Is that site considered "deepfake"?
I always assumed "deepfake" to mean altering existing images to add actual people's likenesses (like the classic "celebrity porn" images). In particular, applied to video.
This type of thing is likely to become de rigeur, for astroturf and scam accounts. Disturbing, but not particularly newsworthy.
Before, people would just scrape images from stock photo sites, or even from some poor sap's Facebook photos. Doing it this way simply makes it less likely to be uncovered.
It's newsworthy if the fake account's posts are used to assassinate the character of real people and the fake persona is pointed out for being fake:
> Reuters was alerted to Taylor by London academic Mazen Masri, who drew international attention in late 2018 when he helped launch an Israeli lawsuit against the surveillance company NSO on behalf of alleged Mexican victims of the company’s phone hacking technology.
Since it is not deepfake ( it may very well be an enhanced pic for all we know ), question becomes what is the purpose behind this author's post. Did they just learn about deepfakes and wanted to capitalize on it? Did they want to propagate a specific point ( and if so, which one )?
I miss the days I took articles at face value ( ignorance is bliss ). Now, I can't help myself and analyze former bastions like reuters. It is sad to me.
The technology got easier, everyone became a publisher, and now we have a more prickly name for it.
2. This looks like any other sock puppet account but the benefit of using a generated person is that they won't accidentally cross paths with the real one and get outed.
This is not an innovation just marginally more effective approach to obfuscating your trail.
I blame social media. It comes with the implication that it somehow makes sense to care what total strangers think of things.
All based on created or assumed identifiers and imagery.
I am of course referring to myself. Though there may be others.
Perhaps you meant, "_without_ verifying who is writing them"?
Anytime I see things about GAN, deepfakes and bots it reminds me of the videogame Deus Ex Human Revolution (2011).
The game is set in the not so distant future and revolves around technology, ethics and humanity.
It touches on various concepts like; AI, biomedical advancements, hacking, weaponry and civil rights.
The particulary profound bit that I relate to situations like this is around a TV news broadcaster called Eliza Cassan.
---Spoilers Ahead!---
Eliza works for one of the most powerful media and news agencies, called Picus.
She is a celebrity and with her abilities to get the latest news and coverage, she is able to direct the conversations on various political and ethical issues that I mentioned earlier.
As you progress through the game and break into the Picus News HQ, you come to realise that no employees have met or dealt with Eliza Cassan in person.
This is because she is revealed to be an AI powered hologram composed from what I would assume are desirable human aesthetics.
It is not established who controls Eliza but you can tell she has a degree of autonomy but is penultimately controlled to operate for the benefit of her operators.
Her AI is also intrusive and appears to be able to mine data from around the world (cameras, hacked devices), allowing her unprecendented global access which she uses to analyse an direct conversation for her viewers.
A thoroughly good game which gives technologically pro/negative people a taste of what our future "may" be like.
Not sure how accessible is the Israeli tool used for detection of manipulation in the article.
There is an urgent need gap for accessible tools for detecting deep fake[1](Sorry for the plug, it's my problem validation platform, if your startup has such tool or you know of any open-source tool feel free to comment there).
[1]https://needgap.com/problems/21-deep-fake-video-detection-fa...
https://en.wikipedia.org/wiki/We_begin_bombing_in_five_minut...
https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
The cartoon is from 1993
Plato's cave allegory is going to be a very real, practical concern for people in a few decades. There won't be any record, any evidence that couldn't trivially be falsified at a large scale. You can have AIs generate serious-looking thesis written by serious-looking people with serious-looking credentials from a serious-looking university in a real-looking city and all of it be faked at scale. Terrifying indeed.
I'm here if you got any questions...we're genuinely trying to bring transparency, in a complex battlefield of falsehoods.
GAN images perhaps are not called Deepfakes, I guess.
GANs have not been seen in the wild except ones taken from https://thispersondoesnotexist.com/
If this is not taken from https://thispersondoesnotexist.com/ then it's a really big deal, this is what would be interesting if you can prove it either way.
I would be stunned if there weren't thousands or perhaps millions of "people" on Twitter that are just mercenary bots farming timeline history and meow-meow beans, waiting to be activated. It's just too easy NOT to do, right?
For an interesting look in to this, see The Commissar Vanishes: The falsification of images in Stalin's Russia.
https://www.indexoncensorship.org/2017/08/commissar-vanishes...
https://www.amazon.com/Commissar-Vanishes-Falsification-Phot...
Dangerous for the government, good for the citizens.
In newspapers? There's a trivial solution: check that your authors actually exist and are who they say they are.