Deepfake scams are here and we're not ready
twitter.com
twitter.com
At last year’s Super Bowl there were real celebrity endorsements of a real scam, FTX.
Honestly, the big scams are going to spend real money and get the real endorsements. I don’t know how much difference in real life deepfakes will make.
these deepfakes will be done on the cheap, and aimed at those likely to buy the BS -- i.e. your mom. Don't mean that as in insult ("yo mama..."), but like literally, these are the scams that will target elderly, non-technical women, kids, etc.
Remember 10-15 years ago? If it was on the news, thats because it happened. If an important politician said it that's because it's important. If the 'expert' said it then it must be true! If you need a refresher go visit old political threads on reddit from 10+ years ago. You'll recognize your old sheltered political views, and yes it was somehow even more naive and coddled than reddit of today.
Deepfakes will be a net good. It will make more ideas have to stand on their own merits (because there will be limited authority to validate the medium thats conveying it, mitigates reliance on fallacies) if you can make <famous politician> say <total opposite of their entire platform>. It will put a damper on how seriously people take everything on the internet outside of official sources (thank god). It will effectively weaponize social media's ills against itself, the more its abused the greater that effect.
People are finally starting to ask the questions they should be asking on a large scale, "where is chatGPT taking us, do we want more of this?", those kinds of meta-questions were never being asked even a few years ago. Almost no one asked that about smartphones when they came out. Better late than never.
This is pretty strong blanket statement. A disturbing amount of people have bought into conspiracies like Qanon in the last few years.
It was only the last 100-150 years or so that people have really gotten locked in to the single minded ideology wars while totally locking down their sources of information.
The internet really was that though until the last 8-10 years. Once it reached a critical mass and started being actively gamed by world govs as a major objective it changed dramatically to what it is now. Now the masses are becoming just as paranoid and skeptical as a baseline as the cypherpunks were.
This is a reasonable position to take, but I'm afraid it's somewhat at odds with the current reality on the ground.
The problem being stated here isn't that "absent the ability to provide image/video/audio evidence and trust it as untampered, we will be left adrift in a sea of confusion", it's that we aren't ready to compensate for the problems that lack of trust creates yet.
For the vast majority of people in our society, nothing has changed since 15 years ago. They still trust that everything they see on the news (at least, the news source of their choice) is real. (This despite the fact that it's been terrifyingly easy for decades to create a narrative with no actual alteration of the source footage—just clever editing. See, eg, "The Illusion of Truth", from the 4th season of Babylon 5, aired in 1997, for a great illustration of this effect. Not to mention a great example of how science fiction draws attention to real social issues of the time, but can't easily predict what technologies we'll have available—or they just would've made deepfakes of John and Delenn there instead.)
All in all, I hope you're right—and I think that ultimately, we will develop a more robust ability, as a society, to make up our minds based on principles and values rather than on who says what...but I'm afraid that we will have some very rough times in the near-to-medium term before we can get there. Because we're not ready yet.
I'm less pessimistic on how many people believe media at face value. There's been large reduction in people who do, and the beginning of that shift is willingness to doubt their 'trusted' sources.
When someone says 'this happened' and you say 'actually that turned out to be fake news/not the whole truth' there's a much greater shift in reaction to that than a decade+ ago. The kneejerk is increasingly less condemning.
The best lies, and the best cons, always use the truth as a weapon
The real value is in creating deepfakes of politicians that can influence elections.
Anyone who disagrees join my start up where we generate the voice of dead family members so we can get people to pay indulgences to escape hell.
Regulation is fine, but prohibition rarely works.
Here's the thing, you're almost most likely to be successful as a scammer with a poor deepfake with a good deepfake.
This is why so many scams are so unbelievable in their premise (Nigerian princes , UN Lottery, etc...). Falling for the premise is a good indicator that the leads they get will turn a profit because they indicate vulnerability.
This is partly why I don't think regulation would stop much of the scamming.
So I think it does matter how authentic the deepfake appears. I’m ambivalent on whether banning the technology is a good idea or even how feasible it could be, but to the extent that deepfakes are limited I do think it would be bad for scammers.
But there's a thousand other ways to get that exact info, even/especially online.
But you can Google "the anarchist cookbook" and get that information just fine (for better or for worse[1])
[1] https://warontherocks.com/2018/09/books-as-contraband-the-st...
No surprise Disney (and others) are going to be getting on that bandwagon.
I'd like to ban "social media", but how do you define that in a way that kills what you want (eg twitter, facebook, /r/all) and preserves less detrimental forms of it (eg HN or old phpbb forums) ?
edit- Oh and furthermore, how do you deal with every country who decides to not ban it?
Why is it the difficulties of regulating something brought up as if it means it prevents any action taken at all.
“Id like to ban murder but how do how do you define murder and ban the cases we want (murder) and allow the cases we do (self defense)? And what about countries who decide not to ban murder?”
Seems reasonable on its face, but the real issue then becomes damage done-vs-effort needed to prevent it
Your example is murder, but the problem is then a scaling issue; not many people really want to commit murder. It takes effort, and the sample size is so small we can evaluate it on a case-by-case basis. But it's trivial to deepfake one face onto another, so it's likely a lot of people will try it for their own amusement.
So basically murder to most people is brutal and ugly. A Deepfake is clicking a few buttons; it's abstract and hard to see as a real crime for most people.(...ya know, kinda like how I'll pirate a song but I'm not going to stuff CDs into my pockets at the local retail store)
And now we can expand that idea- what counts as a deepfake? The face editing is obvious here but what about training an AI to replicate someone else's voice? What about using the Lamba character.ai site to replicate Sonic or Indiana Jones? That's where it really gets tricky and where I feel politicians will be at a loss to construct any meaningful legislation.
From my perspective, seems quite difficult
People should have to learn not to be scammed rather than trying to ban scams. People should be taught that speech is not violence instead of trying to ban speech.
If this development is what it takes to finally empower people to not take their health and fitness advice from Joe Rogan and some other internet chumbo, I say it can't come soon enough. That a few additional people might fall for these things is a small price, and one I'm willing to pay.
What you are talking about is simple prior restraint on computer generated art, in the cases where the speech is not already covered by fraud laws.
Not only is the behavior that is being complained about (people fraudulently pretending to be something they aren't) already illegal in most jurisdictions the unintended consequences of such a poorly thought out policy are complete unconsidered.
The objections are obvious but to list some:
* Behavior should be regulated, not technologies. Otherwise someone will use an alternative technology to perform the same attack and it won't be covered.
* This behavior is already covered by fraud regulations
* How would you even "ban AI" when people can't define it
* The only approach that could be considered would be to take a licensing approach to writing any software. The downsides to this are considerable and hopefully self-evident to this audience.
Celebrity/influencer marketing is one of the few marketing strategies that actually works consistently.
What’s the world coming to if you can’t trust that the person pushing a product isn’t actually a paid corporate shill but is instead a scammer trying to separate you from your money.
There’s probably a distinction between the two, right?
Paris Hilton promoting CryptoKitties vs a fake Paris Hilton promoting CryptoKitties — the difference is obvious.
However influence culture already does this. Influencers represent identities fostered entirely for the purpose of convincing you to consume product.
You could make similar claims for the mountain of garbage writing that falls under the category of "content marketing". It's an attempt to create authentic information for the real reason of selling you a product.
The point being that influencers, content marketer and similar sophisticated advertising strategies already are artificial intelligence. Joe Rogan is a personality created to sell you products, most blogs are empty content designed to attract your attention just enough to convince you to trust them to sell a product.
The distinction between the real Joe Rogan selling you a product and an artificially generate Joe Rogan selling you a different product is only a concerning thing to people paying Joe Rogan to sell their product.
For the average consumer, they have already have been duped by AI for years now.
Sorry, broke my own standards. LTT = Linus Tech Tips. YouTuber.
Unclear if the OP would prefer makeup recommendations from Linus over Kim Kardasian or if they just don't buy makeup.
Not to be confused with “Linus” of Linux fame who doesn’t contribute to the channel.
Anything other than that doesn't matter. It doesn't matter if the influencer is a YouTuber who does their original research and tech breakdowns to give an informed opinion of a product, or if it's an Instagram model that is willing to hold a purse for a photo. Both, when paid to endorse a product, will influence people into buying that product. LTT's sponsor breaks in the middle of videos (that usually have nothing to do with the subject matter of the video itself) influence people into buying those products. So, LTT is an influencer.
The consequences of moving to a zero trust society will be major.
If the video was a re-post or hosted elsewhere - then you would start to reconsider it?
Key sharing is annoying but at least you have continuity in the same sense as normal “reputation” and Keybase showed us it’s possible to like PGP keys to social media accounts with decent usability.
And most people say they wouldn't fall for it, but for some things, you might not need to move the needle very far.
I think it might be refreshing for everybody to completely disbelieve everything they see and hear on screens. Where we can only believe what we can hold in our hands in front of us.
It's going to be hard to run a democracy though.
Even during the crypto boom there were deepfakes to manipulate the market, which didn't even work.
The real problem are the not-endorsements. The "our cameras caught this when he thought his mic was off". For politics especially its the unscripted moments people believe.
Equally it offers plausible deniability to anything that is captured. Just claim its a deep-fake and the vast majority of supporters will believe you,because they want to.
Phrases like "a picture never lies" or "I'll believe it when I see it" all start falling away.
In a world where people want to be right, not informed; when anger wins more elections than hope, where fear of the future overwhelms the comfort of the present, deep fakes erode one more thing you can trust.