Deep Live Cam: Real-time face swapping and one-click video deepfake tool | Hacker News Reader

Deep Live Cam: Real-time face swapping and one-click video deepfake tool

deeplive.cam

251 pointsblini2077168 comments

FTA: "Ethical Use Safeguards

Built-in checks prevent processing of inappropriate content, ensuring legal and ethical use."

I see it claims to not process content with nudity, but all of the examples on the website demo impersonation of famous people, including at least one politician (JD Vance). I'm struggling to understand what the authors consider 'ethical' deepfaking? What is the intended 'ethical' use case here? Of all the things you can build with AI, why this?

For many (notably mastercard and VISA), when they say “ethical” they really mean anything but porn.

that seems like an overreaction. the card processors ban much more questionable trades - such as weapons and terrorism financing

That's because the state is forcing them to.

The state has never stopped the funding of terrorism.

https://www.nbcnews.com/politics/us-taxpayers-may-funding-ta...

Sure, but that doesn't mean it wants competition in that space.

I can't find anything to support your claim about weapons. Seems pretty much all online arms dealers I can find selling anything from grenades, machine guns, and even rocket launchers take credit cards and I'm fairly certain stores also accept them too.

I am not sure what the current state of the issue is, but there was an initial effort to restrict gun sales in various devious and deceptive ways since it is illegal to overtly do so because it is legal trade and economic activity.

I would not be surprised though if the clear illegality of the violation of the Constitution of such efforts were brought to the attention of the payment processors, and they were reminded that they would severely regret hastening attention on an effort that still needs to happen, a public electronic payment processing capacity.

It's not illegal to not enable weapon dealing. Try selling a Glock on Amazon and let me know how long it takes before they ban your account.

> grenades, machine guns, and even rocket launchers

Umm, yeah - what country are you buying live grenadesor working rocket launcher online with a Mastercard? Cuz it’s not the US or Canada. And if it’s not a live grenade or working rocket launcher, it’s no different than any hunk of metal.

uh. yeah... us and canada are a tiny fraction of the world, also what is really buying using visa or mastercard? if i use visa to byy crypto and then get explosives (which can be transparently done) there is nothing they can or will do about it... - buying things online has nothing to do with countries or borders, nor is it always clear, to payment providers or even customers, what kind of scheme enables a payment..

I buy weapons with my Visa card all the time.

And backed off from blocking Onlyfans.

That they do, but perhaps the relevant context is that while porn is globally unregulatable, but the one entity that has proven its ability to regulate it (or at least exercise some control over it) have been payment processors like Visa and Mastercard.

FT had a fantastic podcast on the porn industry and the guy behind Mindgeek. Like many stories about multinational entities, you constantly hear the usual refrains - noone can regulate this, the entities keep changing their name and face, there is no accountability, etc. But when Visa and Mastercard threaten to pull their payments, the companies have to listen.

Visa and mastercard are the de facto regulators of porn today, and mostly do so to prevent nonconsentual and extreme fetish stuff from being displayed on mainstream platform.

From what I gathered from the podcast, they're not super keen on being the regulator - but it's a dirty job and somebody has to do it.

They don't care about the content, they care about the correlation with customers who have an exceptional rate of chargebacks or other payment avoidance on legitimate purchases.

Cryptocurrency and the like may offer a way out of that problem by allowing direct purchases, but only for companies willing to deal with the support burden of making everything nonrefundable.

Finance is also an extremely socially conservative industry. There are very recessive attitudes behind closed doors.

And also the recessive policies come out in the open in front of the world from people like the ones who created "operation chokepoint", who are perhaps not help us as being super 'socially conservative'.

Your choice of entertainment, information and tools is under attack from all sides when they can get away with it.

cryptocurrency doesn't offer credit card chargebacks, but why can't you refund customers by sending them the same amount of crypto back from where it came from? I've gotten merchants to give me refunds to a different credit card before.

Chargebacks are done by the customer and card agency, not the merchant.

There have been some instances where they have been 'about the content'-

The statements about chargebacks and porn were certainly true around 20 years ago, but that became less and less over time.

I don't think this is the case - because then we'd see them pull card services for porn websites altogether. This clearly didn't happen, nor was it the intention. They never did anything that would reduce revenue.

Instead, it was more a case of regulation to avoid looking like their services were financing illegal or illicit content.

So....

  ffmpeg -i porn.mp4 -vf "crop=crop_w:crop_h:coord_x:coord_y" "definitely not porn.mp4"
  *faceswap "definitely not porn.mp4*
  ffmpeg -i porn.mp4 -i "swapped definitely not porn.mp4" -filter_complex "[0][1]overlay=coord_x:coord_y" -c:a copy "deepfake porn.mp4"

Got it

    # process image to videos
    if modules.globals.nsfw == False:
        from modules.predicter import predict_video
        if predict_video(modules.globals.target_path):
            destroy()

I'm a big fan of explicit checks like this.

Did they ever work?

For the user, yes, you can choose what the software does on your computer.

Of all the things you can build with AI, why this?

That can be asked of 90% of what's come out of the latest AI bubble so far.

Like a lot of technology, AI has so much potential for good. And we use it for things like games that simulate killing one another, or making fake news web sites, or pushing people to riot over lies, or making 12-year-olds addicted to apps, or eliminating the jobs of people who need those jobs the most, or, yes, pornography.

We can do better.

> AI has so much potential for good.

Like what?

I'm hoping that at some point the novelty and hype will die down so that the headline grabbing "send a follow up email" or "summarize call" will get out of the way so the more impressive things like detecting medical conditions months/years earlier than human doctors will be a much more visible. The things for making people lazy are a total waste to me.

My wife used ChatGPT and Adobe’s AI to design our wedding outfits so there’s that. Turned out great!

Congrats on your recent nupitals!

Thank you :)

The answer is that anyone working on deepfakes doean't care much about ethics or they wouldn't be doing it in the first place.

OTOH now that we know the technology is possible, would you prefer that only some actors had perhaps the ability to do that. or perhaps not and having the lingering doubt that anything you see could be deep fake but there could always be plausible deniability that it would be too hard to actually carry it out.

If the technology is actually made widely available that just reveals that the Pandora box was actually already open

The claim of "deepfake" will be much more difficult to disprove than "my account was hacked"

I think this is an oversimplification that undermines your goals.

If you're unwilling to recognize the benefits of something, it becomes easier to dismiss your argument. Instead, the truth is balancing trade-offs and benefits. Certainly there is a clear and harmful downside to this tech. But there are benefits. It does save a lot of money for the entertainment industry when you need to edit or do retakes. The most famous example might be superman[0].

The issue is that when the downsides get easy to dismiss, it becomes easy to get lost in the upsides. It'll get worse because few people consider themselves unethical. We're all engineers and we all have fallen for this trap in some way or another. But we also need to remember that the road to hell isn't paved with malicious intent...

[0] https://www.youtube.com/watch?v=2nxanN85O84

> But there are benefits. It does save a lot of money for the entertainment industry when you need to edit or do retakes.

I think the downside is 10 orders of magnitude larger than this benefit.

I also think there are more people who'd call this usage a downside than a benefit.

> I think the downside is 10 orders of magnitude larger than this benefit.

I actually agree that the downsides outweigh the upsides.

The intent of my comment is not to defend this work, it is actually more about how to better construct arguments against it. That is why I do not begin with "you're [tdeck] wrong" but specify that the argument undermines the goals.

The point is who your speech is targeted at. If your audience is people who already agree that the downsides outweigh the benefits, the argument is fine. But it also isn't that fruitful, is it. But if your argument is intended to persuade people to agree with you, who already do not, then I think the argument will only amplify such disagreement.

If we recognize that most people aren't intentionally malicious, then if we are to persuade them to be in agreement we must also understand what persuaded them to be in disagreement. It is easy to brush this off as "money" or "stupidity" but doing so won't help you construct an effective argument.

I also need to stress my point in that this construction is harmful to yourself! If we are quick to simplify and see how obvious something is through hindsight, it will make us ill equipped to prevent such mistakes beforehand. Because what's obvious post hoc is not a priori. So don't dig your own grave. Especially because the grave is dug slowly. It's far more effective to be able to recognize it when the grave is shallow and you can still climb out.

In this case, the road to hell seems to be paved with intent to... make it easier to goof around and make silly prank videos, I guess? A lot of deepfake projects seem to be aimed in that direction and while there's nothing wrong with that in itself, it's hardly a compelling use case that outweighs the obvious harms that everyone has been talking about for years now. That's why I say that if someone cared about those harms they wouldn't be making this. Of course there are always things we tell ourselves: "if I didn't make this someone else would", "by making this easier (faking videos of real people) I'm training the public to be more skeptical", etc... etc... At what point is it obvious that these are excuses and the person really doesn't give a damn?

So the truth here is that the reason they're doing this is because they aren't yet good enough to sell to Hollywood. Not to say that Hollywood isn't using deep learning[0], but there's typically a combination of classical tools and deep learning tools. But these companies all seem to have an aversion to traditional tools and appear to want to be deep learning all the way down. This is a weird tactic and the fact that people are funding such companies is baffling. I can't even imagine a future where you don't want traditional tools, even if ML could do 99%. Hell, even 100%. Language is pretty lossy and experts are still going to want to make fine grain edits.

[0] Disney Research is pretty cool: https://www.youtube.com/@DisneyResearchHub

The number one use case for this will be to beat KYC checks, which means KYC procedures will get more annoying and bothersome for everyone else!

I'm a researcher who's made one of the best face generators. I'd like to address your questions and discuss a larger more critical point.

I too have ethical concerns. There are upsides though. It is a powerful tool for image and video editing (for swapping, you still need a generator on the backbone)[0]. It is a powerful tool for compression and upsampling (your generative model __is__ a compression of (a subset of) human faces, so you don't need to transmit the same data across the wire). It is easy to focus on the upsides and see the benefits. It is easy to not spend as much time and creative thinking directed at malicious usages (you're not intending to use or develop something for malicious acts, right?!). But there's two ways to determine malicious usages of a technology: 2) you emulate the thinking of a malicious actor, contemplating how they would use your tool, and 2) time.

But I also do think application matters. I think this can get hairy when you get nuanced. Are all deepfakes that are done without consent of the person being impersonated unethical? I think at face (pun intended) value, this looks like an unambiguous no. But what about parody like Sassy Justice?[1]. Intent here is not to deceive, and the deep fakes add to the absurdity of the characters, and thus the messages. Satire and parody itself doesn't work unless mimicry exists[2]. Certainly these comedic avenues are critical tools in democracy, challenging authority, and challenging mass logic failures[3] (which often happens specifically due to oversimplification and not thinking about the details or abuse).

I want to make these points because I think things are post hoc far easier to dismiss than a priori. We're all argumentative nerds, and I think despite the fact that we constantly make this mistake, we can all recognize that cornering someone doesn't typically yield in surrender, but them fighting back harder (why you never win an argument on the internet, despite having all the facts and being correct). And since we're mostly builders (of something) here, we all need to take much more care. *The simpler you rationalize something to be post hoc, the more difficult it will be to identify a priori.*

Even at the time, I had reservations when building what I made. But one thing I've found exceptionally difficult in ML research is that it is hard to convince the community that data is data. The structure of data may be different and that may mean we need more nuance in certain areas than others (which is exciting, as that's more research!), but at the end of it, data is data. But we get trapped in our common datasets to evaluate[4] and more and more, our research needs to be indistinguishable from a product (or at least a MVP). If we can make progress by moving away from Lena, I think we can make progress by moving away from faces AND by being more nuanced.

I don't regret building what I built, but I do wish there was equal weighting to the part of my voice that speaks about nuance and care (it is specifically that voice that led to my successful outcomes too). The world is messy and chaotic. We (almost) all want to clean it up and make it better. But because of how far we've advanced, we need to recognize that doing good (or more good than harm) is becoming harder and harder. Because as you advance in any topic, the details matter more and more. We are biased towards simplicity and biased towards thinking we are doing only good[5], and we need to fight this part of ourselves. I think it is important to remember that a lie can be infinitely simple (most conspiracies are indistinguishable from "wizards did it"), but accuracy of a truth is bounded by complexity (and real truth, if such a thing exists, has extreme or infinite complexity).

With that said, one of my greatest fears of AI, and what I think presents the largest danger, is that we outsource our thinking to these machines (especially doing so before they can actually think[6]). That is outsourcing one of the key ingredients into what defines us as humans. In the same way here, I think it is easy to get lost in the upsides and benefits. To build with the greatest intentions! But above all, we cannot outsource our humanity.

Ethics is a challenging subject and it often doesn't help that we only get formal education through gen ed classes. But if you're in STEM, it is essential that you are also a philosopher, studying your meta topic. Don't need to publish there, but do think about. Even just over beers with your friends. Remember, it's not about being right -- such a thing doesn't exist --, it is about being less wrong[7]

[0] https://www.youtube.com/watch?v=2nxanN85O84

[1] https://www.youtube.com/@SassyJustice

[2] https://www.supremecourt.gov/DocketPDF/22/22-293/242292/2022...

[3] https://www.gutenberg.org/files/1080/1080-h/1080-h.htm

[4] I do think face data can be helpful when evaluating models as our brains are quite adept at recognizing faces and even small imperfections. But this should make it all that much clearer that evaluation is __very__ hard.

[5] I think it is better to frame tech (and science) like a coin. It has value. The good or evil question is based on how the coin is spent. Even more so how the same type of coins are predominantly spent. Both matter and the topic is coupled, but we also need to distinguish the variables.

[6] Please don't nerdsplain to me how GPTs "reason". I've read the papers you're about to reply with. I recognize that others disagree, but I am a researcher in this field and my view isn't even an uncommon one. I'm happy to discuss, but telling me I'm wrong will go nowhere.

[7] https://hermiene.net/essays-trans/relativity_of_wrong.html

This seems like the beginning of a treatise, a useful one, to help us come to grips with all this AI stuff.

Upvoted.

In a way it is. Practicing writing to HN but when I write similar things I often get no feedback. Positive nor negative. I'm worry verbosity is my issue but I don't know how to state so much briefly. Thanks for the reply, and honestly my goal is to start conversations.

Fair. YW.

Being a deep thinker, which I assume you are, leads to those results, in my view.

It sounds like you don't want to just "do the punchline" either, that you want to lay it out and moreover you want to share kind of the experience of unrolling your thoughts, so it really makes sense that you're not just sharing the punch line.

I mean that's my interpretation and you know I haven't really met you so take what I'm saying with a mountainous size grain of salt.

  > Being a deep thinker

I think I'm just unsatisfied with "because" lol. Realistically I think I'm just asking "why" a few more times.

  > you don't want to just "do the punchline"

I think the punchline is not only non-obvious, and actually counterintuitive or easy to disagree with (if it was obvious, we wouldn't have the issues, right?). So I think just stating it would likely be ineffective. So then it becomes about how show the logic. Which if disagreed upon the steps or assumptions, I'm more than happy to update.

  >  I haven't really met you

Actually that's why I find this particularly helpful. Different kind of bias. I do not find these conversations difficult in person, but it is harder in comments. Maybe just blog form is best for online.

And thanks for the comments. They do give me a bit to think about.

Well, I understand how it works, and I still find it freaking amazing. The quality is... impressive.

On the flip side, the ability to deep-fake a face in real time on a video call is now accessible to pretty much every script kiddie out there.

In other words, you can no longer trust what your eyes see on video calls.

We live in interesting times.

It's interesting, because the subconscious ability of the mind to identify discrepancies is incredible (even if we ignore that feeling we get about something).

The feel of counterfeit bills, the color someone choose to wear, the sound that doesn't quite fit.

I think deep-fakes are mostly a danger to people without a lot of source material for their minds to compare against. You could trick me into believing I was taking with Elon, but not my son.

You could trick me into believing I was taking with Elon, but not my son.

And yet there have been several recent studies that show the younger someone is, the more likely they are to be scammed online.

> In 2021, Gen Xers, Millennials, and Gen Z young adults (ages 18-59) were 34% more likely than older adults (ages 60 and over) to report losing money to fraud,[1] and some types of fraud stood out. Younger adults reported losses to online shopping fraud – which often started with an ad on social media – far more often than any other fraud type, and most said they simply did not get the items they ordered.[2] Younger adults were over four times more likely than older adults to report a loss on an investment scam.[3] Most of these were bogus cryptocurrency investment opportunities.

https://www.ftc.gov/news-events/data-visualizations/data-spo...

Older people are less likely to invest in high risk, high yield.

Older people are less likely to have their entire personas and private lives fully documented on social media.

Younger people really should consider this point.

Personally, I don't use streaming video outside of work and there are no videos of me on youtube or any social media to train a model on even if someone wanted to.

My mother in her 70s doesn't even have a debit card. She thinks the idea is ridiculous and insecure. She writes paper checks and that is it. To put her account number on an electronic device would be completely unthinkable.

While the average older person might be more easily confused by social engineering the attack surface for an electronic scam is so tiny compared to the average younger person.

Only one photo is needed. I've not looked deep into this specific project, but speaking as an early developer of likeness transfer trained algorithms, only one image is needed, and it can even be a crude sketch - but if one's true likeness is captured by an image it can be recreated in full 3D. The catch is an individual's specific facial expressions, such as the real individual has a slightly lopsided smile, or they have smile dimples, or simply their characteristic at rest facial positions are absent, so they don't look like themselves to those that know them.

She writes paper checks and that is it.

As my accountant says: "For every person in your neighborhood committing check fraud, there are ten-thousand people around the world trying to steal your money online."

> She writes paper checks and that is it. To put her account number on an electronic device would be completely unthinkable.

But she's handing a plaintext copy of her account number to everyone she pays with a check..

> And yet there have been several recent studies that show the younger someone is, the more likely they are to be scammed online.

I think you are misreading the post. Pretty sure they meant

you could trick me into believing I was talking with Elon, but you could not trick me into believing I was talking with my son

To which I agree personally, though I don't know how universal this is.

i fell victim to such scam this year (first time i got scammed over 40 years). Key factor was that i got link to scam shop not from social ad but from my wife :) she got it from insta ad. So basically my wife scammed me :)

What if that source material for young brains gets more and more contaminated with artificial junk?

The key take-away, for me, is that I should "keep my guard up" on any video call about money or other important matters, even if other participants on the call are colleagues, friends, or relatives. There are no guarantees of authenticity anymore. My new motto for video calls is "trust by verify."

*"trust but verify"

There’s interesting ambiguity in this comment. I interpret the comment as saying, “I could be tricked by a deepfake of a stranger due to a lack of experience with their ‘true’ behaviors, but would not be tricked so easily when it’s someone I know well.”

Others here seem to be interpreting the statement as, “I could be tricked because I am an older person, while a younger person would not be so easily deceived.”

> You could trick me into believing I was taking with Elon, but not my son.

This was my thought about people in general, until more and more stories came out about the phone scammers pretending to be a grandson/daughter/family-member in need of a wire transfer/money to get them out of trouble. I still find it difficult to believe those are real scams that seem to work. This will probably escalate those even more with more people going to video calls. The panic of a loved one/child will not create a calm enough mind for thinking "hey maybe this is a deepfake" in most parents, atleast from my observations.

> The feel of counterfeit bills, the color someone choose to wear, the sound that doesn't quite fit.

You think there are no counterfeit bills that feel exactly the same as the real thing? Pretty sure you're wrong.

Indeed. Just recently a company got fooled into hiring a remote employee who turned out a North Korean hacker:

https://arstechnica.com/tech-policy/2024/07/us-security-firm...

Post-truthism has reached live video and is accessible to everyone. Turns out it's still only a few weirdos who love to use it for grifting purposes. I think most normal people are like "what do I need this crazy sh*t for?"

It's got a lot of uncanny valley going on. Zuck looks like a corpse. I'm sure it could fool some people, but I'm not terrified yet.

I recommend setting up code words with people. I haven't gone that far myself yet, but in my mind, there are clear phrases I could say to the people in my life that they would be convinced it was me. Unfortunately, until I have the specific talk with them, I suppose anybody could impersonate me to them.

Looks like this project is a fork of the discontinued roop[0] with primarily some UI improvements. One of roop's main developers has been working on facefusion[1] for the past year and it produces by far the most convincing results from the ones i've seen and it also supports real time webcam face swapping.

0: https://github.com/s0md3v/roop/

1: https://github.com/facefusion/facefusion

Interesting... This project is built upon "GFPGAN v1.4" (https://github.com/TencentARC/GFPGAN) and "FaceSwap Extension - Automatic 1111 - Proof of Concept" (https://github.com/revolverocelot1/-webui-faceswap-unlocked). The GFPGAN project is grounded on its own in the paper "GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior" by Wang et al. (https://xinntao.github.io/projects/gfpgan)

This is not a new face swapping technique, it’s a wrapper around inswapper (aka InstantID, an IP adapter): https://github.com/haofanwang/inswapper

Relevant source https://github.com/hacksider/Deep-Live-Cam/blob/main/modules...

Technically impressive but I fail to see a good use case for it that's not related to propaganda or scam and the website doesn't seem to list one either.

If I had to dig way, way down to the bottom of the barrel for use cases, it would be very funny if everyone showed up to a meeting wearing one of the attendee's faces.

That would have been the dream of students during remote schooling times.

Don't they already?

One legitimate one I could imagine is if people want to pursue a career in the adult film industry but without having to reveal their true face (not with a celebrity face of course)

I have been thinking about this comment and if it isn't a celebrity then just someone you know?

If not someone you know then just a random stranger?

Not that I can think of a better use case but it is telling if this is the best we can do.

I just meant a synthetic human (a dalle/stylegan/stable diffusion face output).

Job interviews, bank loans, anywhere racism or discrimination exists (or might exist.)

There's some sort of filter on Instagram (or maybe it is some deepfake tool) that replaces girls' eyes with a set of nice eyes, but it seems the tool only has that pair of eyes, so all the videos of girls with these eyes are so noticable. And so many "influencers" have this pair of eyes, it's depressing.

It's even more amusing when one sees glitches like eyes appearing in front of a strand of hair...

Only things I can think of:

- streamer goofing around.

- Perhaps something like this could be used to map your facial expressions onto video game characters in real-time.

- could take tictok style social media to the next level of absurdity. make me into a meme. Ghana says goodbye etc.

It's fun for goofing around. Imagine a conference call with your buddies and each one comes with a different deepfake. Kind of like a costume party but on camera.

I don't think that compensates all the bad uses it will probably have.

Oh those without the imagination: this is gold marketing for makeup and fashion advertising companies. The "good use" is the multi-billion dollar makeup and fashion industry. People will submit their own images so they can see themselves randomly appear in their own media feeds in the latest fashions. This is a no brainer for those with the connections to fashion marketing.

Just a few off the top of my head:

Movies and TV:

  - As an alternative to motion capture for animation
  - As an alternative to existing de-aging CGI when you want to flash back to a younger version of a character (especially for cases where newer sequels are being made for much older movies)
  - As an easy way to get some additional footage if an actor no longer looks the part

In a professional setting:

  - Conduct job interviews were interviewees faces are mapped to the faces of a few pre-defined images, to reduce a major source of implicit bias in interviewing
  - Get some footage of yourself when you're looking your best, with great lighting, and use that rather than being off-camera if you're joining a meeting when you don't look great
  - Create virtual spokespeople to represent your company in marketing, and allow that person to be played by different actors

News and Politics:

  - An alternative to blurred or blocked out faces for people giving interviews or whistle blowing
  - Allow people to testify in court (virtually) without revealing their identity and risking retaliation

None of these uses - most of which benefit a slim percentage of society or are needlessly complicated by this technology - outweigh the severe downsides to society. It's the apex of foolishness to act glibly about this.

This all ends extremely badly.

The counterpoint is that some or all of these could make money and not enough people care how it ends if money is being made. I suspect it will have to take a terrorist plot using generative AI or something similarly significant to shut the door and even then it will be disallowed by us commoners, not the big four or five AI companies and not to the rest of the world.

Whether the upsides outweigh the downsides or not is a different discussion. My point is that there are plenty of ways someone might use this technology. If you do think that this technology is a net negative to society and should be controlled or prohibited, then it's still important to understand the potential ways someone might want to apply it so that you can be prepared to make your argument.

Personally, I have mixed feelings. I think that most of the outcomes we're most concerned about are going to happen one way or another, and developing in public or even commoditization of access to it is going to be a net reduction in harm over locking it up and pretending it doesn't exist while allowing people (and nations) with the resources to run large models in secret to develop and use the technology against people who are ignorant of what's possible.

Most people don't need to know how to code or have access to hacking tools. Their are such limited use cases for such tools and a great deal of harm can be caused the abuse of them.

All the replies to this question read like some sort of corporate marketing robot coming up with ideas.

In the overwhelmingly large majority of cases it'll be used for porn, scams and maybe 5% of the time it'll crop up in a meme of Donald Trump and Joe Biden singing memey Chinese songs.

Can't wait to hear all the stories of grandma losing her life savings cause a scammer can use Timmy's hyper realistic deep faked face at a click. AI truly is the future

Just need to add voice augmentation and every grandma and grandpa in the world will have their bank accounts cleaned out! Better go warn mine now! XD

Just need to add voice augmentation and every grandma and grandpa in the world will have their bank accounts cleaned out!

Only if by "grandma" you mean "Millennial" and by "grandpa" you mean "Gen Z." Your ageism doesn't jibe with reality:

https://www.ftc.gov/news-events/data-visualizations/data-spo...

I think you should read that again. It's clear that different age groups fall for different scams and have different impacts from them.

Grandparents absolutely fall for some scams at disproportionate rates. And are less likely to be able to recover. (A 19 year old who loses everything has many more productive years to recover than a 72 year old.)

Also, humorously, millennials are starting to become grandma and grandpa. Elder millennials are in their mid 40s. It's young, but not impossible for them to be grandparents now.

In the world? You'd be surprised by how many grandmas out there that don't have bank accounts or access to the Internet...

A practical use of this would be to animate your face onto a CGI model which was independently posed for the purposes of video meetings - which is something I've always wanted.

Let me separate my face, body and words and craft the experience.

Face tracking has existed for years now. I frankly don't see what's different between what you described and FaceRig.

You're missing the point: I want a fake version of myself.

I want a model which is made photoreal with my own image, so it can be given a voice in real time with my words, but a filtered version of my facial expression and pose.

So how I look and act is essentially scriptable.

I think that’s already happening. You can buy a trained model of someone to do 24/7 live-streaming peddling products, or even black-mirror-esque bringing back deceased ones. Company in china selling this is Silicon Intelligence.

https://www.technologyreview.com/2023/09/19/1079832/chinese-...

https://www.technologyreview.com/2024/05/07/1092116/deepfake...

Strit up look into vtubing, and it's already done. https://inochi2d.com/

Is there a legitimate usecase for this?

Like when they were brainstorming this as a product, what was the persona/vertical they were targeting?

This makes me think there could be use for a very very (very) easy-to-use tool that allows two parties to choose and store a secret (verbally pronounceable) passphrase known only to the two of them, for use in situations in which it might be necessary to ‘sign’ a video chat or audio conversation in which one party’s identity might be in doubt.

a very very (very) easy-to-use tool that allows two parties to choose and store a secret (verbally pronounceable) passphrase known only to the two of them

So, quite literally, a "password" in its original pre-internet meaning.

Haha yes. Maybe a secret knock?

I always liked that idea (not original, I know) in the Harry Potter books (Harry Potter and the Deathly Hallows, more specifically), where two people ask a private question they should only know to be sure they are not being impersonated.

I've seen people try and share such a "password" verbally on a video call. With recording and live transcribing on. From free-tier extensions with wobbly privacy policies.

This won't work.

I've resorted to using OTP apps with family and coworkers.

So you force your parents/kids/aunts/etc to give you a one time code every time they want to talk to you?

That seems extremely clunky and impersonal and I couldn’t imagine anyone in my family willingly agreeing to do it.

I mean, not every time, but if someone is asking for money because they're in the hospital/jail/something, it seems prudent to have some sort of system set up these days.

you can't plan on everybody never having something come up.

You're missing the point.

If somebody deepfakes someone in your network, it will most likely be for scamming / monetary purposes. (I don't care much about being pranked)

So you don't need to verify identity for a casual chat with a friend or family member.

On the other hand, here is a real situation that happened 3 days ago.

I get an instant message from my COO Nirina: the bank website says she needs a new auth code to be able to enter new wire payments for me to sign and pay the team. She provides me with the link the bank has referred her to ; a URL the bank says I need to access (with my higher privileges) to generate a new auth code for her.

First of all,

- Whoever the source is, there is no way I'm clicking that link. If I'm going to do anything about bank access codes, I'll login by manually typing the URL and I'll navigate the website myself.

Second,

- I would trust Nirina with my life. But should I trust that the person sending me that link is Nirina? I mean, if you got hold of her laptop on a business day, it wouldn't take you more than a couple minutes of snooping around to figure out that she's messaging back and forth with the company's CEO and hereby lies opportunity.

So how do I check it's her?

- I could call her (and fall for a deepfake)

- I could ask for something "only she would know" (and risk the chance that the answer can be guessed, or can be found somewhere in our several-years-long messaging history or on our 10-year-old online Drive)

- or... I could say "what's the code that proves this is from Nirina?" and wait for her to send me an OTP code matching mine.

Obviously I am not doing that every time someone from my team sends me a message.

Oh and by the way, I put this is place the day after someone tried to get access to our Meta Business account by impersonating someone on a Zoom call with me... it's not paranoia if they're after you!

"Built-in checks prevent processing of inappropriate content, ensuring legal and ethical use."

A software engineer says to himself, if only I could keep these guns from jumping off the table and shooting people.

Facinating software although I hope the idea "we're gonna rely on people to be good humans and DO THE RIGHT THING" is quickly abandoned and instead there is just as robust development of detection software that goes along with newer and better deep fake tools.

Detection is ultimately impossible. Anything you can detect can be explicitly evaded.

I dream of a world where a web of trust signatures are taken seriously. A few hops should get you to a real human holding the camera who claims it's a real recording. If that person or someone along the way is regularly flagged as malicious by others that you trust, you can blacklist them.

I think the terminal solution to this, in the US and maybe the EU, will be putting identifying code/chips into all devices capable of connecting to the Internet that will tag all content (video, text, audio, images) in some way where browsers will have to legally change to interpret them. This will make everyone either unable to use the Internet or known to anyone that "needs to know".

Huh. Thought politics was dead with VASA-1 [1], EMO [2], and Animate / Outfit anyone [3], so we could clothe people in anything, animate them any way we want, put them anywhere, and have them say anything to the public. "Thoughts and prayers victims..."

However, this really nails that pretty dead itself. Wonder if I can:

- Sit at home in pajamas.

- Change my face to Sec. of Def. Lloyd Austin.

- Put myself in a nice suit from TV

- Call the White House with autotune voice pretending to be going in for surgery yet again because of life threatening complications

- Send the entire military into conniptions (maybe mention some dangerous news I need to warn them about before the emergency rush surgery starts)

Edit: This [4] might be an Animate / Outfit anyone image... It's difficult to tell. Even with huge amounts of experience, the quality has become too elevated, too quick to check 1000's of depressing murder images for fakes because it might be a BS heart string story. All stories on the WWW are now, "that might be fake, unless I can personally check." Al-arabiya upvoted casinos and lotteries for muslims recently. [5] "they all might be fake."

[1] https://www.microsoft.com/en-us/research/project/vasa-1/

[2] https://humanaigc.github.io/emote-portrait-alive/

[3] https://humanaigc.github.io/animate-anyone/

[4] https://www.reuters.com/resizer/v2/https%3A%2F%2Fcloudfront-...

[5] https://english.alarabiya.net/News/gulf/2024/07/29/uae-grant...

It's really getting to the point where multimedia online shouldn't be trusted unless it's from a reputable source and cross verified.

I wonder, is there a universe where maybe cameras are updated to add some sort of digital signature to videos/photos to indicate they are real and haven't been tampered with? Is that feasible/possible? I'm not skilled with cryptography stuff to know, but if we can digital sign documents with some amount of faith...

I've heard folks mention trying to tag AI photos/videos, but it seems like tagging non-AI photos/videos is more feasible?

The idea of a signed and verified stream has only been used to enforce old-school distribution rights. Because of this, the implementations are clunky and have zero incentive for consumer adoption. Why buy your own handcuffs?

The incentive is to prove that the video is not AI generated. Useful for consumers, news organisations, camera manufacturers, etc. The idea would be you can still copy the file/change the video, but the signature will no longer be valid. It's not mean to be restrictive like handcuffs/DRM.

I feel like this is one step closer to the Black Mirror episode where the grieving widow orders an AI version of her late husband.

And I don’t say this with excitement.

This is getting scary to be honest

And this is the worst quality it will ever be. In the future it will be impossible to know who we are talking with online.

I think AI + deepfakes will increase the value pressure on in-person interactions - ie, the only time when you can (for now) believe your eyes and ears.

I wonder how politics can be transacted in such an environment. Old-timey first-past-the-post might be the optimal solution if you can't trust anything from out of earshot.

> I wonder how politics can be transacted in such an environment.

Codes and seals predate computers (by quite a bit.)

https://en.wikipedia.org/wiki/Code_(cryptography)

https://en.wikipedia.org/wiki/Seal_(emblem)

Those are relevant to the workings of government.

The process of politicians debating and getting elected is going to have to be much more local. Just look at how easy it is to spread misinformation now.

My hope is that we (as a global society) re-learn to value honor and honesty.

> The process of politicians debating and getting elected is going to have to be much more local.

I'm no expert on government but that seems like it would be a good thing. IMO the best but most expensive form of government is Quaker-style Consensus Decision Making:

https://en.wikipedia.org/wiki/Quaker_decision-making

https://en.wikipedia.org/wiki/Consensus_decision-making

In person interactions like this CIA mask expert tricking a US president? https://spyscape.com/article/master-of-disguise-how-the-cias...

This is a nice article, but I don't think it works as a counter argument to GP. Deep fake shenanigans are way more scalable and thus more likely to affect average people than these elite spy techniques.

How long until we see "anti-cheat"-like software to try to detect this stuff for video chatting?

I'm guessing that finding a technology to try to detect this would be over-engineering. I'd love to see a sample where the person with the swapped face passes their hand with spread fingers over their face, and see how it handles that.

I've tried it, it currently does not handle that scenario well at all.

Wait three months, it'll be fixed.

Perhaps we’ll see it a requirement to use a closed platform like an iPhone where it would be much easier to attest that the feed is not tampered with.

It’s already a requirement sometimes to take a video of your face from multiple angles using your phone - some identity verification service forced me to do it. I imagine that stuff like this will evolve to check for hardware attestations more, or use info from depth/lidar sensors to verify video and other sensor data align.

I am not at all comfortable with this. Even though it is an amazing demo I feel the release of this tool could not have been more ill-timed. It has all the potential of wrecking US elections this year. I don't really know what the optimal release time should have been but I don't see how this can be used for good at all. And I am still just considering implications of this restricted to elections. I am not even going to think about what it would mean to child porn, terrorism and even entire assassinations orchestrated to destabilize a Government leading to Civil/World Wars. Lots of things can go wrong with this tech.

Another huge issue is fraud. People impersonating others for financial fraud, etc.

Question is, is it stoppable? Doubt anyone thinks it can be stopped unless you get into fascistic/communistic/authoritarian tactics of arresting people for just using it at all.

https://github.com/C0untFloyd/roop-unleashed does this for free on your own computer

So does this, the landing page is a wrapper for https://github.com/hacksider/Deep-Live-Cam

I can see this changing how people interact over video calls, we might start having to exchange a "key phrase" whenever we join a call, so that both sides can make sure the person on the other side is who they say they are (or appear to be), even within families, perhaps not yet but soon enough. Crazy times.

What’s the technical difference in how this works vs. previous face swapping tech (like Snapchat filters)?

That will be funny once those neural nets can be infered on a normal person beefy workstation (or a few of them). Just thei "ethical" goes down the drain.

It is already ez to run text troll AIs on normal workstations... so...

where are all these "wow impressive" comments coming from? clicking "Get Started" dumps you into a loop of landing on the home page

the README.md seems straightforward enough.

Where is the repo? Stuck in the landing page loop here and no github link I could find.

Yeah all of the page's links just go to the same page, except the "experience live cam" link at the top. That goes to this:

https://github.com/hacksider/Deep-Live-Cam

Took me multiple minutes to find too.

Well, this will make for some "interesting" viral campaign fodder.

Next 3 months is going to get interesting.

The future of v-tubers is here

Neil DeGrasse Tyson says it best the Internet will die because of this type of technology https://www.youtube.com/shorts/AxM2XTwaaUA

AI will kill the Internet we know today and the new one im guessing you will have to have a Internet license attached to your identity which is backed by your internet reputation which you always want to keep it high for veracity/validity! You can still post anonymously but it wont hold as much weight compared to you posting using your verified Internet identity. This idea of mine i posted good number of times here and it gets downvoted but with the IRS in bed with ID.Me (elon musk is involved with them in some capacity) you can see what i mention with ID.me and the IRS being a small step in this direction. Otherwise no one uses the Internet (zero trust of it) .. it dies and we go back to reading books and meeting in person (doesnt sound all that bad yet ive never read a book before).

Now the question is, what is the use case that does not entail misinformation or personal amusement?

Cam girls will get some competition from guys.

Technologically seems cool, and the first use that pops into mind is "wouldn't this be funny to prank my friend?"

But maybe no, it wouldn't. Maybe it'd be deeply disconcerting. We have very strong norms around honesty as a society, and maybe crossing them in video just for a joke is comparably crass to giving somebody a fake winning lottery ticket.

I miss the recent past where new tech felt exciting and inspiring but for the last few years new developments are often coupled with an anxiety for the new harms possible and often unclear benefits. I wonder how much is 'inevitable', at large enough scale we will always exploring new possibility spaces as they become available and how much is our choosing – we put resources to build these things in full awareness because we think they bring value over focusing on other things. I realise though it is useful for society to develop understanding and defences for these things early

I've notice I've steadily become more ashamed to be associated with tech. I'm still processing how to react to this and what to choose to work on in response

Am I in a bubble? Do you share similar feelings or are yours quite different? I am very curious

> I've notice I've steadily become more ashamed to be associated with tech

Are you actively contributing to these areas you feel ashamed about? If not, you shouldn't really feel ashamed about what other people chose to work on, even if both of you work "in tech".

I'm sure not all people working on medical research agrees with what all other researchers are working on, but you cannot really control what others are working on, so why feel ashamed over what others are working on?

you shouldn't really feel ashamed about what other people chose to work on, even if both of you work "in tech".

Why not? Someone who builds boxes that hold bombs can be ashamed of being in the munitions industry, even if they don't make the actual bombs.

Right, but if you're in general "manufacturing", there isn't much point of feeling ashamed about some parts of the industry focusing on munitions manufacturing.

I feel the same way. I can't think of a single legitimate use case for this. I wish all those GPU teraflops were being used for something else.

Definitely with you. It used to be a higher entry point so a certain passion was necessary. And it was less about money and more about sharing info and joy. Now networking tech has been "democratized", it's another medium where the usual human pain and greed play out. High school again, but with real consequences on peoples' lives.

Deep Live Cam is a cutting-edge AI tool that enables real-time face replacement in videos or images using just a single photo. Perfect for video production, animation, and more.

I don't want to see faked videos.

Son, you no longer have an option on our free service. But you can subscribe to our deep fake-free service for only $89.99/month.

You won't know the difference.

That only makes it worse.

resistance is futile. profits go brrrr.

Maybe you have been this whole time...

Reply on news.ycombinator.com