Hacker News Reader

Comment by d2049 | Hacker News Reader

d2049

Is anyone else thinking that it might not be a good idea to give away your voice and face to a startup that is making digital clones of people?

View on HN

People were prone to install the OpenAI app and use the voice assistant, forgetting that the recorded voice can be used to create fake audios (see Scarlett Johansson).

Same for Google Assistant, Siri & co.

So basically I don't see why people should be concerned only for the usage by a small startup, instead of being scared by tech giants

I've been trying to clone a public figure's voice for a meme, it seems that the major offerings in this area don't let you do that because they're trying to be respectable. (I don't think there are laws about this yet, but there will be soon.) So I've had to experiment with smaller, less "serious" services.

I assume a similar logic applies here.

Sure but to me this sounds paranoid and as pointless as the movie industry trying to create non piratable technology... As in, worried about things out of their control. You cannot go about your life without using your voice unless you're a mute by choice or physically, and all a company needs is a few seconds of your voice to recreate it. If a company is hell bent on getting a voice, they can get it. If you're not widely known, or hold some kind of power, no one likely cares about your voice, and if you are, its likely there's already lots of audio sources of you out there... Even if you're not widely known, if you've ever made an instagram post, a reel, a tiktok, vine, youtube vid, etc, you're out there. Probably makes more sense to go on about your life and resort to legal means if your voice is used without your consent.

Same with your face... You leave your home, other humans see your face, cameras see your face. You do not get to control who sees your face or even who captures your face when you're in public, but you can decide whether or not you consent to your face being used by an entity for profit.

We make the distinction between humans consuming information and machines because humans can't typically reproduce the original material. So like, you can go see a movie, but you can't record it with a device which would allow you to reproduce it. But what if human brains could reproduce it? Then what? Then humans could replay it to themselves all they want, and to those near them, but wouldn't be allowed to reproduce it in mass for profit, or they'd get sued. I think the same stuff applies to data ingested by AI models. People care so much about what is fed in when the same information is fed in to humans around the world which increases their knowledge and informs their future decisions, their art, their thoughts. Humans don't have to pay to see a picture of the Mona Lisa, or pictures or any other art out there, even if it'll influence their own art later on. But somehow we want to limit what is fed to models based on it having gotten the permission to be influenced by its existence. I agree, we can't feed protected IP, or secret recipes, formulas for things that are not in the public sphere.. etc.. But other than that, not sure how people expect to limit what is fed into it that it can draw inspiration from.. As long as it doesn't copy verbatim... I get that images have been generated where original material has come out, but if its sections of, or concepts of, then its the same as a human being influenced by it, I honestly don't think that matters.

Then comes the idea that this is owned by a private company who's profiting from it all... Thats true... But there's also open source models that compete with them. Not sure what the best answers to it all is.. But to go back to the original point, if your unique voice, or image isn't copied precisely for profit, then whatever... It'll get used by models, or humans in their thoughts, you can't control what your existence affects in the world, just who gets to profit off of it.

They clearly explain how you retain ownership of your own data and they allow you to monetize the data for your own behalf where they get a sub fraction percent on top of if they sell or use your data internally or externally you get a set value or scalable metric corresponding to usage?

Right?

We might need to do a better job of explaining this- but this is true. You retain ownership of your data and we don't sell or use it (other than to train your specific clone that you can delete at any time). Personally, I think too many AI companies are playing fast and loose in this space, so I get the concern. We want to do it right.

I don't know what you mean by ownership, do you mean you don't store everything you need to clone my likeness?

Unless this data is never stored server side or else is client side encrypted then you are putting a target on your back for hackers to extract this data for nefarious purposes no matter what your terms of service says

If your company is on the brink of collapse and you need funding to stay afloat, will your new majority shareholder be just as trustworthy? If you make if big and the difference between a million in revenue and a billion in revenue is misusing personal data, can you resist the temptation? Those are the concerns and you're not providing answers to them.

Like it or not, 23andMe is going down this path right now with millions of customer's genetic data and you're going to get the same scrutiny when you ask people for personal, intimate data.

Yea I start to load the chat and then was like wait a sec and noped out.

Same here. I was thinking maybe I'd give microphone permissions but didn't see why I had to show my video. Does the clone see my face? Maybe it does. That may creep me out more tho lol.

The AI looks at the video to get clues on what to talk about. I have books behind me, it asked about my books.

whatever

This is a valid concern, but we’ve always been very serious about consent and privacy. Our models cannot be used without explicit verbal/visual consent and you hold the keys to your clone.

No snark intended...if you're making it much easier to make clones of people verbally and visually, why would I feel confident in you accepting a verbal/visual consent from "me"?

> you hold the keys to your clone.

Can I run it on my computer?

If it doesn't run on my computer, what keys are you talking about? Cryptographic keys? It would be interesting to see an AI agent run on fully homomorphic encryption if the overhead weren't so huge - would stop cloud companies from having so many intimate, personal data of all sorts of people.

Does the end user optionally get like a big safetensors of their own digital twin?

No way I'm going to trust a small company/startup (move fast, break things) with this. Especially in the US.

> we’ve always been very serious about consent and privacy.

That's quite a commitment, guys, I am sold

I don't trust any of you AI people with that.

You think the rep from the AI doppelganger company is a people? Voight-Kampff may say otherwise.

Probably the phrase "you hold the keys to your clone" should give anyone pause.

I once worked at a company where the head of security gave a talk to every incoming technical staff member and the gist was, "You can't trust anyone who says they take privacy seriously. You must be paranoid at all times." When you've been around the block enough times, you realize they were right.

You can guarantee you won't be hacked? You can guarantee that if the company becomes massively successful, you won't start selling data to third parties ten years down the road?

And you promise to never get acquired right?

Reply on news.ycombinator.com