Meta says its new speech-generating AI tool is too dangerous to release | Hacker News Reader

Meta says its new speech-generating AI tool is too dangerous to release

techradar.com

29 pointshomelesscodes51 comments

So what, they set out to make a speech-generating AI tool and only after they finished it realized that they had made a speech-generating AI tool?

The engineer/maker types sometimes fail to consider second order effects of whatever they are building if the coolness factor is high enough.

I would guess that “too dangerous” is both hype and also means it utters insensitive statements. Not that it’s primed to topple human society.

This is the only thing that makes sense.

What do you mean it 'utters insensitive statements', this is not a chat bot, it's text to speech, it generates speech for exactly what you type in, the threat is that it can be trained to sound like exactly anyone and the potential for fake news etc is high.

The trouble is, this is already partly here and going to be increasingly available. For smaller startups that already do this like ElevenLabs, they have no reason to hold back, as it's there entire business, Meta can for now, but it's a fruitless exercise. These tools are coming.

Meta doesn't want the flack hurting or causing investigations into it's current business is more what it means, it will release the tool when others have released there's and it will no longer be the one to blame.

'Utters insensitive statements' is clearly the domain of humans, checking on my threads and I realise what an ass I sound like, I didn't mean to be so combative with it. Apologies.

The FBI warning about malicious communication using loved ones' voices is very damning.

the post love age

Yeah they don't want to be blamed for more deep fakes and election interference. This sort of tech is a strategic risk to meta.

More like the engineers and scientists built it, but legal and PR think it's a bad idea to release. Which they're probably right about considering the reception Galactica got.

Nice marketing piece.

It’s too dangerous to release until enough buzz is built about how dangerous it is. Then it will be released (or “leaked”).

Glad I'm not the only one who read it that way. It just smacks of "we are just way too good at what we do".

Not entirely sure why this was downvoted, it was my initial thought too (though I must concede I’m reading the comments before the article… which may give me my answer…)

Well they’re good at copying, so they’re doing exactly that with the OpenAI marketing strategy

Next version of "You WON'T BELIEVE how DANGEROUS this Meta voice AI model is".

I can't believe it's not butter! Sorry, ai.

I 100% agree.

This article is a good promotional article to get people hyped about this new product. I must admit, it has sparked my interest. I'm eager to use it. At last, I won't have to deal with inconsistent voice actors for my wizard demos / support pages.

Surely, they may have to incorporate some safety measures, much like OpenAI has.

It's demo wasn't very good compared with other state of the art tools available these days. You could hear so many glitches in audio.

Also OpenAI said the exact same thing about GPT but still released it. Meta's tool isn't worth hyping like that, not in its current state.

Its demo wasn't good even compared to the Adobe AI voice generation demo from 2016.

Does anyone know who the leader is in affordable speech synthesis for hobby projects ?

Atm I am using Amazon Polly but am unsure if another offering is far superior. eleven labs is not affordable for my slightly serious level of job ting but aws and google seem well priced. Any other suggestions for a product I can try out?

https://github.com/neonbjb/tortoise-tts

Give this a try. You can run it locally if you have a good enough GPU. It is pretty slow at generation tho

I've tried this and it took ~5 minutes to generate 10 seconds of studio on my 3080. That just doesn't work if you're trying to generate an hour long podcast.

That sounds way too slow, just go with lowest or the second lowest quality. I think even the lowest quality works just as well.

Also you can rent a GPU on vastai or runpod, go with 3090 which should cost like 0.29 or even 0.15 if you go with spot pods.

Also there's this which offers a speedup to normal tortoise: https://github.com/152334H/tortoise-tts-fast

Take a look at Piper. It's the tts solution used by the open source home automation project HomeAssistant. Produces decent quality speech in a couple seconds on raspberry pi class hardware.

I think this is the link. Not 100% sure though, is this correct?

https://github.com/rhasspy/piper

They said the same thing about GPT-3. It's a marketing push.

Oh no, our incredible scientists made a thing that's simply too amazing and they're afraid it might take over the world! We just thought you might like to know how nice and ethical we're being with our immense power...

What was the success criteria of the project? sounds like they did what they wanted and only after realised they didn't want to? why?

The goal of that kind of project in a huge tech org is to have capability in-house. So that they can play that game if they have to. Seems like they don't see themselves in a situation where their position without the tech would be worse than with the tech deployed. In some ways it's a bit like defense spending in a way, the most successful defensive army is the one strong enough to never get tested (in many other ways it's not)

Their scientists were so preoccupied with whether they could, they didn't stop to think if they should. - Dr. Ian "Goldblum" Malcolm

Adobe demoed a similar project as far back as 2016 called "Project Voco" [1] which was also called "too dangerous to release" at the time, even though it apparently still needed as much as 20 minutes of source material (vs. allegedly a mere 2 seconds here).

It was never heard from again afaik - even though Adobe is not known to shy away from an opportunity to increase revenue, so one cannot help but wonder...

[1] https://en.wikipedia.org/wiki/Adobe_Voco

What would happen if only bad actors had access to LLMs, deepfakes, and stable diffusion models? They could convincingly fake image/video evidence and accompanying online support for disinformation, and most people would buy it.

But today, given the widespread availability of these tools, most people know what they're capable of, and when a meme photo of Donald Trump in handcuffs gets circulated nobody takes it seriously because they have seen dozens of fakes before.

So IMO if you really want to minimize the potential for your AI tool create chaos, release it to the public. Show people what it's capable of. Once people tire of memes of Obama spouting the navy seal copypasta, they will be prepared to call out disinformation generated by these tools as well.

I believe everyone can understand this - and they will need to releases it to public. Surely, they may have to incorporate some safety measures, much like OpenAI has.

Anyway a fantastic promotional strategy to get people hyped about this new product. I must admit, it has sparked my interest. I'm eager to use it. At last, I won't have to deal with frustrating and inconsistent voice actors.

AI may be meta's only and real "grow out of newsfeed business" card. They should probably pivot, release their own version of ChatGPT/VoiceGPT, start charging for it and maybe rebrand (metai?). They are currently not being taken seriously, despite the chops they have, due to their stupid obsession with social and connecting people.

Perhaps by consumers but among the AI industry/NLP I would view them as the top 3 most sophisticated companies. They have also gained a ton of respect from me for not joining in with the "closed" approach other companies have all now adopted.

They have, but when OpenAI started spouting "too dangerous to release" was about the same time they effectively dropped the "Open" part. I hope that is not the same for Meta. But wasn't Meta not releasing model weights except for a leak? In that respect are Meta and OpenAI already equivalent?

OpenAI isnt even oublishing papers any more, and Meta has been publishing model weights for many models for quite some tome and have also commited to releasing their next language model open

apologies for typos, typing quickly on an ipad

Well they did that because they were in third, ie they had little to lose from 'commoditizing their competitors'

Bell labs thought something similar about the first answering machine, fearing people would be held to their word resulting in fewer telephone calls.

Now big tech worries that "perfect" trust in their communication platform goes away, but the paradigm shift happens no matter what. Many governments already have the power to post as users on platforms and that should be cause to usher in this new trust calculation faster.

Open AI used the same "too dangerous" marketing technique with GPT2 in 2019. https://www.theguardian.com/technology/2019/feb/14/elon-musk...

hmm, hasn't speech been spoofable before? Is there a major difference between this and video?

Perhaps this defeats common security by voice systems, something that even good APIs like Eleven Labs cannot (I assume).

If so, they really should disclose that. At this point, voice recognition is just a security time bomb.

Was it ever anything else? I thought voice authentication was only a movie trope, not something actually deployed in the real world.

My voice is my passport. Verify me.

"Too dangerous to release" just means "we don't think anyone else has something better."

Just wait, if their next version of LLaMA beats GPT4 it'll be "too dangerous" otherwise it'll be released as soon as they think it's peaked.

Can't fix facebook. Can't find a reason to get the whole world to go to work in VR. Can't release their apocolypse Robot.

What else can't they do?

Google also thought their AI was too dangerous to release....so chat-gpt released theres. See a pattern?

Whoever needs to use it at deep-state level for nefarious purposes will either access it or find something similar that does the same job.

I think it should be released so that the public can see what can be faked and they get used to the new paradigm of everything can be faked so they don't immediately believe in everything they see/hear.

TL;DR: bad guys will use similar tech anyway.

There was also a Google employee who said the same about Bard

lol, what BS.

We could release it, but then we'd have to kill you.

Reply on news.ycombinator.com