Hacker News Reader

Comment by notTooFarGone | Hacker News Reader

Anyone believing AI will be a nice thing in the future: With data from social media it's a fucking nightmare - billions of people having their relationships visible to everyone and a singular voice recording/video is enough to scam your friends, family, workplace.

I don't see a future of the current communication channels without strong authentication measures - people already get scammed with shitty tactics by billions of dollars per year and now you can automate it.

View on HN

I doubt a single voice recording could generate speech with enough emotional depth that I'd be fooled by a fake call from my better half. Do people really have such a shallow connections to their loved ones that they wouldn't instinctively feel that something isn't right? C'mon, we humans operate with a lot of extra clues. Its not like we just STT and "read" the resulting output.

It doesn't have to work 100% of the time to be a successful scam.

You're right to be suspicious but wrong in your interpretation of reality. The issue is that there's a big difference in how you'll interpret things in different environments and stress levels. Under pressure a lot of your normal checks will go out the window. This is why conmen often put you under time pressures, because they're stressing your system. Under normal circumstances, you're prefectly right, but under duress weird things seem normal. I mean duress is abnormal to begin with.

Also enshitification is an important aspect here. We readily overlook things like sharp cuts in video, degraded signals in video and voice, distortion, etc. Because that stuff happens normally. Post hoc it is often easy to see scams and think how dumb someone must have been to fall for something so obvious, but that's actually a tool of the scammers themselves. They love that because the more embarrassed you are, the less likely you will report it due to shame. Be careful with over confidence, because that's what they target.

As someone who works in IT for roughly 30 years, and who never fell for any kind of phishing, I beg to differ. I think my interpretation of reality, as you put it, isn't as bad as you might believe. Also, all the phishing-victims I know have something in common, a certain lack of rigour. You seem to claim that everyone can fall for phishing. I beg to differ, its a certain class of not very thorough (l)users.

Your comment regarding enshitifcation has some truth to it I didn't really think about yet, thanks.

However, my original comment still stands as it is. I challenge anyone to create a voice scam that fools me.

> that I'd be fooled by a fake call

Can you say the same thing about all of your friends and loved ones?

You don't have to personally fall for the scam for it to negatively impact you. Maybe your mom falls for it, and now she can't pay her mortgage.

Fake call scams already generate billions of dollars in stolen money. Telecoms let spam phone calls run rampant for a decade before being forced to fix it. AI video/audio deepfakes at scale could 10x this industry.

There was an interesting paper about a year ago that talked about synthesizing voices with minimal training data, iirc on the order of 30 seconds. The results were imperfect but wholly impressive given the input.

"Think of what will be possible just two more papers down the line."

From experience, it is the people who are confident they won't be tricked that are most vulnerable to being tricked.

The spouse of a former boss of mine regularly mistook me for him, if I happened to answer his phone while he was at lunch or in the toilet or something (this was long before people had personal cell phones).

Once she even said "Hi! This is your little snuggle bunny!". Mutual embarrassment ensued.

Reply on news.ycombinator.com