Rising scams use AI to mimic voices of loved ones in financial distress

arstechnica.com

46 points·lame-robot-hoax··4 comments

About two weeks ago my father (76 y/o, technologically-savvy) nearly got burned by this scam, with the "caller" impersonating my nephew who is currently residing in Holland.

The "caller", crying, indicated that he had been in a drunken-driving accident the night before, was currently in jail, other party (a PREGNANT WOMAN) in critical condition, and in need of $9000 cash in short order. Oh yeah, the caller also indicated "whatever you do, don't tell my mom!"

A call to "mom" with a subsequent text message to my actual nephew showed the entire situation to be fabricated.

Horrifying. My grandfather fell for this years ago, no AI required. I hate to imagine what a skilled con with a competent techical consultant could do these days.

In no time at all live video will be inadequate to verify someone's identify. It won't be enough to verify that you're talking to a human being at all.

TTS that fed into an LLM (brownie points: one trained on fraud artist conman techniques) with replies fed into face video + voice generation. Open Zoom and Facetime and start making calls.

Not AI, but human voice impersonators have even got away with murder when people have suicided themselves. It also shows that telecoms companies, news organisations and legislators have got away with murder under the UK's joint enterprise.

https://en.wikipedia.org/wiki/Suicide_of_Jacintha_Saldanha

Reply on news.ycombinator.com