A similar problem is emerging for photos and videos. We also soon need cryptographically signed devices in order to be used in journalism or to be admissible in court.
Otherwise we are going back 150 years where we depend on in-person communication and eyewitness accounts.
Scammers aren't looking to defeat or even challenge voice identification. They're looking for that one person who's having a bad day and is susceptible to getting tricked. All they need is to find that person to earn their quota for the day. They'd actually appreciate it if 99% of the population used Voice Supr-Sure-Auth 3000™ technology, because that would make it more efficient for them to reach the 1% who don't.
This is why the Nigerian prince emails have typos. They're not trying to convince you their email is authentic. They're trying to find the person who isn't sophisticated enough to think in terms of email authenticity.
More broadly, I think this an instance of how AI/Deep Learning is turning over technologies (photos, video, voice communications) we have come to rely upon, and for us to continue to rely upon, they will need to be radically reworked with security as a starting point, not an afterthought.