Ex-athletic director arrested for framing principal with AI-generated voice
thebaltimorebanner.com
thebaltimorebanner.com
> Police say the clip was received by three teachers the night before it went viral. The first was Darien; a third said she received the email and then got a call from Darien and teacher Shaena Ravenell telling her to check her email. Ravenell told police that she had forwarded the email to a student’s cell phone, “who she knew would rapidly spread the message around various social media outlets and throughout the school,” and also sent it to the media and the NAACP, police said.
So, in this case, clear motive, a prominent figure, and a suspicious chain of custody gave the cops a reason to dig in a bit, but not everyone's gonna be that dumb about it.
The swatters who do this to random people are gonna have a field day with this.
> Police wrote in charging documents that Darien had accessed the school’s network on multiple occasions in December and January searching for OpenAI tools, and used “Large Language Models” that practice “deep learning, which involves pulling in vast amounts of data from various sources on the internet, can recognize text inputted by the user, and produce conversational results.” They also connected Darien to an email account that had distributed the recording.
One of our best defenses against crime is that criminals are often kind of dumb. Tyler Cowen makes the same point this morning regarding airport security: "a lot of criminals are simply some mix of stupid and incompetent or poor on execution" (https://marginalrevolution.com/marginalrevolution/2024/04/wh...).
Not just white collar, but many people go missing never to be found. So I assume there are likely competent murders who understand enough of biology and chemistry to not leave a forensic trail and not attract too much attention.
https://slate.com/business/2014/05/richard-branson-tax-fraud...
If Harold Shipman had stopped with the first 100 murders or so he would probably never have been caught: https://www.britannica.com/biography/Harold-Shipman
Usually that just creates more evidence
But just as petty crime has its bounds, so does legitimate business. This is why you have big tech companies that built their empires on lies and fraud. The justice system isn't optimized to catch this kind of crime, so it takes longer to get punished, and meanwhile every one of your competitors either learns to commit the same crimes or goes bankrupt. Furthermore the kinds of crimes you can commit once you've built a legitimate business are a lot more subtle and more lucrative.
So you have a society with an honest core of people doing the actual productive work, while being scammed by incredibly corrupt CEOs, who themselves distract the honest core with fears of petty criminals so that the justice system continues to expend its budget on cheap wins against the common crook.
While the rate at which murders are solved or "cleared" has been declining for decades, it has now dropped to slightly below 50% in 2020 - a new historic low. And several big cities, including Chicago, have seen the number of murder cases resulting in at least one arrest dip into the low to mid-30% range.
https://www.npr.org/2023/04/29/1172775448/people-murder-unso...
And it's also everyone who doesn't vote to actually enforce and improve tax laws and tax brackets.
That seems like a really terrible way to handle it.
All of these problems will be resolved. In which case most people either won’t raise the question of authenticity or won’t trust audio recordings to begin with.
Elections at all levels are at risk along with accountability more generally.
A screenshot is considered far more trustworthy by non-tech people than "I spoke to Fred and would you believe it, he told me X and Y were doing Z behind the bike sheds!!!"
Cryptography sometimes seem to rely on a stronger version this too. With enough computing power, you can brute force a lot. Some authentication seems just expensive enough that only a nation-state actor would have enough resources to break it, and then rests on the assumption that the small subset of people who could put in enough effort won't care enough to do anything worth being concerned about.
This is also why, say, people lock their doors even if they have windows.
The evidence submitted with the legal case was a cell phone with the messages app open ... and it appeared to be photocopied. Then they scrolled up ... another photocopy, and on and on and on.
Granted further investigation can be done, but it was amusing to see.
Someone saying something about what they talked about does not. You can say one thing today and another tomorrow.
This is why the written word has magic that the spoken word does not. And the longer the append-only chain of written words goes, the more magic it has.
They were arguing about what color to paint them! How scandalous!!!
We now have to adjust our expectations back to when we had to rely on multiple pieces of evidence from multiple sources and evaluating the trustworthiness of those sources before we get out the pitchforks.
You don’t need AI to falsify that evidence. An emotive image with a vague quote is often enough.
But I remember watching northernlion like 8 years ago and thinking "you could probably spoof this guys voice really easy with a nueral net"
The interesting thing is that in the Max Headroom days we thought that it would come about because of scheming corporate interests, not because of movie audiences that had pressed for ever better special effects, and fake Tom Cruises, all of these years.
I did. So I’m sure many others too. They just likely met the same opposition that I did.
Without defensive strategies today that allow those at risk to continue their risky behavior unencumbered then any warnings of tomorrow get drowned out by the demands of today.
Many professionals give talks that are recorded, whether lectures, promotional materials, Q&A, etc. People can be recorded surreptitiously, etc.
I think the only way to have avoided this is to be so paranoid as to have never allowed anyone near you with a phone or other recording device.
So what you're saying is this is unworkable. I mean decades ago private investigators were working with miniature digital recording devices, I can only imagine they have been reduced to sizes that are embeddable and undetectable in almost anything.
A) Have money; and
B) Really don't want even fake stuff getting out.
For me, I couldn’t care less. I’ll upload it to YouTube. A mommy vlogger, on the other hand, might not want her squeaky-clean image soiled.
Those were reversed because they were particularly egregious, but I can completely see individual students getting falsely accused of using AI and treated as guilty until proven innocent, going totally under the radar because single incidents don't get much public scrutiny.
Drug dogs who "hit" on command are already used as probable cause generators. Breathalyzers are basically magic boxes that produce "evidence" of a crime. It's inevitable that we're going to keep using AI and computers to automate convictions.
I use GPT to help write wiki entries for a site I'm working on and I have to be really on my game as it will hallucinate facts. I know some have got to have slipped through.
These free generators need to include some kind of audio watermark or key to indicate they are ai imitations. At least raise the barrier for this kind of action to being able to run your own llm or something.
This is a known and often repeated trend in tech: make something under the guise of disruption, with no regard for safety or regulations. Complain that those things are a hindrance to progress and then spend billions and destroy lives permanently to eventually fix the problems when the regulators issue an ultimatum that professionals warned you about.
I've seen wild swings in comments before based upon the time of day. E.g. mid-day upvotes, late day downvotes. Or vice versa.
It might be worth trying, but I'd bet that it's less than 6 months before running it locally means "download the app off the front page of your app store of choice".
Most criminals are lazy and/or not very tech savvy. Raising barriers and prosecuting the worst offenders cuts down on all sorts of malicious behavior that is technically feasible.
What's the point of considering potential abuses if you're just going to facilitate them regardless? If anything that's worse than not considering abuse at all, because it implies that you know what you've created will result in kids killing themselves after fake nudes of them spread around their school, or enable rampant fraud and extortion through voice cloning, but you believe that's just the price of progress.
If so, that is easily provably wrong, unless you think there's an enormous conspiracy involving thousands of people to pretend that they're working on it when they really aren't.
That is to say: I think this was inevitable.
I think that would result in the average person being less aware of the capabilities existing and therefore being less prepared to defend against it. It isn't like this would be a world law that was universally enforced anyway.
We are never ready for the consequences of technology. We adjust to it long after it has been invented.
I don’t know if this is the best use of the tech we’ve found yet, but it’s already a huge time saver.
Someone who is losing the ability to speak might want to have this tech so they can still have somewhat normal phone calls with their loved ones.
I think the potential for abuse is pretty high with this tech but it's foolish to pretend we can keep it from being used.
1. The obvious: There will be a lot of fake speeches floating around that spout fake news and hate-filled views.
2. The less obvious: The prevalence of hatred and sensationalized rumors will embolden those who find themselves agreeing with the extreme views seemingly endorsed by some authority figure, and will add their support and authentic voices to the mix.
Perhaps this is just an extension of what has already been happening with the internet. We will find ourselves more fragmented and divided than ever, filling our chambers with literal echos of synthetic voices.
4. Deniability in everthing that wasactually said. I think it's either Elon Musk or Donald Trump whose lawyers have argued in court/public that you can't be certain they actually said it.
Society benefited from a productivity gain by moving everything online, in a (relatively) high-trust environment. That is now becoming more expensive (due to higher % of frauds), or even infeasible.
So, a drag on economic growth for years to come.
Mobs will always remain dumb and quick to anger, so this will not be an isolated case.
How did he allegedly bypass payroll procedures? An athletics director should not have passwords to the school payroll system. I wonder if social engineering was used? In any case, their security procedures need an audit!
edit: typo
Wild times.
I don't think that tyrants work this way –- at least not in America's educational system. They're very happy to employ double standards.
They are no better than Al Capone.