I'm tired of the framing in the media these days.
"Mythos will end the world!!"
"How?"
"By finding a bunch of wide open security holes that have existed for years."
Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?
"Mythos will end the world!!"
"How?"
"By finding a bunch of wide open security holes that have existed for years."
Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?
But scale and accessibility are absolutely a new class of problem.
In the 1960s you could pay thousands of people to watch hundreds of cameras and listen to hundreds of phone lines to monitor people, but the cost was so enormous that unless you were in East Germany or Moscow it wasn't a realistic threat model.
Now with computers we can cheaply have thousands of cameras with cheap storage that's retained forever and automatic image processing that means everyone is exposed to that kind of surveillance, which is a brand new problem.
I've often shared my prediction that future historians will study us all and that every living human will be the subject of someone's PhD thesis. I'm updating that prediction to be that those future someone's will be silicon based.
Compared to "pay humans to watch hours of footage and take notes", $700 for a 16TB hard drive that will hold months of video footage is an absolute steal.
There's nothing government (corporate or political) has made "costly" or "dangerous" about not having them, society did that all by itself: people will actively pay more to have these things because they see the benefit more than the risk, video calls with friends and family, not a hacker being able to duplicate their keys from one photo.
There's 6 cameras on my desk right now, attached to internet-capable devices. Two phones, two laptops. I've got covers over all of them, which is easy, and sometimes mandatory e.g. when visiting the headquarters of certain big-tech firms.
I'd never buy a smart camera. Don't trust them not to spy on me. But I do have a Raspberry pi upstairs, with a NoIR camera module, and an AI-coded bit of webcam software. Might consider it for seeing what animal gets into the garden at night.
That's the risk of tools like Mythos/Fable/any LLM. While a human's eyes would glaze over what looks like a standard memcpy, an LLM with the right context might instantly realize the payload length was never actually verified.
And since Heartbleed existed for years, despite the full bug existing in pretty much one file, in one of the most important libraries out there, it's right to be afraid of what other obvious bugs exist and are just waiting to be found.
¿Porque no los dos?
All AI risk can be described with a narrative that ends in "some human were lazy and didn't care enough", it's just which humans and how much caring was enough.
“6-12 mo to shaky profitability + ability to quickly iterate” is a business that has a good chance of surviving while “However long it takes to be fully secure” is a business that is not only rigid but needs massive up-front capital to get there and even then there’s no guarantee that the market fit is right
And after that is something we could call the “Pareto spiral”: if a company find market fit and builds an excellent product, competitors can survive at 80% of that quality. If the “100%” fails for any reason, the competitors become the new ceiling and now their competitors can survive at 80% of that (now 64%)
And only one round in, how secure could that 64% company be?
Seems given that the new security equilibrium baseline will settle much higher.