I'm not familiar with this parable, but that sounds like a good thing in this case?
Notably, this is not a gun.
Notably, this is not a gun.
For example using your LLM to criticise, ask questions or perform civil work that is deemed undesirable becomes evil.
You can use google to find how the UK government for example has been using "law" and "terrorism" charges against people for simply tweeting or holding a placard they deem critical of Israel.
Anthropic is showing off these capabilities in order to secure defence contracts. "We have the ability to surveil and engage threats, hire us please".
Anthropic is not a tiny start up exploring AI, it's a behemoth bank rolled by the likes of Google and Amazon. It's a big bet. While money is drying up for AI, there is always one last bastion for endless cash, defence contracts.
You just need a threat.
how dare they invoke law
Right?
The imagined ideal of a smart gun that perfectly identifies the user, works every time, never makes mistakes, always has a fully charged battery ready to go, and never suffers from unpredictably problems sounds great to a lot of people.
But as a person familiar with tech, IoT, and how devices work in the real world, do you actually think it would work like that?
“Sorry, you cannot fire this gun right now because the server is down”.
Or how about when the criminals discover that they can avoid being shot by dressing up in police uniforms, fooling all of the smart guns?
A very similar story is the idea of a drink driving detector in every vehicle. It sounds good when you imagine it being perfect. It doesn’t sound so good when you realize that even a 99.99% false positive avoidance means your own car is almost guaranteed lock you out of driving it some day by mistake during its lifetime, potentially when you need to drive it for work, an appointment, or even an emergency due to a false positive.
People acccept that regular old dumb guns may jam, run out of ammo, and require regular maintenance. Why are smart ones the only ones expected to be perfect?
> “Sorry, you cannot fire this gun right now because the server is down”.
Has anyone ever proposed a smart gun that requires an internet connection to shoot?
> Or how about when the criminals discover that they can avoid being shot by dressing up in police uniforms, fooling all of the smart guns?
People already do this.
This is stated as if smart guns are being held to a different, unachievable standard. In fact, they have all the same limitations you've already pointed out (on top of whatever software is in the way), and are held to the exact same standard as "dumb" guns: when I, the owner, pull the trigger, I expect it to fire.
Users like products that behave as they expect.
You’ve never had a misfire or a jam? Ever?
Gun owners already treat reliability as a major factor in purchasing decisions. Whether that reliability is hardware or software is moot, as long as the thing goes "bang" when expected.
It's not hard to see the parallels to LLMs and other software, although ostensibly with much lower stakes.
But zero smart guns are on the market. How are they evaluating this? A crystal ball?
Why do we not consider “doesn’t shoot me, the owner” as a reliability plus?
It doesn't take a crystal ball to presume that a device designed to prevent a product from working might prevent the product from working in a situation you didn't expect.
> Why do we not consider “doesn’t shoot me, the owner” as a reliability plus?
Taking this question in good faith: You can consider it a plus if you like when shopping for a product, and that's entirely fair. Despite your clear stated preference, it's not relevant (or is a negative) to reliability in the context of "goes bang when I intentionally booger hook the bang switch".
I'm not trying to get into the weeds on guns and gun technology. I generally believe in buying products that behave as I expect them to and don't think they know better than me. It's why I have a linux laptop and an android cell phone, and why I'm getting uneasy about the latter.
As far as your comparison with misfires and jams, well... for one thing, your average firearm today has MRBF (mean rounds before failure) in the thousands. Fingerprint readers on my devices, though, fail several times every day. The other thing is that most mechanical failures are well-understood and there are simple procedures to work around them; drilling how to clear various failures properly and quickly is a big part of general firearms training, the goal being to be able to do it pretty much automatically if it happens. But how do you clear a failure of electronics?
Arguing that something might fail and therefore any additional changes that can introduce failure modes are therefore okay is the absolute worst claim to hear from any engineer. You can’t possibly be trying to make this argument in good faith.
Facial recognition: 3D Infrared
Fingerprint: CapacitiveOr you didn’t press it in the right spot?
Or the battery was dead?
If I’m out in the wild and in a situation where a bear is coming my way (actual situation that requires carrying a gun in certain locations) I do not want a fingerprint scanner deciding if I can or cannot use the gun. This is the kind of idea that only makes sense to people who have no familiarity with the real world use cases.
Dressing up in police uniforms is illegal in some jurisdictions (like Germany).
And you might say 'Oh, but criminals won't be deterred by legality or lack thereof.' Remember: the point is to make crime more expensive, so this would be yet another element on which you could get someone behind bars. Either as a separate offense, if you can't make anything else stick or as aggravating circumstances.
> A very similar story is the idea of a drink driving detector in every vehicle. It sounds good when you imagine it being perfect. It doesn’t sound so good when you realize that even a 99.99% false positive avoidance means your own car is almost guaranteed lock you out of driving it some day by mistake during its lifetime, potentially when you need to drive it for work, an appointment, or even an emergency due to a false positive.
So? Might still be a good trade-off overall, especially if that car is cheaper to own than one without the restriction.
Cars fail sometimes, so your life can't depend on 100% uptime of your car anyway.
Try using this argument in any engineering context and observe how quickly you become untrusted for any decision making.
Arguing that something doesn’t have 100% reliability and therefore something that makes it less reliable is okay is not real logic that real people use in the real world.
We famously talk about the 'numbers of 9s' of uptime at eg Google. Nothing is 100%.
> Arguing that something doesn’t have 100% reliability and therefore something that makes it less reliable is okay is not real logic that real people use in the real world.
That wasn't my argument at all. What makes you think so?
I'm saying that going for 100% reliability is a fool's errand.
So if the device adds a 1/1,000,000 failure mode, that might be perfectly acceptable.
Especially if it eg halves your insurance payments.
You could also imagine that the devices have an override button, but the button would come with certain consequences.
Sadly, we’re already past this point in the US.
Who decides when someone is doing something evil?
That seems a valid use case that'd get hit.
Anti-State libertarians posit that preventing this capture at the state level is either impossible (you can never stop worrying about who will watch the watchmen until you abolish the category of watchmen) or so expensive as to not be worth doing (you can regulate it but doing so ends up with systems that are basically totalitarian insofar as the system cannot tolerate insurrection, factionalism, and in many cases, dissent).
The UK and Canada are the best examples of the latter issue; procedures are basically open (you don’t have to worry about disappearing in either country), but you have a governing authority built on wildly unpopular ideas that the systems rely upon for their justification—they cannot tolerate these ideas being criticized.
It'll become apparent how woefully unprepared we are for AIs impact as these issues proliferate. I don't think for a second that Anthropic (or any of the others) is going to be policing this effectively or maybe at all. A lot of existing processes will attempt to erect gates to fend off AI, but I bet most will be ineffective.