A school locked down after AI flagged a gun. It was a clarinet | Hacker News Reader

A school locked down after AI flagged a gun. It was a clarinet

washingtonpost.com

103 pointsreaperducer133 comments

I feel like as soon as a particular type of student learns that this is used, they'll have an excellent way to get that test that they didn't study for postponed, and even have plausible deniability that they didn't intend to lock the school down. At least for the first one or two times, after that it's back to triggering the fire alarm.

At my high school it was bomb threats. At least two or three a year

That happened at my mom's school (back in the 1960s) until the school put all the kids on a bus and brought them to the local armory gym and made all student sit quietly (no talking) on the floor until the end of the school day. Once the bomb threat wasn't a way to get out of school on a nice day there was never another one.

I'm not sure how to apply that to this situation, but it is one every school should think about when students try things.

> Once the bomb threat wasn't a way to get out of school on a nice day there was never another one.

If I think of my school time, I would believe even the fact that a bomb threat would be an annoyance to teachers would a be sufficient reason (of course, in the schools of the country where I live there were other methods than bomb threats to be an annoyance to teachers).

An annoyance you can’t brag about to anyone, however.

That depends on who you think needs to learn a lesson about buying an AI from Temu.

That is a child abuse!

Solution is to make school non mandatory!

Not making kids go to school is child abuse. Sure they won't like it - learning new things is hard. However society cannot function without well educated adults. (maybe in the future some post scarcity society will emerge, but we are not there and there is high risk it won't come before they become adults)

> Solution is to make school non mandatory!

The obvious unintended consequence of this is that then nobody learns how to read (because who wants to go to school?) and then our economy crumbles and everyone is stupid and then everyone dies.

Or maybe some kids do go to school, so they become the global elite while the other 98% are illiterate and they flee the country because obviously and so the economy still crumbles and everyone dies.

See, these are the things we need to be thinking about.

Tangential question: what's an "armory gym"? Google gives me a specific gym (in Australia) called "The Armoury".

As I understand it, it's a US thing. In areas with snow the local military branch had a drill hall as part of the armory, to carry out drills in winter. These were large open interior spaces.

The military moved away from using urban armories. These became repurposed as sports halls, places for events, and so on. For examples, https://fightingillini.com/facilities/ui-armory/5 and https://www.wakefieldma.gov/Facilities/Facility/Details/Dril...

It's not exactly the same situation, but this happened before AI. At least the clarinet owner didn't get shot & killed like this guy:

https://en.wikipedia.org/wiki/Killing_of_Harry_Stanley

On 22 September 1999, Stanley was returning home from the Alexandra Pub in South Hackney carrying, in a plastic bag, a table leg that had been repaired by his brother earlier that day. Someone had phoned the police to report "an Irishman with a gun wrapped in a bag".[2]

> the prosecution evidence is insufficient to rebut the officers' assertion that they were acting in self defence

Awesome, they got away with unlawfully killing a man.

In protest at the suspensions, over 120 out of the 400 Metropolitan Police officers authorised to use firearms handed in their firearms authorisation cards, with Glen Smyth, a Police Federation spokesman saying, "The officers are very concerned that the tactics they are trained in, as a consequence of the verdict, are now in doubt."[10] The officers' suspensions were lifted shortly afterwards.[11]

What a joke.

Many, many people think the tactics they're trained in are exactly the problem - a few hours at most training in deescalation, little to no training in recognising or dealing with the mentally unwell, hundreds of hours of training in putting a bullet inn somebody

They got away with accidentally unlawfully killing a man while doing an inherently dangerous job by the book. You can blame the system for that certainly and propose changes, but I don't think it's fair to charge the officers with murder for acting in what they believed to be fully justified self defense in the normal course of their job.

> At the junction of Fremont Street and Victoria Park Road in South Hackney, close to his home, Inspector Neil Sharman and PC Kevin Fagan, the crew of a Metropolitan Police armed response vehicle challenged Stanley from behind. As he turned to face them, they shot him dead at a distance of 15 feet (5 m).

While it may have been an accident, it was a negligent accident. If I accidentally killed someone with a gun I would go to prison. In addition they were found guilty by the jury. It was overturned due to political pressure.

https://en.wikipedia.org/wiki/Killing_of_Harry_Stanley

I think what's fair is to expect police officers to have as their top priority the protection the citizens they are serving. Too often they seem to have as a top priority the protection of themselves, which means as a consequence that they'd rather risk the lives of others than their own. What good are they to us then?

You can't afford police that give up their own safety.

How much would your employer need to pay you for you to sacrifice your own safety for your job?

I travel to work, and engaging with the outdoors, traffic, etc. involves sacrificing safety that I wouldn't have if I stayed at home.

So, how much would my employer need to pay me? Not much, I guess? I definitely get paid less than the average cop in my city.

There are plenty of jobs that involve sacrificing safety, but very few of them give you the opportunity to kill people because you "didn't want to give up your own safety".

Very weird question. What do you think most jobs entail? Driving, mining, warehousing, even office jobs all come with risks. Perhaps the most relevant example though is firefighting. Just like in policing, there is obvious risk you take on in exchange for early retirement etc.

The last mindset I want policemen to have is to go into every situation assuming the worst because of a small chance their safety may be at risk. Which is exactly what it seems happened.

You can't just shoot someone who's holding a bag with something in it just because some random person has judged it could be a firearm.

“The officers had told the inquest that Mr. Stanley had turned around ‘in a slow, deliberate, fluid motion’ and pointed his wrapped-up table leg at PC Fagan, adopting a classic firing posture, which prompted Chief Insp. Sharman to open fire, hitting him in the head.” [1]

Yeah right, that definitely happened. An innocent man is dead because some damn pig wanted to kill someone. That is all this is.

"Times Online reports a “furious reaction from fellow police marksmen, who are threatening to lay down their weapons in a rebellion which could hamper security plans for the G8 summit of international leaders. Some members of the Yard’s elite SO19 firearms unit are already refusing to carry guns, saying that these arrests have shattered morale.”" [1]

barf

[1]: https://www.wsws.org/en/articles/2005/06/stan-j07.html

Being a policeman isn't even very dangerous, I don't think it even breaks top 50 most dangerous jobs.

It's not normally dangerous, on average. That doesn't mean confronting an armed suspect in a dangerous part of town is "safe" and that there's no reason for officers to get jumpy when that person turns and appears to point a concealed gun at them.

I'm not saying I know a better solution, but punishing officers for acting in what they believed to be self-defense in a very scary life-threatening situation doesn't seem particularly helpful. If you put your employees in a situation where they feel like they have to chose between a possible trip to jail and a possible trip to the morgue, they're going to pick jail every time, assuming they're even dumb enough to put themselves in that situation by working for you in the first place.

I agree it's not the correct solution because the issue is systemic, not individual. Punishing individual officers won't affect the overlying system.

But it's important to note that officers are specifically taught to be jumpy and aggressive. That's not a natural response. The reason we see so many accidental shooting is because the training around it encourages it. Accidentally shooting a few innocent people in the pursuit of stamping out crime is a necessary evil in the eyes of American police.

Ultimately, the police only really know how to shoot, and we put them in situations they're not equipped to handle. It's not shocking, then, that people end up getting shot. They truly have no business doing wellness calls, or mental health assistance, or even routine traffic stops.

On a long enough timeline, without any changes, this garbage is going to get a kid murdered by overzealous LEOs (or teachers in places where they want the carrying.)

“School resource officers” shoot kids (and themselves) with terrifying frequency, the kids typically unarmed and often autistic/neurodivergent.

(The worlds laziest lit review) https://letmegooglethat.com/?q=school+resource+office+shoots...

Or the next shooter will disguise their gun to look like a musical instrument.

Edit: I can't find it now but there is a body cam video of a Colorado school shooting where this almost happened. It wasn't due to an AI mixup but the school police was armed and I believe fired at one of the bad guys, but as the local PD responded he almost got shot because of the confusion: "school shooting, respond, guy with a gun"

That has been a common trope in gangster movies for ages. A violin case can easially hide a gun (depending on the gun of course, but you get to select this). You can find craftsmen in your gang who can make a violin case that also has a gun so it can even be opened an inspected by someone who doesn't know how to find the secret gun compartment, and it isn't hard to teach someone to play some simple songs on a violin thus giving a reason to carry a violin (you don't need to be good, just be able to show you have another lesson next week). I don't know if this works as well in the real world as movies, but it is plausible.

Also the plot of El Mariachi https://m.imdb.com/title/tt0104815/ and its sequel, Desperado.

What is LEOs?

Law Enforcement Officers, ie the Police.

Law enforcement officer, or police.

What cops want to be called instead of "cops", "the filth", or "pigs".

What's your point?

If the technology is even somewhat capable of detecting actual guns, it will probably save far more lives in the long run.

Cops killed a lot of people with negligible effect on crime. You're making an assertion that needs some support in light of the track record of US policing.

The obvious outcome of increased security and scanning to get into schools is what happened at Annunciation where the shooter just shot from outside the security area since all of those people still gather and walk past unsecured areas.

https://en.wikipedia.org/wiki/Annunciation_Catholic_Church_s...

I thought it was legal in America to bear arms? Is the constitution magically suspended in schools?

In many ways, yes. With guns specifically, the Supreme Court (in opinions authored by Justices Alito and Kavanaugh, believe it or not) has set a precedent that there are some government-controlled "sensitive places" within which it is reasonable and prudent to place restrictions on firearms. Legislative assemblies, courthouses, government buildings, etc.

So not even the most conservative justices buy into the idea that Americans have a right to bear arms everywhere.

"Some of you may die, but that's a risk I'm willing to take!"

I thought it's well established that LEO's dont enter schools where there is a suspected shooting? So a clarinet toting suspected gunman should he just fine.

They tried to find contraband, they found a marching band!

The sleazy AI company that sold this "tech" to the school has to face the music.

Sleazy? I'll have you know, the ToS clearly says it's in beta.

Next time the contraband could be a contrabassoon.

Seventy-six trombones caught the morning sun

With a hundred and ten cornets right behind

The AI thought about it long and hard

calling up the national guard

Cause of the horns of ev'ry shape and kind.

---

There were copper bottom tympani in horse platoons

Thundering, thundering all along the way.

Double bell euphoniums and big bassoons,

And Swarming SWAT Teams Goons so they say

---

There were cross fires and blown tires

And reporters from the local news

Clarinets of ev'ry size

And trumpeters who'd improvise

And video games that went pew-pew!

----

on multiple edits: tried to find a layout that fit the song

Those banned band books… they might have come across One Thousand and One Vulgar Marching Band Formations!

Well at least it wasn't a bag of chips this time. A clarinet is at least a step towards the correct shape. Sounds like the AI training is going well!

Is the gun situation hopeless in America? As far as I can tell, we just need to reduce the quantity of guns. Somehow 20% of the country takes that idea as an existential threat, even though not doing so is a material threat to real people.

If it's actually a mental health crisis, then how do we solve that one? Especially with what the Republicans are doing right now?

the guns don't even seem to work for their supposed primary purpose

uninvited and unwanted federal troops are roaming around cities against the wishes of state governors

so seems the US has the worst of both worlds: unqualified morons owning assault weapons, plus the tyranny

Because you have Republicmas idiotically saying "Guns don't kill people; people do" and then don't do anything about the people or the guns. Conservative gun owners simply accept that people must die by guns in order for them to feel safe and make us all more vulnerable because of their irrational fears. They're babies with weapons.

That's simplistic. Actually, to better understand the situation you must follow the money. Many 2nd Amendment supporters are reasonable, but unfortunately, over decades their casual support has been utilized by lobbyists whose goals do not necessarily align with many supporters. The challenge is to communicate that message in a way that reaches everyone.

We just need to reduce the number of guns. I've not met a 2nd amendment supporter who understands this basic idea. They are always convinced one of the following retorts should be the end of the conversation (they also proudly think you've never heard these cliched arguments):

- They have knife stabbings in China. (Yes. A gun is more lethal.)

- A bad guy can still get a gun. (Yes.)

- Hand guns are more dangerous than rifles. (This means let's reduce both.)

- The gun doesn't kill people. People kill people. (This means let's reduce how many people have guns.)

- Mass shootings aren't the majority of gun deaths. (Let's reduce the total gun deaths and mass shootings then.)

Come up with as many ridiculous retorts as you like. If you had reduced the total number of guns, most of the shootings could not have happened.

Don't you think the populace needs to be armed though? I think its a given that eventually the government will be intolerably corrupt and a revolution will be necessary. Nobody denies that less guns -> less shootings. The logic is that some amount of shootings are tolerable to preserve democracy, and that if our goal is to reduce mass shootings, social reforms intended to improve mental health are the correct choice.

Imagine you are in charge of a monkey enclosure. The monkeys sometimes go crazy and kill each other with rocks. You can:

A: Remove all rocks. Monkeys stay suicidally miserable but can't inflict harm as easily. Problem solved?

B: Mitigate conditions that make them suicidally miserable. Some say its impossible, but then again, just a generation ago the monkeys had rocks without frequent violence.

Why is it that only our country needs to be armed, but none of the others do? Do Germans need to be armed? Do Indians need to be armed? Maybe you could argue that Chinese people need to be armed, but every Chinese person I've met seems to like their government right now and is content with the levels of surveillance. Maybe arming the Uyghurs could have helped but somehow I doubt it.

All populations should be armed. The current democratic, liberal order in Europe is just a side effect of America's dominance, the same America that is the product of the revolution of a well armed population. Your counterpoint might be the U.K., which has arrested 12,000 people for social media posts recently.

The catch is, it only works with an enlightened, well educated population with philia and a sense of civic duty. Arming inner city Chicago has been a disaster.

That's tough to maintain that state, but we have to try, because if a population doesn't fit that description the country turns to shit and you won't want to live there. To disarm is to admit we can't be an enlightened country anymore and we won't try, and after that its just a matter of time until there is nothing special about America and its just another mediocre third world dump.

If a government were run by quakers, should the population require the same level of armament as if it were run by Attila? Perhaps by creating better governments we could reduce the need to arm populations.

I just don't see what arming citizens is going to do against a militaristic government.

Yes, because the idea is to establish an armed populace before society succumbs to tyranny, not in response to it. The central tenet is that even if a society is run by Quakers now, it won't always be, because in the absence of proper inputs societies tend to decay to their stable basal state which is despotism. When that happens, the population must be armed in order to revolt and restore a democratic system. I would even say its better to arm the populace while the government is still Quaker because that would establish the proper cultural mores surrounding gun ownership in an enlightened environment - e.g. knowing that gun ownership is a responsibility and right, connecting it with ideas of liberty and civic duty, viewing them as a last resort, learning about guns from your father and not your homie on the corner.

You need to have the population armed beforehand. Its not practical to try to dynamically adjust how armed the populace is in proportion to perceived governmental Attila-ness.

To your last point, an armed populace makes revolt feasible, and there is a spectrum here. The key is that the oligarchy will need to convince the army to stay on its side and punish the revolting populace. The more sacrifice and violence that is required, the harder it is to keep convincing them. Also, look at the insurgencies in Iraq and Afghanistan: an armed, hostile populace is just much harder to control than an unarmed one. It dramatically increases the cost of every excursion from a military base, the number of soldiers required to subjugate an area, etc, and the grand lesson from those conflicts is that boots on the ground are still needed to control an area, and that technological solutions like drone strikes still don't scale well enough and aren't cleanly targeted enough to change that. Perhaps that will change in the future, but I actually suspect that the prevalence of consumer drones will maintain the power of the public to resist the military. Look at Ukraine and Russia; the dominant weapon system now is the consumer drone, eclipsing even artillery, which has democratizing implications for the future of the tug-of-war between societies and their governments.

Well it's been interesting talking to you. In good faith, I honestly cannot conceive of how arming the population prevents tyranny though. You give the example of Iraq and Afghanistan. Presumably you're saying the tyranny was the US occupation? Weren't these groups armed not before, but as a result of first (I believe) soviet and then American occupations?

Are there examples in modern times of a stable society consisting of a heavily armed population such that as a result of this tyranny has been curbed? Americans are the most heavily armed population in the world and it seems that tyranny is measurably setting in right now. The stability seems like it was higher in the beginning of the 20th century too. The 60-70's and now are the most unstable periods I believe, and the number of guns has only increased. So I don't think the US would be a good example.

In good faith, I cannot see how arming civilians reduces tyranny in modern times, unless your model actually is Afghanistan and Iraq. In those cases it's not that all civilians are armed. There are armed groups. That's not a world I want to live in though, anyways.

Yes, the tyranny in Iraq was the US occupation, though I am merely using it as an example that an armed, civilian populace can resist military control, not commenting on the morality of the occupation.

FWIW, Iraq was already well armed before the occupation, and looting of state arsenals in the chaos of the invasion amplified this. Afghanistan was a similar situation but armed networks were already organized moreso before the US occupation.

I don't think there are examples in very modern times of an enlightened, armed populace revolting against tyranny. The most recent I can think of is the Irish war of independence. They had low gun ownership, but correctly recognized the attainment of arms as of utmost importance, and it was through arms that they obtained liberty. I also still think that the American revolution is a fair example because the fundamental dynamic is still relevant. The Algerian war of independence comes to mind as well, which was more recent, though they were neither enlightened nor well-armed at the outset. Generally an enlightened society will produce a democracy which will take centuries to decay to the point of warranting revolt, and we are still in the first generation of these.

To your point about the US, merely having an armed populace does not gradually move society away from tyranny. The mental model myself and the founding fathers have is that even in the case of an armed populace, democratic institutions eventually decay to the point that the government is corrupt, despotic and intolerable. Its just entropy, as happens to our bodies. The population then revolts, and installs a democratic government, which then starts to decay again, and the cycle repeats. The fact that America is moving in a bad direction is confirmation of this tendency to decay, and not in any way antithetical to my stance. Eventually the decay surpasses a threshold which triggers the guns to come into play and reset the system.

And its been interesting talking to you too.

This list is spot on, and the biggest fuel on the fire is the problem of huge financial incentives. I can assure you there are some supporters out there who do understand. Some who do not understand have certainly been fed talking points by entities who may or may not care about the intent of the 2nd Amendment exactly, but definitely do care about making money.

The thing I think about is that, as far as the two major parties are concerned, they both support denying Americans the right to bear arms, and they both support gun control. The difference is that one party won't admit to it, making any debate difficult or impossible.

On the flip side, I feel like the two parties see things differently: One party is mostly about prevention, while the other side is predominantly about punishment.

> If it's actually a mental health crisis,

That's the scapegoat. It's an easy scapegoat because it blames others. And others are easy to hate. They aren't us. "They" are others. Yes, mental health is an issue, but it's not so much higher than in other nations to explain the much higher rate. It's a scapegoat because it's easy to say, "Wow, no normal person would do that, so they must be crazy." That sounds reasonable if you don't think about it for a moment.

I think it's a scapegoat, but I want to give the scapegoaters a chance to show their asses, and I mean show whatever study they've got to see if there's anything there.

Since 1791, the right to be armed has been considered a fundamental human right in the US, on the same level as freedom of speech and religion. You're fighting an uphill battle by trying to argue we should take away people's fundamental human rights in the name of public safety. To many, it's essentially no different from if you were trying to argue we should take away people's rights to free speech because some people use that right to promote violence.

Whether you think that makes the gun situation "hopeless" depends on whether you think there are other ways to reduce violence without taking away the rights of law abiding citizens.

The more fundamental human right is the right to not be blasted by an angry teenager who's been reading crazy things on the internet. In the hierarchy of human rights, that once has precedence.

So we should take away people's rights to "read crazy things on the internet" then, right? Because the right to not be killed is more important.

Do you see the problem with that sort of thinking?

You can misread what I wrote and argue against all that strawman all day if you like.

You haven't even represented the true content of the 2nd amendment so there's nothing to talk about here. The 2nd amendment is about militias, not "give everyone and their toddler access to massive fire power at all times".

"A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed."

Tell me where that says guns should be available to deranged person who wants one. Everyone should get a hand gun, a rifle, and a shotgun, and make it double.

I didn't misread, you're missing the point. I'm saying your argument for infringing on the second amendment is equally valid for infringing on the first. (Which is to say, not very valid at all.)

You did misread my post. My argument does not say that at all, and I cannot see how it possibly could. My argument is that reducing the number of guns would reduce the number of gun deaths.

That's simply not true. That's a very modern interpretation of the second amendment.

The /NRA/ lobbied for some of the first gun control laws in SanFran, because the Black Panthers were open carrying. Even in the days of the 'wild west' there were plenty of towns and cities with enforced gun control ordinances.

Just as we have anti-defamation laws despite freedom of speech being a constitutionally protected right, you can have gun control laws despite carrying guns being a constitutionally protected right. I don't think that makes what I said untrue. Such laws (in both cases) are kept on a very short leash, both from a legal and a cultural perspective.

Presumably these shootings are done with the guns of parents/friends? How else is it possible for a school kid to get access to firearms so easily?

I think the solution is to make it difficult for underage kids (U21's) to access guns so easily. And maybe start fostering some healthy habits and relationships by starting with the young.

In countries with rational gun laws the owner is held fully responsible for everything their guns do, until they declare it stolen/lost. And then an inquest is held to determine if the owner was negligent in allowing the firearm to be lost/stolen in the first place. Subsequently, guns are either kept on the person of the owner (in the case of a handgun), or they are locked away in a Proper Safe (not a filing cabinet that's painted black ffs).

>If it's actually a mental health crisis

If?

The institution you use to socialize your youth results in some attempting to murder their co-participants.

Even if you took away the guns so they couldn't follow-through, why are your schools fucking up child rearing so badly in the first place?

Okay how do you want to solve that? If it's by making the schools better I'm all for it. Why is it that the people who love guns vote to defund the schools though?

> “We don’t think we made an error, nor does the school,” Alaimo said. “That was better to dispatch [police] than not dispatch.”

Of course the co-founder of the company that made the error would say that.

Good grief, imagine if it had been an oboe.

A weapon of mass diminuendo.

Or a bassoon!!

https://archive.ph/WbOWl

This hardly needs to be said here, but there should have been human review of the AI output before taking any drastic action. That would (I assume, though since I can't read the original article I don't know if that assumption is correct) have immediately let them know that the alleged "gun" was nothing of the sort, and avoided the massive disruption of a totally unnecessary lockdown.

Not sure how much that would help overall.

Unless it's completely clear that it's not a gun, the reviewer is essentially always going to pull the alarm. The risk of a false alarm is going to be seen as minimal, while the risk of a false negative is catastrophic.

False alarm makes the news for now because it's novel, we all go "What the hell, guys?" and life goes on.

Nobody wants to end up sitting in front of a prosecutor, the media, etc explaining why they chose not to pull the alarm, when the AI _clearly_ identified the gun, and instead chose to let all those kids die.

One more instance of offloading blame to a computer system: "It's not my fault the cops shot the kid, the system said it was a gun."

The only way this gets fixed is if there are consequences at every level for false positives.

One approach for this is that the person who makes the call needs to be on-site and in the front of the situation --- similarly, a judge signing off on a No-Knock Warrant --- the judge needs to be at least be present, and should be required to walk through the building/home/apartment after the warrant is served. If it's not important/severe enough for a judge to do this, then I would argue that there's no need for the "no knock" aspect.

That sounds good on paper, but is really impossible to implement in any practical way.

In this case, the kid was holding the clarinet like a weapon, and though we have not seen the actual video, the descriptions of it make it sound like overall resolution was poor.

The alternative to the false positive here, is to not report anything that you cannot be 110% certain of, which means that you're likely to miss some true positives.

Overall this situation mostly reads like everything worked as intended, and the press turned it into more than it needed to be. School shooting are a real thing, there is plenty of evidence of that. Weapons detection has become a necessary component of a school safety strategy. For many reasons, it is not practical to have personnel at the school, or within the district, act as the first-pass reviewer of AI detections of weapons.

Don't be defeatist. The situation under consideration here is probably monitored by security cameras and body cams end to end. Everyone not following correct protocol did so on camera. Punishing willful ignorance and incompetence is certainly possible.

>The only way this gets fixed is if there are consequences at every level for false positives.

Do we really want consequences for false positives? If a kid is smoking a cigarette in the bathroom and the smoke detector goes off, the school should evacuate. The Smoke Alarm went off. No principal is going to sign off on the assumption that "Timmy is smoking, it's not a real fire". The principal shouldn't be punished for responding to the alarm. Timmy...probably should get reprimanded, but that feels off-metaphor.

In the example we are given, Timmy did nothing wrong. Having a clarinet is not contraband, and he should not be punished. The admin who called a lockdown did nothing wrong, as they were responding to the system in the way they were trained to use it. This is all in the name of safety, where things are done in 'an abundance of caution'.

>"It's not my fault the cops shot the kid, the system said it was a gun."

No, its the cop's fault. The cop hasn't been trained to use the AI security system, and is instead given their own SOP for assessing threats.

Make the failure billable for failures in the contract as it is a significant downtime.

For every false alarm you need to pay the salaries that were wasted and the snacks and therapists for the kids.

Likewise for every missed gun hazard pay for teachers and therapists for kids.

If they aren’t confident enough to back a service that has such a mental impact on failure they should not be selling it.

This is just conjecture, but I suspect there will be as much review of photos, application of good investigative work and overall professionalism as is conducted during anonymous, virtually untraceable Swatting incidents that terrify the victims, if not get them killed.

There was. From the article: '''

ZeroEyes said that trained employees review alerts before they are sent and that its software can make a lifesaving difference in averting mass shootings by alerting law enforcement to weapons on campus within seconds. At Lawton Chiles, the student flagged by ZeroEyes was holding his musical instrument like a rifle, co-founder Sam Alaimo told The Washington Post. “We don’t think we made an error, nor does the school,” Alaimo said. “That was better to dispatch [police] than not dispatch.”

'''

This appears to be a ZeroEyes customer, which means there was a human in the loop. One of ZeroEyes' main selling points is their use of 24/7 staffed SOCs to review anything the AI software flags before sending an actual alert to the site/customer.

Nearly all police responses in the US are armed responses. Americans don't realize how dangerous that is. They recognize swatting as a crime, but don't realize that a Karen who lies to police in order to get them to bully someone they don't like is very dangerous. Comparable to aggressive driving and road rage. It needs to be prosecuted more often. By more often I mean more than zero.

Lots of jokes made, but in reality this mistake could have been made by a real human watching a video feed. The student was intentionally mimicking a weapon with a long, tubular object. It's not like he was just walking down the hall with it in his hands, and suddenly SWAT TEAM!

AI is often awful, but I'm giving it a pass on this one.

I can't give a pass - kids do things like this often enough that you can't call the police on every incident. Sure it might be stupid for kids to do, but they do it, and since it isn't a real gun it is harmless fun until someone overreacts. The real issue here is fatigue: the more you overreact the less your seriously you are taken when something really does happen.

Once you can make a mistake and be okay, but you soon you become "the little boy who cried wolf", and people don't move to safety when there is a real issue!

> The student was intentionally mimicking a weapon with a long, tubular object.

That's... pretty much every student with any long, tubular object. Of which there are many.

The article notes the kid was (1) wearing a military costume and (2) then using his clarinet to simulate a weapon.

I don't know about the AI's decisions, but he would have caught the eye of a casual human observer. My point is: it was a bit more involved than just a 5yo pointing his finger and shouting "POW! I got you!".

A bit more. But not much.

Do they have a way to uncall the alarm? Presumably between the time the kid mimicked a weapon and the police arrive, the kid stopped mimicking a weapon and it became clear that it was a clarinet?

Its weird how object detection models are "AI" now. These models and their weird errors have been around for quite a while. The issue is vendors claiming that there is no chance of errors. Ideally you would have a 2 eyes system such that if AI has a tolerable false positive rate and have a human review. But of course, you cant fire people with AI so why would we do the sane thing.

And of course there are policy wonks who would make gun ownership a human rights issue even though its fundamentally unsafe to have such free gun ownership.

According to the article, they did have a human verify the images before sending the alert. Apparently they and the school still think they made the right call.

AI has also been around for quite a while. LLMs are hardly the first instance of AI we've seen, just the one that's suddenly getting all the hype. But yes, people trust it too much.

weapon of brass destruction

At least it wasn’t a percussive device

If it was, they'd need to get the cops on the horn quickly

Don't these cops receive an image/video of the event in addition to the AI description?? Who tf. designed the system?

Maybe its not wrong? Clarinets have harmed me far more than guns ever have...

As much as I believe this is a story that needs to be told, why did the author of the article choose one of those annoying two-sentence millennial titles? Do they actually generate more publicity?

A man saw a snappy two-sentence title. He was mildly annoyed.

why did the author of the article choose one of those annoying two-sentence millennial titles?

In big journalism organizations like this one, the writers don't come up with the headlines. There are other people for that.

Another false positive of a system working as intended.

https://news.ycombinator.com/item?id=45684934 (the doritos gun)

Next up, the guy, probably named Buttle, will get billed for the intervention.

It’s all perfectly in order. We have the receipt.

WHAT HAVE YOU DONE WITH HIS BODY!?

Edit: I don't know about these specific lines, but this script was co-written by the great and recently deceased Tom Stoppard

I have so, so many questions.....

Why are there cameras in schools? Why are they looking for guns? Why are they using AI to do it?

Could this not all be avoided by not letting kids have guns or am i missing the point?

There are cameras in schools for the same reasons there are cameras anywhere, to try to maximize accountability.

Kids are already not allowed to have guns.

As a parent, I would like more cameras in schools, since students’ abuse gets ignored, and the person fighting back gets more punishment than the instigator.

How would cameras solve the punishment issue? Aren’t most schools still “zero tolerance” for physical violence?

The post you are responding to is about punishing the victim because teachers are too lazy/cowards to punish the culprits. Cams incentivize them to do the right thing.

Does it work? I'd be interested to know if you know of any statistics around this.

Probably not but won't you think of the children?

It would take the incident from "he said/she said" to video evidence. Why wouldn't that work?

Are you interested in the statistics or in knowing if I know any?

I hate camera proliferation but, judging by the YouTube plenty of bodycam footage, it does work.

You won't likely find the numbers you want because the very nature of the problem is that cameras make visible something that wasn't.

Even if the situations are noticed and seen fully, does it cause the schools to not punish the victim? The stories I've heard about zero tolerance policies were that _even when the situation was fully obvious_, victims got punished because they took part in an altercation.

The video evidence is just one piece of the puzzle that is needed to help administrators properly adjudicate conflicts, and to help the public hold the administrators accountable.

If the rot is so deep that even who was right and who was wrong does not matter, then that is a separate issue that members of the public need to sort out with each other.

By removing the defense of plausible deniability for administrators, just like bodycams do for cops and dash cams do for bad drivers.

It should be legally required for daycares to have cameras. Those kids cannot communicate, so there is zero accountability there. And this only becomes a thing because it’s so incredibly cheap to add the accountability.

Obviously, in before times, when it was too expensive, a cost benefit decision has to be made to go with trust only. But now that the cost is trivial, that cost benefit decision has to be revisited.

As a former student/child, who fortunately has grown up in a place where none of the ridiculous restrictions typical for US schools existed, I find extremely sad that there are such places on Earth where the management of a school has decreed that the student was guilty even without possessing any weapon because "was holding his musical instrument like a rifle".

If not even the children can play any more however they want, for fear that automated surveillance would identify them as "pretending to have a weapon", which can result in punishment, I believe that such a society has serious problems for which it certainly did not find the right solution. I would not want for myself or for my children to live in such a place.

As someone who read the actual article, nowhere did it say that the student was decreed guilty. In fact, the police explicitly said "no further action was needed." Unclutch your pearls.

I believe that you have not read carefully either the article or my posting.

I have not said anything about the police.

I have quoted directly from the article exactly what the school management said.

It is the school management who has said that the guilty party in this incident is neither the school nor the surveillance equipment, but the student who "was holding his musical instrument like a rifle". (This does not even take into consideration that there may be legitimate reasons for holding a musical instrument like a rifle, e.g. for checking it for defects or dirt.)

It is also the school management who implied that they will reprimand any student for "pretending to have a weapon".

I do not know how you played as a child, but "pretending to have a weapon" is certainly an exceedingly common behavior, perhaps more so in countries where people would never think about using true weapons.

https://en.wikipedia.org/wiki/%27No_Way_to_Prevent_This,%27_...

Because USA, land of the free, and armed.. There is too much violence and tools enabling there, so everyone needs ways to survive. Cams are useful to locate all kind of problems; gun are not the only tool used.. I guess this is the price of liberty.

How do you propose to not let kids have guns?

Yeah, there's nothing that can be done about it, says the only nation overrun with this problem... <credit to The Onion>

You're missing the point of my rhetorical question. A school can't influence national gun policy. So if the school wants to not let kids have guns, then they have to check for guns.

Because every other country on the planet when faced with the same kind of issue tightened its gun controls in response - Britain had a famous school shooting which immediately led to outlawing all hangun possession by private individuals, there hasn't been a shooting since(despite gun ownership in the UK being actually relatively high, it's pretty common for farmers to have shotguns). In contrast, whenever a school shooting happens in the US(and it's almost 100% every single day of the year now), legislators propose loosening gun restrictions to solve the issue. With some prominent voices saying that well if only teachers were armed, this wouldn't be a problem. And thus, the school shootings in the US not only happen every day, they are such a normal occurance that no one protests at the idea of automated AI cameras scanning students for weapons or schools having metal detectors or armed officers on site, all of which are insane in pretty much every other society. But no, in US you can't solve that - and I have no doubts that we'll have replies to this very comment explaining why US is so special that these same measures couldn't possibly work or worse - how those daily school shootings are a necessary price to pay for freedom of gun ownership and upholding the constitution.

Just ban music instruments altogether then! No need to fix the dang LLM prompt this way! /s

Reply on news.ycombinator.com