Manual for a popular facial recognition tool shows how much the software tracks
themarkup.org
themarkup.org
> The Santa Fe Independent School District’s neighbor, the Texas City Independent School District, also purchased AnyVision as a protective measure against school shootings. It has since been used in attempts to identify a kid who had been licking a neighborhood surveillance camera, to kick out an expelled student from his sister’s graduation, and to ban a woman from showing up on school grounds after an argument with the district’s head of security, according to WIRED. > “The mission creep issue is a real concern when you initially build out a system to find that one person who’s been suspended and is incredibly dangerous, and all of a sudden you’ve enrolled all student photos and can track them wherever they go,” Clare Garvie, a senior associate at the Georgetown University Law Center’s Center on Privacy & Technology, said. “You’ve built a system that’s essentially like putting an ankle monitor on all your kids.”
As a society I think we need to be concerned about what happens when this technology becomes more wide spread and normalized. I’m not holding my breath though.
No examples, outside of social media and authentication, come to mind for the US. I’m probably out of the loop on this one.
Edit: I did find mention of facial recognition being used for 7-Eleven’s in Australia and trialing of the technology at Target and Walmart in the US [0].
> [7-Eleven] confirmed that the data captured by the facial recognition software that is being introduced nationally, will be used to verify customer feedback rather than as a measure to prevent theft – in the US, Target and Walmart among others have experimented with facial recognition to combat shoplifting and fraud.
The article also had this at the end.
> Facebook also submitted a patent for in-store facial recognition tech that would provide retail staff with customer information drawn from social media profiles in a bid to deliver a more personalized service.
Edit: And this ACLU article [2] from 2018 implies many large US retailers are using it in store as well without notifying customers.
Edit: Oh and schools as mentioned in this article.
On a related but maybe ironic note SF and some surrounding municipalities have suspended since mid 2019 the use of facial recognition by government departments (with the exception of ports) [1]. So at least there is that. However private companies and people can do as they may.
> In the Bay Area alone, Berkeley, Oakland, Palo Alto and Santa Clara County (of which Palo Alto is a part) have passed their own surveillance-technology laws. Oakland is also currently considering whether to ban the use of facial-recognition technology.
> The ordinance… outlaws the use of facial-recognition technology by police and other government departments
> However, the ordinance carves out an exception for federally controlled facilities at San Francisco International Airport and the Port of San Francisco.
> The ordinance doesn’t prevent businesses or residents from using facial recognition or surveillance technology in general — such as on their own security cameras
0. https://techwireasia.com/2020/07/why-7-eleven-australia-is-u...
1. https://www.cnn.com/2019/05/14/tech/san-francisco-facial-rec...
2. https://www.aclu.org/blog/privacy-technology/surveillance-te...
Otherwise, "think of the children" will always prevail in western cultures, and parents will sign everything just to protect their kids, even from nonexisting harm... sadly.
Is this currently mainly matching people with their past self's model, or is this also integrated with data connecting us to our identities (for the average suburbanite who doesn't visit locations which scan IDs frequently)
The second thing one can do is draw or place stickers on your face with additional eyes, noses, and/or mouths on your face. Similar to the fantasy/art makeup look where people paint an additional set of eyes on their forehead - the FR will see 6 eyes or 3 mouths and will reject that as being a human face.
A lot of the "rich" world had anti-mask legislation, and failing the presence of legislation a willingness to put pressure on their citizens to be "identifiable" in public at all times (where I am in Quebec some attempts being repelled were still news as of 2017 iirc)
"The school district originally purchased AnyVision after a mass shooting in 2018, with hopes that the technology would prevent another tragedy."
I'm curious how facial recognition would prevent a mass shooting. Is there some database of faceprints of known future mass shooters?
I suspect that people are drawn to "The Bourne Supremacy" look and feel of their command center, just behind the parent-teacher conference whiteboard.
I do want to draw attention to Destin from Smarter Every Day's gun detection system[0], where he even mentions part of the above situation. Maybe this could accelerate police response and automate it, but this honestly sounds like an over engineered system.
I mean, it is not like at any school there is (or can be) a sort of SWAT team on alert and capable of an intervention in - say - 5 minutes.
And - cannot say on the specific 2018 shooting at that school - from what I understand the typical sequence is:
1) someone enters a school (or church, etc.) with a gun
2) as soon as a suitable target is within firing range, the shooting starts
and there is no more than 1, 2 or 3 minutes between #1 and #2 above, if the detection of the gun happens before (let's say outside when the shooter is approaching the building) you can maybe have a couple minutes more time.
And - even if you actually had this fast intervention armed team ready - can you only imagine the risks of false alarms/incidents with perfectly innocent people, it cannot possibly work.
Besides, there is no real need for facial recognition in this scenario, i.e. anyone, no matter who he/she is, should be detected if carrying a gun.
I would guess something between 10 and 15 minutes at the very least (not counting the hypothesis of a local presence of policemen or military personnel).
If what you want to prevent is that kind of unconditional mass shooting this doen't seem to be effective, by the time the police (early alerted by the automatic system) arrives, the shooter will very likely have fired all the ammunitions and already killed all the people that happened to be at range.
It could also sound a warning for the school so that students could preemptively run or hide. Imagine this system hooked into the lighting and lights in the immediate vicinity of the shooter could turn a dangerous red, yellow nearby and green along routes that would escape the shooter. Students could evacuate more easily if they could trust that system. Another innovation might be strobe lights that could focus on the attacker to disorient and distract him.
I think there's a huge responsibility to such technology and much danger to it, but there are potential uses.
Yes, but the reaction time is "after" the alarm, the point I was trying to make is that this surveillance/weapon recognition approach may trigger the alarm earlier (of a couple minutes, maybe 5) but if the reaction time is in the tens of minutes range, it would make no particular difference in the actual outcome as all the shooting will happen anyway before the police can arrive.
About the trust on the system, it's tricky business, as always, the amount of false positives need to be reduced to 0 or next to 0 to gain trust, and it is very unlikely, besides the errors of the system, like innocent maintenance people - say - plumbers or electricians carrying pipes or tubes, or some power drills, etc. (let alone carpenters with nail guns) I would bet that the new rage among the school kids would be finding perfectly legal items (let's say for the sake of reasoning antennas, cactuses, umbrellas) that would triggger the alarm, just for the fun of it.
Good point about alternative "distracting" means (lighting colours, strobe lights etc.) but again, when (hypothetically) such a system will be installed I doubt that it can avoid daily, weekly or monthly false alarms and after a given number of them happens, usually the system is disabled or - if possible - set to very low (ineffective in practice) sensitiveness.
The problem is black swan events[0] are really hard to predict
https://www.theverge.com/2017/11/2/16597276/google-ai-image-...
Its dark, but I can envision someone giving a version of this bad boy to someone they would like to prank. Kids will be kids...
There is no need for this razzle-dazzle anyhow.
The more ethical solution is to add a beacon to the id card or bracelet or whatever you need to move around the school, like an EZ-Pass. You can use a shielding bag or turn them off or leave them at home when you aren't at school.
You can use human presence sensors to identify people in general and where they aren't. When theres another tragic shooting, you can failsafe open the maglocks or access controls except where people are taking shelter, and either track the person directly or track where there is a person without a beacon.
Then, give cops the regular camera feed for situational awareness.
“The mission creep issue is a real concern when you initially build out a system to find that one person who’s been suspended and is incredibly dangerous, and all of a sudden you’ve enrolled all student photos and can track them wherever they go[.]”
I don't personally find the rationale persuasive, I'm just answering the question how it could possibly be considered useful.
Still you should fully expect to be tracked in higher-security areas, like banks or airport security zones. Large expansive stores, likely too. If not by robots, then by human guards.
Privacy in a public place is the classical security through obscurity. Expect it to erode; preferably never assume it. If you read the famous cyberpunk novels as fantasy, it's now time to rethink them as a rather sober and real warning.
When I was in China, I saw how many cameras there are everywhere. And how Chinese officials pass by high-speed trains with cameras.
For Chinese citizens, the state has trained facial recognition on millions of faces. They use it to detect people passing roads in red lights (they show names and faces in big screens)
But, I wonder what is the error rates for non-Chinese. Without enough samples I can be identified as any other person similar to me in skin color. Can I mistakenly be identified as some "person of interest" for the Chinese government? That was an scary though.
I see the usefulness of this technologies, but "pro-active" policing has high rates of false positives and many innocent people will be harassed or even worse just because a machine failure. All that without taking into account when the machines are right at persecuting opositors in dictatorial regimes.
Cameras deployed all throughout schools with AI watching will inevitably lead to errors in misidentification, capture of information that isn’t pertinent, or misuse. At best the system errors too much, occasionally acting as a useful system to stop known trespassers. At worst, you send the police chasing down the wrong person, or you accidentally record something you wish you didn’t and the video is leaked.
I’m not quite sure how deployment of this system would lead to intended results, like stopping school shooters. It reminds me a lot of the Brussels airport that suggested having human “fire spotters” strategically stationed around the building as a substitute for a working and functional fire alarm system.
2. Could you please elaborate your first examples? I can’t see how that could work without (1)
https://www.theregister.com/2021/07/01/seoul_ai_bridge_rescu...
https://techcrunch.com/2017/11/27/facebook-ai-suicide-preven...
IMHO, The only reasonable use for this kind of biometric fingerprinting is creating biometric models for validating key public figures against deep fake videos. This approach uses motion biometrics, so it's not even face-ID, but it is a form of "compare vector against existing database" so I count it. This is also super specialized and only applies to figures with hours of video to train on.
Also my opinion, any system which does identification of the general population is going to inevitably be abused.
Face detection, eye tracking, emotional and sentiment analysis, etc, has many uses in the videoconferencing space. The vector data only exists in the scope of a session.
The part that makes it nasty and authoritarian is any time you build a database/index coupling identities (even pseudonymous) with features, whether it be face, gait, voice, from CNNs, HoG, or even just browser fingerprints, this is the dangerous bit, and where laws should focus their effort.
Re-id in public (where you vectorize and fuse tracks starting with first glimpse) is a grey area and probably should be very scope-limited, e.g. active warrant, specifically signed by a judge to authorize use looking for only that person. Even then, kinda icky.
Disclaimer: I work in the video emotional recognition field
Like the upside here is if Mr. Rogers is in power you can unlock your iPhone and (if everything works perfectly... good luck?) you can get on an airplane ~1 minutes faster. But if Hitler is in power you have modern Phrenology (which we're already seeing).
What am I missing here? What is the great use case that warrants this?
https://www.taylorwessing.com/en/insights-and-events/insight...
I hate to be pessimistic, but I don't see how it doesn't become ubiquitous very soon, with all the downsides ubiquitous surveillance are going to lead to.
I don't care to speculate about outcomes. Optimistic or pessimistic, it always presumes a lack of agency. I just can't bring myself comment like a bystander or a fortune teller on something that is still ongoing and that I, like all people here, am involved in.
A lot of it is just buzz and network effect with little long term thought going into it.
Do these providers subsidise costs to normalise the use of their products in a sector, does the bogeyman of school shootings get rolled out for an easy sign off, or a bit of both?
It's absurd but once you say "this can prevent a school shooting", who can say no to that?
We are heading into dangerous territory with this, similar to the "social credit score" system in place in other countries.
If this gets tied into other databases, such as criminal databases, I can see a scenario where people are ranked in real-time as potentially undesirable. Criminal convictions from years ago can now follow people around, putting security guards and store employees on "high alert" to watch people carefully, or simply deny them entrance.
well thats ominous.
>> This Manual for a Popular Facial Recognition Tool Shows Just How Much the Software Tracks People
Ugh. After seeing that I'm really not sure this is an article I want to read. Why do people write titles like that? It just makes it very hard to trust anything written in an article.
Thanks type0 for making the title less awful, btw.
If so, how do you explain communist China's extensive surveillance / camera system? How about socialist countries in Europe?