Not sure about this take. You need data at the end of the day, to build good AI models.
These days it feels like anyone's public takes are more or less just a means to promote their own cause.
And I say this as a happy and long-time Signal user.
> These are ultimately surveillance systems that are being marketed to those who have power over us generally: our employers, governments, border control, etc., to make determinations and predictions that will shape our access to resources and opportunities.
And hey, the more arbitrary or even false something is, the more obedience you can demonstrate by going along with it just because computer said no, so that's another "bonus".
Here's a chilling real-life example:
https://www.politico.eu/article/dutch-scandal-serves-as-a-wa...
> In 2019 it was revealed that the Dutch tax authorities had used a self-learning algorithm to create risk profiles in an effort to spot child care benefits fraud.
> Authorities penalized families over a mere suspicion of fraud based on the system’s risk indicators. Tens of thousands of families — often with lower incomes or belonging to ethnic minorities — were pushed into poverty because of exorbitant debts to the tax agency. Some victims committed suicide. More than a thousand children were taken into foster care.
Let that sink in.
https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scand...
> Let that sink in.
The need for governance is important. In this case, an automated (imperfect) labeling system was the arbiter for classification. That should never happen.
Don't do that part, nothing to do with AI, it's about humans being lazy
We're still at the step of raising awareness, and that is achieved by making it clear that the situation and the trends are really fucking bad, no ifs and buts and maybes about it.
This tech as a body pillow stuff is really grating to me. Tech doesn't care about you, tech doesn't know you, tech doesn't need you -- other people (and arguably animals) do, right now.
Although, I would say she is not the most informed person in surveillance task in AI. Another surveillance field in AI is the "person re-identification" task. It aims to determine if two pictures of a person from surveillance cameras depict the same person or not. This is my field of research and you have a tons of papers each year in big AI venues and also big company in the author (alibaba, tencent, mostly big chinese tech companies, I saw one paper with Google once on a related task).
Also, I would like to highlight something. She said we need some people to annotate the datasets. For this surveillance task, actually, we have a bunch of methods that aims to less rely on human annotations : LUPerson and LUPerson-NL datasets and respective methods (CVPR 21 and 22) for example.
Finally, I would like to conclude that I mostly agree with her. A bit of nuance would be to see the interaction with the nature of data and the surveillance capabilities. AI on camera data are obviously surveillance. When it comes more blur it's what "surveillance" is for "code taken from the internet to train Github copilot method"? Or what is the surveillance capability from database of digital arts? Those questions seem to be much more relevant and hard in regard of the use of data for surveillance. AI is just a name of a method that we use to process big data at scale in this context.
> “You know, you walk past a facial recognition camera that’s instrumented with pseudo-scientific emotion recognition, and it produces data about you, right or wrong, that says ‘you are happy, you are sad, you have a bad character, you’re a liar, whatever.’
While I'm not aware of any technology that claims to classify faces as liars the rest of the statement is broadly true. And if you wanted to cross-check a face against a database of known liars, the entire statement is true.
According to https://www.cyberlink.com/faceme/insights/articles/363/smart... "It can be difficult to discern how consumers feel about a retail experience, or about the products they are buying or avoiding. Facial recognition can help with this by assessing and charting emotions: happy, sad, angry, surprised, indifferent, etc."
It also says "By cross-checking everyone who enters the store against a database of identified shoplifters and unruly customers, facial recognition can detect and deter repeat offenders" and "Using facial recognition, retailers can know precisely when an employee arrives, takes breaks, and leaves for the day"
Personally a lot of this technology seems pretty doubtful to me - don't shoplifters wear hats to keep their faces off CCTV? And every 'emotion recognition' system demo I've seen has used faces with emotions so ridiculously exaggerated they look like they belong in a youtube thumbnail.
I don't remember the name, but a defense company demonstrated a facial recognition software which identified the face while the person is running in the dark, in very low light (IIRC akin to a poorly lit harbor-like environment). The camera was shooting from the side, so the system saw the face for a couple of frames.
So, a small camera mounted to the entrance turnstile of a supermarket with a fairly lit environment can scan almost everyone going in. Are you wearing a balaclava in the store? It's also suspicious in most cases.
Well, I don't think the "bad character" thing is there either. The problem is with these statements is they are lies that contain some truth. Cameras exist. AI to try and detect basic facial expressions exists. Mapping from basic facial expressions to underlying emotions exists. But I doubt many people are trying this, except as a way to justify a marketing budget (we want to push our smile numbers from 8% to 8.2% - give me $5m for my marketing department). And I have never heard of anything connecting "liar" or "bad character" to AI detection.
> And if you wanted to cross-check a face against a database of known liars, the entire statement is true.
Filling in the gaps with technically true statements should not justify the emotional change in you this sort of article seems to be trying to engender. It's pretty much not true, and the bits that might be true and honest - e.g. emotion detection - doesn't work well, and is probably not deployed anywhere.
As for using facial recognition to replace time cards, and replace photos of banned customers - sure? Might work?
That's not the thing I was talking about.
> Probably a little bit before it’s actually capable of doing it reliably.
I think you said that already?
And in fact you yourself already hedged that this probably is happening, “ But I doubt many people are trying this, except as a way to justify a marketing budget.” Aka you believe that at least some people are already attempting this.
This is getting painfully repetitious, but that was about facial recognition, not deciding if someone's a liar or is of bad character via AI.
> Identifying protesters isn't the same as an AI filming you and deciding if you are of bad character.
Same comment.
Am I reading this right now?
It may be. Not that it will be.
> we agree AI-based polygraphing will exist
We didn't mention lie detecting, but no - I was saying liar detection, which was the thing from the article, also sounds like fear-mongering.
> you don’t think AI-identify-bad-intentions-button exists currently
Yes.
> therefore we are supposed to doubt that AI is a major surveillance hazard
No, that's a category error. "Major" and "hazard" are far too vague to decide upon. Will AI be used in surveillance in some capacity? Of course - it likely has been for decades, depending on how you define AI. But implying that you'll walk into a shop and be classified as a liar based on a video feed is not right. I understand websites need emotion for clicks, but I don't think it's right to do it this way.
Ofc you "can" - in principle.
Is any of this currently being done, however?
Companies aren't obligated to disclose the training process. So, we can't tell. Until, in the matter of months, it is available, accessible and almost impossible to regulate. And every company will start using it to not fall behind.
There's a _long_ list of unlawful things that are technically possible, sometimes trivial even. Only _because_ they are possible in the first place, are there laws against them. This process seems slow compared to a human lifespan, but lots of terrible things people used to do got outlawed eventually.
There are three reasons:
- once it's widespread - it's a standard (targeted ads, cookie tracking, SEO)
- we'd need a lot of infrastructure to track all the data recorded, where it goes, how it is processed, etc. etc.. And if a big company is caught.. they'll pay fines for being caught and let away to continue the same shady practices, as we have seen with privacy scandals at Facebook.
> ...but lots of terrible things people used to do got outlawed eventually.
That's the third reason - surveillance is evil and bad, but it will never be advertised as that. It will be something good and convenient with a surveillance as a side effect, similar to social media and smart appliances.
Maybe, I am just too pessimistic and you're right with: 'This process seems slow compared to a human lifespan'.
You just need to punish the people using the data. If you can then track down where they got the data from, that's a bonus. And if you can track down where those second ones got their data from, that's an extra bonus. But everything after the people using it is optional.
I know in America this would be painted as an attack on the Holy Minority and therefore no one will be allowed to discuss it without being branded a blasphemer, so I'm curious what people on here think.
I feel like businesses should have this right, but I worry about slippery slopes. It might start with shoplifters, and 20 years later when 99% of stores are owned by the same 3 megacorps, people could get banned for wrongthink and for shaping the political views of the masses at large. Kinda like what they do now on social media.
Hadn't there been similar systems for phone conversations, based on voice stress analysis? I totally expect this to move to AI based systems, including audio and visual data, if it hasn't been done already. (Also, if a person has been tagged as "liar" – truthfully or not – in any customer history file, this data will be available, as soon as a face is matched.)
Not a technology, but the "science" has a name: https://en.wikipedia.org/wiki/Phrenology?useskin=timeless#Ap...
There are (and have been for many years of development, although I do not know about deployment) methods for identifying people as criminals.
There's no science behind it but that never stopped someone from selling software to the police, high tech polygraphs are probably out there. Closest thing I've found in production is more "nervous person recognition" used at airports for triggering secondary screening:
>> the Department of Homeland Security (DHS) Science and Technology Directorate (S&T) seeks to improve DHS’s ability to quickly and objectively screen individuals for malintent. FAST combines cutting-edge behavioral and physiological science with deception detection theory and state-of-the-art sensor technologies. [0]
I don't think her hot take is mere self-promotion, but rejection that building "good models" is necessary or helpful to average people, maybe there's some benefits but it increases the power of the state more than the individual (old debate in computing)
[0] https://www.dhs.gov/sites/default/files/publications/Future%...
I don't think she is saying that AI is predicated on surveillance as a source of training data (although that does happen in a few cases). She seems to be saying that surveillance is the primary use case for AI.
Actually I don't think that's true, consider manufacturing robots. However, mass surveillance, a societal scourge and urgent threat to democracy, does seem to be greatly enhanced by AI, so I think this is a bit of worry take.
> These days it feels like anyone's public takes are more or less just a means to promote their own cause.
Wasn't it always true? Trivially all people advocate their beliefs, with a few marginal exceptions like the guy who reads the weather report.
We're long way from making fully AI-driven robots. But much closer to other uses.
Our current world is very data-centric: companies have been collecting, selling, sharing and stashing metadata with websites, social media and smartphones for the last decade. Now, language models will help them process it. I have one question: for what cause? Surely, for the good of humanity. That's what the companies exist for, right?
It's not like we need laws to force companies to not screw us over, or in America they need lobbyists to force politicians to force companies to not screw them over. Currently, AI is absolutely unregulated wild lawless west, and it's frightening. And not because of bad actors, but because of the 'pretending to be good' ones.
> Wasn't it always true? Trivially all people advocate their beliefs..
You would be surprised, but people need constant reminders of that. Or they will start treating mega-corporations as privacy-respecting, consumer-oriented, eco-friendly or any other bullshit their PR-team trying to push today or will try tomorrow.
Someone else asked that question in this same thread [0].
It's a good question.
The purposelessness of much technology is a problem Neil Postman addressed very robustly in "Technopoly" [1]. To date I have not heard any sane responses to his "Six Questions Regarding Technology" [2].
> not because of bad actors, but because of the 'pretending to be good' ones.
When you don't have any reason at all for doing something, and you're challenged, one tends to use imagination to dream up a plausible "good" reason.
[0] https://news.ycombinator.com/item?id=37656891
https://www.technologyreview.com/2020/03/13/905323/ai-lie-de...
Therefore…
AI is fundamentally a surveillance technology.
SOURCE:
https://arstechnica.com/tech-policy/2009/09/your-secrets-liv...