AI is fundamentally ‘a surveillance technology’
techcrunch.com
techcrunch.com
The thesis of the article should have been articulated "AI is disproportionately used as a surveillance technology owning to its computational requirements, the current state of industrial-user relations and the inability for legal systems to respond fast enough to protect individuals"
Edit 1: I don't know how to make that more consise without losing the meat of the argument
Edit 2: the emphasis should be directed to (yet again), industrial-user relations. We really need the equivalent to unions but for representating the rights of users. We're in an unchecked second guilded age and it's wreaking havoc on not just individuals but future businesses. These ill-gotten gains are going to devastate the tech sector, when it inevitably gets corrected in 5-x years
You mean like https://eff.org?
Is it really inability when not doing so benefits you and your lobbying friends?
Technology develops as the result of focused application of capital and shaping the evolution of that technology to serve the interest of that capital. The driving force behind AI -- when ranked according to impact and significance -- is being funded by surveillance capitalists. In that sense one can say "AI is fundamentally a surveillance technology".
AI and Capitalism are a toxic mix, in my opinion.
The only worse combinations would be AI+Communism, or (arguably what we have) AI+Facism
Hyperbole really serves no purpose when it's so extreme. Instead of talking about the merits of "how can this technology be abused?" you're putting everyone else on the defensive, having to say "I'm not abusing it".
Like,
> AI is a way, I think, to entrench and expand the surveillance business model,” she said. “The Venn diagram is a circle.”
No it isn't.
If you want to say "facial recognition is a surveillance technology" OK. I find it deeply ironic that you then point out a use case where facial recognition is not used for surveillance and is actually protecting people (by blurring out faces) when you've just said that "The Venn diagram is a circle" but ok, at least the hyperbole is not so absurd if we limit it there. But then you just look like you're saying "AI" for headlines because yeah, we've been talking about facial recognition being surveillance technology for like 30 years or more...
This article should have been reframed as something more like "The surveillance impacts of advanced AI techniques on Facial Recognition", to throw out a half assed attempt at a title/ rephrasing. The hyperbole should have been applied exclusively to surveillance applications, which is hardly polarizing by contrast, and the interesting bit would have been to hear about how AI factors in. Instead this was a puff piece with no real technical content, nothing interesting, and is practically a time piece from the early 2000s surveillance discussion with "AI" slapped onto the headline for clicks.
Which is irrelevant if tens of thousands of other AIs, from entities with 4-5 orders of magnitude more times the resources and power over the economy/society/country than you use them for surveillance and to optimize to your detriment.
The claim was never was about the logical or practical impossibility of AI to also be used otherwise.
And facial recognition is just the tip of the iceberg of AI uses in surveillance.
Nor is "blurring out faces" much consolation. What's blurred in, say, a public "Street View" service doesn't mean is also blurred on the same company's raw captured data. And the example wasn't about such cases, but about surveillance cameras anyway - which don't blur anything, just use the AI to identify targets.
We should instead of arguing the point (and especially, the point at large), instead of nitpicking on the speaker's expression, while onstage, during a conference panel talk - as if those would be the best formulated and exhaustive.
Well, yeah, it actually was the claim. It's literally the headline and I even quoted directly from the article where they say that there is literally 100% overlap between AI and surveillance. If you want to say "that was hyperbole", congrats, you've arrived at the premise of my comment.
>If you want to say "that was hyperbole", congrats, you've arrived at the premise of my comment.
The premise of the comment was more like saying a headline like "WWII is deadly for Europe" is BS because Switzerland was just fine.
Your poor take is so unrelated to the article that it suggests you didn't even bothered to read the article you're trying to criticize. It completely fails to even take into account the very basic premise presented in the article. It adds nothing to the discussion, and it reads as uninformed contrarian jabs.
Please refrain from adding noise to an otherwise interesting discussion.
How about when you comment you follow HN guidelines? https://news.ycombinator.com/newsguidelines.html
> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".
I don't think you have. There is nothing in your comment that has any semblance of a relationship to the point made in the article you're supposedly commenting on.
Here's a direct quote from the article:
> “And the use of AI is also surveillant, right?” she continued. “You know, you walk past a facial recognition camera that’s instrumented with pseudo-scientific emotion recognition, and it produces data about you, right or wrong, that says ‘you are happy, you are sad, you have a bad character, you’re a liar, whatever.’ These are ultimately surveillance systems that are being marketed to those who have power over us generally: our employers, governments, border control, etc., to make determinations and predictions that will shape our access to resources and opportunities.”
And here you are, posting inane non-sequitur comments on how it's possible to train models locally?
No, you clearly did not read the article.
You mean like me directly quoting parts of the article from both the beginning and end? Like me pointing out how the article talks about surveillance and then attempts to clumsily tie it to AI?
Fuck off lol
> How about when you comment you follow HN guidelines? https://news.ycombinator.com/newsguidelines.html
That's... not actually stated outright in the article, so I get the confusion. But it seems to me like it's the only logical explanation for what is stated.
This reads like parody.
Let's say that states and police forces alone (for the sake of the argument) used to do surveillance.
For the last 2 decades, Google, Facebook, and co ALSO collect data in 1000x the times the data breadth and detail than even Stazi was capable to collect, including collecting data covertly, and abuses it for all kinds of purposes, both aggregated and targeted.
Not to mention that, beyond the surveillance they do for their private interests and business, even the "states and goverments" vs "Google and Facebook" are not even remotely separate from surveillance by states and police forces, cooperating, sharing data, or even ordered to give surveillance data, to both.
> Not to mention that, beyond the surveillance they do for their private interests and business, even the "states and goverments" vs "Google and Facebook" are not even remotely separate from surveillance by states and police forces, cooperating, sharing data, or even ordered to give surveillance data, to both.
I'll grant you that, but I wouldn't call that their business. They probably would do without if they could.
https://en.wikipedia.org/wiki/Privacy_concerns_regarding_Goo...
https://fortune.com/2019/01/28/google-iab-sensitive-profiles...
I will have to disagree here, that is not the dictionary definition of the word. So attempting to call out a serious abuse of language while doing the same thing is definitely an interesting take.
Just like governments can abuse their position, so do corporations. Google absolutely has surveilled society at large and been caught and fined for doing so.
This is a non-inclusive list of reasons why and whom may perform the act "especially by the police or army", "for the purposes of catching criminals"
There are many acts that are defined by 'who' does it. The act of surveillance is not one of them. For example a private investigator can surveil you. A business can do the same by intentional action (for example selling hidden tracking devices to ensure they can find you if you are kidnapped) or unexpected action (your cell phone reports back to a server every 30 seconds of an exact GPS location of where you are).
> an economic system in which the product for sale is personal data collected on the internet, especially from search engines (= programs used to find information on the internet) and social media platforms (= websites and programs that allow people to communicate and share information on the internet)
I think this is the context folks are reaching for: it's not just a data-driven business model, it's driven by people's personal data in particular. As far as sharing willingly, it's _incredibly_ hard to buy a smartphone and prevent it from collecting your personal information (like location, movement patterns, app usage) without removing much of the functionality that makes it useful.
And once that data is collected, it's open season. I just ran across this piece from today where the government literally goes directly to private data brokers because the NSA takes too long: https://www.404media.co/us-counterintel-buys-netflow-data-te...
That particular case is about tracking VPNs (so not directly related to Google's data collection), but is a good example of how we can limit government's reach, but as long as they can buy data from private firms, there's a loophole in the 4th amendment, likely rooted in the third-party doctrine.
And the idea that AI "requires the surveillance business model" is demonstrably false. Thousands of researchers over the last 7 decades have contributed to the development of the collection of technologies we call AI without that business model.
However this is Meredith Whittaker so we can assume she knows that. So yeah, those are provocative statements made for rhetorical effect to emphasize that power will use technology to do bad things and enshrine and enhance its power.
I mean at some point the conflation itself is rather fair game, given that such massive swaths of the modern Internet apply it directly. The most predominant search engine sells ads. The most predominant social media sites, all of them, run on paid placement of content (ads). In fact, every social media site with even a little traffic besides perhaps Mastodon does (I honestly don't know I've never used it). Twitter/X is trying to get off advertising as it's revenue source and because of that is worth a 10th of what it was when Elon bought it less than a year ago. Amazon you could make an argument isn't based on ads (entirely) but it's also the largest retail website in the history of man and that's probably the only reason, and also, it's largest cash cow at the moment is AWS which is absolutely shoulders deep in ads, ad services, serving ads, and ad targeting.
Podcasts? Ads. Listicles? Ads. Substacks? Ads. Need a driver file to make an ancient videogame work? You'll find a website that hosts it, alongside some ads. Wanna download some ROMs for SNES games you definitely own? Ads. Pirating some pornography with surprisingly good production value that's riffing on Pirates of the Caribbean? Sounds fun, also, the website you get the torrent link from has ads.
And like, you can get into the semantics of there's no other way to fund a website, they didn't all choose it, some don't use surveillance to serve their ads, and all the rest, and to that I say, you have much more trust in corporations than I do. Salesforce would sell my grandmother into sex slavery if they could make $5 doing so so you'll have to forgive me if I just don't necessarily trust the internet ad machine when it says pinky promise we aren't exploiting your data, especially since every time they have a data leak incident, oops, there's all my data again. Aw shucks aw geez.
User data is practically the only currency on the internet anymore, and user attention in turn is the only resource being mined by social media. Of course it's for advertising. What else would it be for? What else could it be for?
> However this is Meredith Whittaker so we can assume she knows that. So yeah, those are provocative statements made for rhetorical effect to emphasize that power will use technology to do bad things and enshrine and enhance its power.
I mean that seems like something worth emphasizing? Ad tech has had a massive influence over the modern internet space, which is why it's so fucking bad. Basically every terrible decision you lament on a daily basis, every dark pattern, every annoying thing, can in some way or another be traced to ad tech. All that shit about when you talk about needing a new hose near your phone and suddenly you have ads for hoses, that's ad tech. The entire pivot to video fiasco! Facebook wanted to stream video and bullshitted publishers and media orgs with cooked numbers that it was "what people wanted" even though demonstrably no they fucking did not, do not, and never will want that, it's what ad tech wanted and therefore it was sacrosanct, right up until it killed literally hundreds if not thousands of websites. Except even that wasn't enough, because ad tech still wants it, and so we have video platforms left and right because these companies do not work for us, they work for advertisers.
Like no it isn't everything, no single entity could be that bad. I'm just saying if all of ad tech was Thanos-snapped out of existence tomorrow, the Internet would be incredibly different, better in a lot of ways, so I feel the conflation is fair game.
Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4543807
I suggest to read (at least) the abstract (too long to be copied here) to get some more proper idea of the debated area.
Not sure about this take. You need data at the end of the day, to build good AI models.
These days it feels like anyone's public takes are more or less just a means to promote their own cause.
And I say this as a happy and long-time Signal user.
> These are ultimately surveillance systems that are being marketed to those who have power over us generally: our employers, governments, border control, etc., to make determinations and predictions that will shape our access to resources and opportunities.
And hey, the more arbitrary or even false something is, the more obedience you can demonstrate by going along with it just because computer said no, so that's another "bonus".
Here's a chilling real-life example:
https://www.politico.eu/article/dutch-scandal-serves-as-a-wa...
> In 2019 it was revealed that the Dutch tax authorities had used a self-learning algorithm to create risk profiles in an effort to spot child care benefits fraud.
> Authorities penalized families over a mere suspicion of fraud based on the system’s risk indicators. Tens of thousands of families — often with lower incomes or belonging to ethnic minorities — were pushed into poverty because of exorbitant debts to the tax agency. Some victims committed suicide. More than a thousand children were taken into foster care.
Let that sink in.
https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scand...
> Let that sink in.
The need for governance is important. In this case, an automated (imperfect) labeling system was the arbiter for classification. That should never happen.
Don't do that part, nothing to do with AI, it's about humans being lazy
We're still at the step of raising awareness, and that is achieved by making it clear that the situation and the trends are really fucking bad, no ifs and buts and maybes about it.
This tech as a body pillow stuff is really grating to me. Tech doesn't care about you, tech doesn't know you, tech doesn't need you -- other people (and arguably animals) do, right now.
Although, I would say she is not the most informed person in surveillance task in AI. Another surveillance field in AI is the "person re-identification" task. It aims to determine if two pictures of a person from surveillance cameras depict the same person or not. This is my field of research and you have a tons of papers each year in big AI venues and also big company in the author (alibaba, tencent, mostly big chinese tech companies, I saw one paper with Google once on a related task).
Also, I would like to highlight something. She said we need some people to annotate the datasets. For this surveillance task, actually, we have a bunch of methods that aims to less rely on human annotations : LUPerson and LUPerson-NL datasets and respective methods (CVPR 21 and 22) for example.
Finally, I would like to conclude that I mostly agree with her. A bit of nuance would be to see the interaction with the nature of data and the surveillance capabilities. AI on camera data are obviously surveillance. When it comes more blur it's what "surveillance" is for "code taken from the internet to train Github copilot method"? Or what is the surveillance capability from database of digital arts? Those questions seem to be much more relevant and hard in regard of the use of data for surveillance. AI is just a name of a method that we use to process big data at scale in this context.
> “You know, you walk past a facial recognition camera that’s instrumented with pseudo-scientific emotion recognition, and it produces data about you, right or wrong, that says ‘you are happy, you are sad, you have a bad character, you’re a liar, whatever.’
While I'm not aware of any technology that claims to classify faces as liars the rest of the statement is broadly true. And if you wanted to cross-check a face against a database of known liars, the entire statement is true.
According to https://www.cyberlink.com/faceme/insights/articles/363/smart... "It can be difficult to discern how consumers feel about a retail experience, or about the products they are buying or avoiding. Facial recognition can help with this by assessing and charting emotions: happy, sad, angry, surprised, indifferent, etc."
It also says "By cross-checking everyone who enters the store against a database of identified shoplifters and unruly customers, facial recognition can detect and deter repeat offenders" and "Using facial recognition, retailers can know precisely when an employee arrives, takes breaks, and leaves for the day"
Personally a lot of this technology seems pretty doubtful to me - don't shoplifters wear hats to keep their faces off CCTV? And every 'emotion recognition' system demo I've seen has used faces with emotions so ridiculously exaggerated they look like they belong in a youtube thumbnail.
I don't remember the name, but a defense company demonstrated a facial recognition software which identified the face while the person is running in the dark, in very low light (IIRC akin to a poorly lit harbor-like environment). The camera was shooting from the side, so the system saw the face for a couple of frames.
So, a small camera mounted to the entrance turnstile of a supermarket with a fairly lit environment can scan almost everyone going in. Are you wearing a balaclava in the store? It's also suspicious in most cases.
Well, I don't think the "bad character" thing is there either. The problem is with these statements is they are lies that contain some truth. Cameras exist. AI to try and detect basic facial expressions exists. Mapping from basic facial expressions to underlying emotions exists. But I doubt many people are trying this, except as a way to justify a marketing budget (we want to push our smile numbers from 8% to 8.2% - give me $5m for my marketing department). And I have never heard of anything connecting "liar" or "bad character" to AI detection.
> And if you wanted to cross-check a face against a database of known liars, the entire statement is true.
Filling in the gaps with technically true statements should not justify the emotional change in you this sort of article seems to be trying to engender. It's pretty much not true, and the bits that might be true and honest - e.g. emotion detection - doesn't work well, and is probably not deployed anywhere.
As for using facial recognition to replace time cards, and replace photos of banned customers - sure? Might work?
That's not the thing I was talking about.
> Probably a little bit before it’s actually capable of doing it reliably.
I think you said that already?
And in fact you yourself already hedged that this probably is happening, “ But I doubt many people are trying this, except as a way to justify a marketing budget.” Aka you believe that at least some people are already attempting this.
This is getting painfully repetitious, but that was about facial recognition, not deciding if someone's a liar or is of bad character via AI.
> Identifying protesters isn't the same as an AI filming you and deciding if you are of bad character.
Same comment.
Am I reading this right now?
It may be. Not that it will be.
> we agree AI-based polygraphing will exist
We didn't mention lie detecting, but no - I was saying liar detection, which was the thing from the article, also sounds like fear-mongering.
> you don’t think AI-identify-bad-intentions-button exists currently
Yes.
> therefore we are supposed to doubt that AI is a major surveillance hazard
No, that's a category error. "Major" and "hazard" are far too vague to decide upon. Will AI be used in surveillance in some capacity? Of course - it likely has been for decades, depending on how you define AI. But implying that you'll walk into a shop and be classified as a liar based on a video feed is not right. I understand websites need emotion for clicks, but I don't think it's right to do it this way.
Ofc you "can" - in principle.
Is any of this currently being done, however?
Companies aren't obligated to disclose the training process. So, we can't tell. Until, in the matter of months, it is available, accessible and almost impossible to regulate. And every company will start using it to not fall behind.
There's a _long_ list of unlawful things that are technically possible, sometimes trivial even. Only _because_ they are possible in the first place, are there laws against them. This process seems slow compared to a human lifespan, but lots of terrible things people used to do got outlawed eventually.
There are three reasons:
- once it's widespread - it's a standard (targeted ads, cookie tracking, SEO)
- we'd need a lot of infrastructure to track all the data recorded, where it goes, how it is processed, etc. etc.. And if a big company is caught.. they'll pay fines for being caught and let away to continue the same shady practices, as we have seen with privacy scandals at Facebook.
> ...but lots of terrible things people used to do got outlawed eventually.
That's the third reason - surveillance is evil and bad, but it will never be advertised as that. It will be something good and convenient with a surveillance as a side effect, similar to social media and smart appliances.
Maybe, I am just too pessimistic and you're right with: 'This process seems slow compared to a human lifespan'.
You just need to punish the people using the data. If you can then track down where they got the data from, that's a bonus. And if you can track down where those second ones got their data from, that's an extra bonus. But everything after the people using it is optional.
I know in America this would be painted as an attack on the Holy Minority and therefore no one will be allowed to discuss it without being branded a blasphemer, so I'm curious what people on here think.
I feel like businesses should have this right, but I worry about slippery slopes. It might start with shoplifters, and 20 years later when 99% of stores are owned by the same 3 megacorps, people could get banned for wrongthink and for shaping the political views of the masses at large. Kinda like what they do now on social media.
Hadn't there been similar systems for phone conversations, based on voice stress analysis? I totally expect this to move to AI based systems, including audio and visual data, if it hasn't been done already. (Also, if a person has been tagged as "liar" – truthfully or not – in any customer history file, this data will be available, as soon as a face is matched.)
Not a technology, but the "science" has a name: https://en.wikipedia.org/wiki/Phrenology?useskin=timeless#Ap...
There are (and have been for many years of development, although I do not know about deployment) methods for identifying people as criminals.
There's no science behind it but that never stopped someone from selling software to the police, high tech polygraphs are probably out there. Closest thing I've found in production is more "nervous person recognition" used at airports for triggering secondary screening:
>> the Department of Homeland Security (DHS) Science and Technology Directorate (S&T) seeks to improve DHS’s ability to quickly and objectively screen individuals for malintent. FAST combines cutting-edge behavioral and physiological science with deception detection theory and state-of-the-art sensor technologies. [0]
I don't think her hot take is mere self-promotion, but rejection that building "good models" is necessary or helpful to average people, maybe there's some benefits but it increases the power of the state more than the individual (old debate in computing)
[0] https://www.dhs.gov/sites/default/files/publications/Future%...
I don't think she is saying that AI is predicated on surveillance as a source of training data (although that does happen in a few cases). She seems to be saying that surveillance is the primary use case for AI.
Actually I don't think that's true, consider manufacturing robots. However, mass surveillance, a societal scourge and urgent threat to democracy, does seem to be greatly enhanced by AI, so I think this is a bit of worry take.
> These days it feels like anyone's public takes are more or less just a means to promote their own cause.
Wasn't it always true? Trivially all people advocate their beliefs, with a few marginal exceptions like the guy who reads the weather report.
We're long way from making fully AI-driven robots. But much closer to other uses.
Our current world is very data-centric: companies have been collecting, selling, sharing and stashing metadata with websites, social media and smartphones for the last decade. Now, language models will help them process it. I have one question: for what cause? Surely, for the good of humanity. That's what the companies exist for, right?
It's not like we need laws to force companies to not screw us over, or in America they need lobbyists to force politicians to force companies to not screw them over. Currently, AI is absolutely unregulated wild lawless west, and it's frightening. And not because of bad actors, but because of the 'pretending to be good' ones.
> Wasn't it always true? Trivially all people advocate their beliefs..
You would be surprised, but people need constant reminders of that. Or they will start treating mega-corporations as privacy-respecting, consumer-oriented, eco-friendly or any other bullshit their PR-team trying to push today or will try tomorrow.
Someone else asked that question in this same thread [0].
It's a good question.
The purposelessness of much technology is a problem Neil Postman addressed very robustly in "Technopoly" [1]. To date I have not heard any sane responses to his "Six Questions Regarding Technology" [2].
> not because of bad actors, but because of the 'pretending to be good' ones.
When you don't have any reason at all for doing something, and you're challenged, one tends to use imagination to dream up a plausible "good" reason.
[0] https://news.ycombinator.com/item?id=37656891
https://www.technologyreview.com/2020/03/13/905323/ai-lie-de...
Therefore…
AI is fundamentally a surveillance technology.
SOURCE:
https://arstechnica.com/tech-policy/2009/09/your-secrets-liv...
Would it be possible to combine machine learning with something like 23andMe and match 100 000 person photos with their DNA. Would we have an AI model which basically can tell how the persons looks like from a strand of hair?
Imagine finding some DNA on a crime scene. Witnesses can say that this person was a bit overweight so you slide some slider to the right a bit and get a pretty good AI generated portrait of a criminal.
How cool and scary that would be.
https://www.fsigenetics.com/article/S1872-4973(23)00045-5/fu...
........................
>DNA Facial Mapping
https://www.policechiefmagazine.org/dna-facial-mapping/
........................
>How Australian police will use DNA sequencing to predict what suspects look like [2021]
https://www.theguardian.com/australia-news/2021/dec/08/how-a...
........................
>The Role of Hair in DNA Testing
But you know just bring it to the real thing: With DNA, Video and ML you will be able to track and find that person through face recognition and probability automatically.
Hail skynet
Seems like more fodder to abuse minorities behind the guise of fancy tech and supposedly "objective" algorithms. "All [minority] look the same" isn't just something racists say, it is pretty well documented that many people struggle to discern distinguishing facial features of other ethnicities and consistently perform more poorly on facial recognition tasks involving races other than their own.
Now you have an AI generated photo to go by that further does some fudging and blurring of distinctive features... Can you imagine which races in America, for instance, would be subject to more false positives to receive police/prosecutor attention and ultimately be more likely to wand up with charges drawn against them unjustly?
If you are poor, having charges drawn up by the DA means a plea deal and time in jail or prison 9 times out of 10. If you have money for a good lawyer your odds are better, but overall are still significantly worse than if you don't have charges drawn in the first place, despite the fact that there is no evidence of a crime beyond circumstantial crap, maybe some testilying by the arresting officer (their own term, not mine), and a bogus facial match.
But the discourse in general seems to always resolve into something indistinguishable from kids arguing that the PS3 is superior to the xbox.
Isn't the "hacker" spirit one which itches to utilize and exploit? To be empowered and to be quick to kill your darlings? Why are we so quick to subordinate ourselves here? To appeal to ambiguous authorities? Is it all just some tacit futurism the AI fanboys have fallen in line with? Where has the confidence, the spirit, the sense of breezy mastery gone? There has got to still be interesting people around, those that are too cool for BigCompany Product.
Isn't that the answer? Someone is saying "entire technology, broadly, is bad" and a bunch of people who love technologies, who want to use this for all sorts of things, are going "what? no".
Who's subordinating? If I were to agree with the article would I be subordinate to techcrunch? No one is "fanboying" we're on a highly technical website with practitioners who, more or less, have a better understanding of the technology than virtually everyone involved in writing this article.
No one is even talking about BigCompany Product. A quick scan of the comments shows no mentions of meta or openai or chatgpt.
Exactly what I am trying to talk about, that is, this concern over big T Technology, rather than simply caring about one technology or another because of what they do or enable. These constant injections of vague existential stakes, when did this start happening? I am not quite that old, but I know it hasn't always been quite like this.
Is it perhaps what we have landed on as an ideology to deal with economic insecurities in the tech world? Like the wider world is disillusioned with idea that one app or another can fix the world, and now stakeholders like us need to go all in on a more profound kind of technological determinism? Where Technology is a Pure Form that needs protection?
Or is it maybe just a new generation? Younger people did not see the same kind of technological advancements older people have, so "technology " feels somehow more monolithic, self-same to them?
And honestly, it is all more interesting than sad or annoying to me... And its not about AI in general, we saw the same kind of rhetoric with crypto stuff too. Its just a shift in the way are people are talking ultimately, and its very strange.
In general I know you are right about this forum, but it has been harder and harder to sift through lately, but maybe thats just me.
I think this is putting the horse before the cart.
There is no doubt that the surveillance business model has captured tech. But "AI" - which to me is just the shortest possible way to say "algorithmic processing of digital information" - is just one more category of tools exploited to buttress and expand the dominance of said business model. It just happens that while other innovations (blockchain, quantum computing etc) are not immediately (or ever) usable, AI tools are practical and deployable here and now.
To better understand my point imagine the polar opposite system: a digital universe that fully respects our various professed humanistic values and uses technology only to empower individuals, bind society together etc. We would still need large scale algorithmic support if we are to manage interfacing our biologically defined brain bandwidth with the vast and growing silicon exo-brain.
In other words, lets us not concede that surveillance capitalism owns every possible future of tech.
When there is no Purpose, or no Purpose that Everyone accepts, and yet things are mindlessly scaled up, the scaler makes enemies. And its all pure chaos what happens next.
Whereas we recognise the pathological parents for what they are, we still think ubiquity is adorable.
It does as long as it's the only way to finance tech, which seems to be the case.
One of the many dramatic flaws of surveillance capitalism is that it cannot really provide the entire tech stack society needs. It thrives as a parasitic model, predicated on political dysfunction and public ignorance. Society must finance the bulk of the infrastructure and then hand over the keys to the digital kingdom as it were.
What is undeniable is that surveillance is the most lucrative mode. This is not surprising: borderline legal / morally dubious activities are always the most profitable.
It's obviously a bigger issue for B2C than B2B.
If everything has to be "free" then it becomes impossible to create sophisticated software or run any kind of service without finding some scuzzy way to double-dip and sell out your users. There is no viable business model that aligns the producer with the customer.
> it becomes impossible to create sophisticated software or run any kind of service
This. While monetizing user behavior is very valuable there is a ceiling to it. It can't make up for the huge potential value of all the things one can possibly build.
At some point we might be forced to conclude that surveillance capitalism makes no long-term economic sense. It is either a bubble or actually a political choice to sacrifice a real digital economy in favor of control.
But, I really wish the article said more than that. It's an overall bland read that doesn't say much else.
Personally, I place some hope in open source, open LLMs, self hosting AI, etc. I say “some hope” because the surveillance industrial military complex has so many more resources than organizations like Hugging Face.
I'm suddenly curious to see man-hour comparisons between these kinds of endeavors and, say, building the Great Pyramids.
in mad men days, advertizers would come up with own their rationale on what to show you in the screen from market surveys; person of interest may get followed throughout the day during investigation; what have you. the motivations of people haven't changed; the technology only made some things more feasible or easier to implement.
in the end, just being cognizant of how internet companies farm our data in general should empower anyone enough to find their own middle ground. or you can wait for the technology to become mature enough for on-device solutions for your homework problem (similar to how Firefox is handling translation - https://news.ycombinator.com/item?id=37658915)
It's almost as if someone spent some time prompting ChatGPT to come up with a list of potential click-bait arguments to make to elevate one's personal brand.
We are ALL vulnerable to disinformation.
We are ALL vulnerable to emotional manipulation.
We are ALL targets for the complex.
Our only saving grace is that we are not ALL worth monitoring directly. Most of us, by a wide majority, are barely even pixels on a screen somewhere. You can look at the ocean all you want but it's hard to pinpoint a single drop of water in it until it makes a splash.
I would go so far as to say it has become an evil complex that one day our 7th great grandchildren will revile and lament for us having lived through it.
Not because of any excessive torture but because of our being stuck in Tantalus's maw, so close to having everything we could want or need to have happy, jubilant lives and yet forever having it snatched out of reach by a relative handful of exceedingly greedy and selfish people.
Closed source software is surveillance technology.
Internet of Things is surveillance technology.
Blockchain / crypto is surveillance technology.
Virtual Reality is surveillance technology.
AI is surveillance technology.
Very disappointed that it took you guys this long to realize that, but it is too late to stop all of it entirely.
For my part, it feels like some tech is designed for specific use cases (eg pinboard, kagi). If you trust those developers they don't need to open source their work.
If you look at it from a systems perspective, ad tech benefits from scale and data, so when a product is built "for everyone" and monetized with ads.... Yeah it's going to be surveillance tech
Passing the buck to someone else will always have surveillance effects.
If you used said limitations to protect something you value, you have to start looking for a better protection. It's a constant process of adaptation. It cannot and should not be stopped. Instead we should be looking for ways to adapt, preferably ahead of time. Restrictions work to a rather small extent.
Be fully aware that a new scientific or engineering breakthrough will have all the potential to result in more deadly weapons, invasion of your privacy, undermining democracy, exacerbating inequality, putting you personally out of job, and disrupting whatever else you hold dear. It's always high time to think about factors which would eliminate or at least limit such consequences, and these must be positive factors, things people do willingly and profitably, not bans.
It's time to think not how to ban the privacy-invading AI (it will never work), but how to have a smaller footprint that may be used to invade your privacy, and, most importantly, how to run things in such.a way that invading your privacy is not profitable, even though still possible. And, inevitably, learn to live with even less privacy by default.
I like 'oblivious' here, as it seems to me a matter of chance about who will achieve a breakthrough. It does seem to me that once a breakthrough is achieved by someone it will be used to enhance their power over others (counterexamples welcome!) To my mind, this power calls into doubt its ethical neutrality. Yes, it is the people in possession who are making the ethical choices... but they are ethical choices enabled by the technology.
It's time to think not how to ban the privacy-invading AI (it will never work), but how to have a smaller footprint that may be used to invade your privacy, and, most importantly, how to run things in such.a way that invading your privacy is not profitable, even though still possible. And, inevitably, learn to live with even less privacy by default.
When 50%+ of the cars driving around your city are Google / Waymo cars which are essentially intelligence gathering tanks with unparalleled surveillance capability, I don't see how anyone is going to be reducing any footprints ? Cashless society? Home assistant robots that "phone home"? Face it, this is not going to be getting better anytime soon. We can try adapt all we like but maybe we should just be walking around with EMP emitting back packs on if we care about our privacy?
Advertising companies are jumping at AI for two things: it is hoped that it can give them an edge, and second, because they already have data, which is in fact fundamental for AI.
https://polisen.se/om-polisen/polisens-arbete/polisens-befog...
Relevant part:
"Om polisen skjuter mot en person ska de sträva efter att bara för tillfället oskadliggöra personen. Skotten ska i första hand riktas mot benen, men om omständigheterna kräver det får polisen skjuta direkt mot överkroppen – till exempel om den hotfulla personen befinner sig nära i avstånd och angreppet går fort."
Google translation:
"If the police shoot at a person, they must aim to render the person harmless only for the time being. The shots must primarily be aimed at the legs, but if the circumstances require it, the police may shoot directly at the upper body - for example, if the threatening person is close in distance and the attack is fast."
EDIT: some data from https://polisen.se/om-polisen/polisens-arbete/polisens-befog... -- out of 1.13 million police interventions in 2022, shots were fired at a person or vehicle 21 times. Out of those, 2 had lethal outcomes.
It's noticeable in the stats of police shootings versus police killings, where countries with these types of laws might see 10 police shootings in a year but 0 or 1 police killings.
https://en.m.wikipedia.org/wiki/Police_firearm_use_by_countr...
Using LETHAL force has even stronger guards on it. Germany itself has no legal grounds for this AT ALL, as far as I know. Some states have a clause in their Police Laws that specifically mention a "Finaler Rettungsschuß" (final saving shot), which is a shot that would kill with "almost certain probability", and it is only allowed if there is no other way for the officer to save themselves or others from significant harm (see quote from law below). The exact legal situation differs between states, but even Bavaria, which has been widely critiqued for its liberal police law, only allows for the final saving shot:
"2) Schußwaffen dürfen gegen Personen nur gebraucht werden, um angriffs- oder fluchtunfähig zu machen. Ein Schuß, der mit an Sicherheit grenzender Wahrscheinlichkeit tödlich wirken wird, ist nur zulässig, wenn er das einzige Mittel zur Abwehr einer gegenwärtigen Gefahr für Leib oder Leben einer Person ist."
See https://www.gesetze-bayern.de/Content/Document/BayPAG-83
EDIT: The amount of deaths by police in Germany reflect this. In 2019, 62 shots have been fired by police officers on people. 15 were lethal, 30 people were injured. https://www.cilip.de/2021/02/02/polizeiliche-todesschuesse-2...
Is it though? Maybe a gun is the most efficient way to send a destructive projectile to a nearby target. Any "killing living things" part is secondary to the primary function.
> which can be used for building, cooking, and also to torture someone.
Okay but that may imply AI is a building technology, & cooking technology... and surveillance technology. That's how disruptive it is, it's coming at us on multiple fronts.
Isn't enough to say that AI can be used for enhanced surveillance by malicious actors? What does it mean to be a "surveillance technology"? Now you have people arguing about whether AI fits the definition of this made up term instead of talking about the potentially nefarious surveillance applications.
AI needs data [1], so in the end people compete on the data - this drives more and more surveillance type behaviour - including using AI to collect data [2].
[1] not controversial I hope [2] eg face recognition, identification by joining the dots from traded data.
It's not necessarily nefarious per se - just the dynamic - and once the data is out there....
And accuracy doesn't matter. If it's 99% and the powerful are whitelisted for human review, that's enough. That's enough to pull a mesh so fine over every human being on the planet, forever, that reaches deeply into them before they can even begin to form a personality.
The jump from individuals listening to individual phone calls as they were happening to what became possible in the decades after that might be smaller than the jump from that to the capability of transcribing anything anyone utters, even in their sleep, automatically, and adding it to a giant corpus of data to fish for suspects or assets or blackmail recipients in, in seconds. With then being able to act on those people as if they were a dataset (set a few flags, which systems or humans downstream heed without question, done).
It's a very grave threat, and I find it interesting that people decades ago who wrote about it called it that, but now that we're in it, we find it very easy to just shrug it off. But it's the same lethal threat to what it means to be human. Privacy is required for thought, without thought there's no humanity worth mentioning, QED.
Which I think makes it even harder to get people to understand how shit this timeline is, because everybody is dreaming about what it could do for them, not to them. And the people who have reason to believe to never be at the wrong end of these sticks, have no idea how it deforms them, either.
Humanity has proven its skill at dehumanizing humans in all sorts of ways, with all sorts of rhetorical and technological means, so yeah, sure, "AI" (which in practicality means "surveil everyone and get them to like/do what you want them to like and do", the rest are gimmicks and chaff) is "nothing new", but it's still like humans going from "always having had weapons" to "fucking nukes".
(it must be Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4543807 )...?
Whittaker's point is about a certain cone of issues. Her position was previously exposed at least at https://slate.com/technology/2023/05/meredith-whittaker-inte...
> Why is it that so many companies that rely on monetizing the data of their users seem to be extremely hot on AI?
Certainly, any badly designed communication system is incidentally a surveillance device. And there are many examples stretching back to the 1940s.
However, a properly designed communication system is inherently a counter-surveillance system. It just took us 80 years to figure out that important functional requirement.
[1] https://research.tue.nl/files/197416841/20220325_Appelbaum_h...
It also provides convenient access for law enforcement to monitor all of my social media accounts and communications and metadata and scan my storage for contraband. Preferably with a warrant but of course probably with any number of backdoors installed by design. We all know at this point that warrants don't matter either.
Radio broadcast can certainly be used for surveillance, in that at the very least the location of the broadcast signal can be triangulated and monitored. Which is how law enforcement can also use the "smartphone" in my pocket to track my location in realtime.
Even when it's turned off. Because the definition of the "off" button doesn't matter, either. It isn't an "off button" it's a button that runs an app that simulates "turning the phone off," but might just put it into a low power state or who knows?
Your computer spies on you. The web spies on you. Your appliances spy on you. Your car spies on you. Your lightbulbs spy on you. The locks on your front door spy on you. Unless you're primitive enough not to use modern "smart" technology (which isn't defined as surveillance technology but which always seems to be in the news because it gets used or abused as exactly that.) Everything, everywhere, spies on you all the time. And if it doesn't, wait a few years, someone will find a way to make sure that it does. But none of it will be surveillance "by definition." Convenience, customer obsession, safety, the definitions will be plentiful but none of them will matter.
but you don't need tech for that, a spy can spy on you
I live in a rural area. The government would never bother to send someone to spy on me here, I can't imagine they suspect me of anything important enough to spend resources on me. But they can spy on me digitally for a very low cost and with little or no humna resources. Even better for them, they can spy on me digitally without cause and dump the data in storage in case they ever want to pull it up later.
Technology may not be necessary for surveillance but it is nonetheless used for surveillance. That other means besides technology can also be used for surveillance is irrelevant to the utility of technology for surveillance.
My mother birthing me made such an impact on the universe that can be surveilled. This is a moot point.
It's only a moot point when you move the goalposts outside of our lightcone.
It is concretely and objectively fact that attempts to avoid surveillance create a profile that can be used to surveil you, because the more you attempt to avoid surveillance, the more you stand out against the noise. Everything leaks.
And your counter argument is that "the material universe exists, and can be observed, therefore your argument is irrelevant?"
If you can't be bothered with a serious reply, just don't reply. It's fine. I don't expect much from Hacker News anymore.
This bubble must be extinguished by all force US regulators possess. There's not much time left, it must banned with prejudice.
The users of Midjourney aren't there for future riches, they use it either for daily entertainment or for work.
I agree and it is the current last hope investors are putting their dreams in for the overvalued stock market to keep going up. “But what if AI changes everything?” I don’t really see their point of view and I would love to see how AI would usher in this new era of value being created if anyone wants to share. Wouldn’t it more be used to lay off millions of people and information workers that can be replaced by AI, that now can’t buy things?
Almost all the software engineers I know are sceptical of this wave of AI at best, or openly dismissive at worst.
The people I know buying into the hope and/or fear are people with no actual hands on experience, or no technical knowledge beyond the 10,000ft view their engineers describe to them.
Generalising, of course, but this has been my observation as the dust settled from the AI boom late last year.
I reckon find some boring undervalued AI startup with a reliable revenue stream from the outset then sit on it for a couple of decades.
VR was a technology step that could have had interesting uses for a new experience, pumped up like crazy by VCs that just wanted a new hype train to fill with cheap money.
NFTs, at least most of them, were complete frauds from the beginning and only existed to scam users in what was basically a ponzi scheme.
How do we protect against the downsides? Do we need a privacy bill of rights? I say yes.
We could pretty easily strip most of the risks out if we cared to, though we'd lose functionality along with it and more importantly it wouldn't be the massive revenue driver it is today.