The AI Trust Crisis

> I took that screenshot on my own account. It’s toggled “on”—but I never turned it on myself.

There is a consent crisis as well, though I agree it's a smaller issue related to trust. There needs to be an actionable, legal definition of consent as it applies to website privacy—my naive assumption is that there was, but clearly that's not true, or it's not good enough or actionable enough—and it needs to preclude implying users must positively grant consent to harvest, process, or transfer data to third parties, when in fact the dirty deeds have already been done in secret.

Good article overall, but I find the analogy between "my phone is eavesdropping on me" and "openai might be lying about how they use my data" somewhat flawed because there are robust checks on third-party apps accessing my iphone's microphone that have no equivalent when my data is handed to a third party in plain. Certainly for the layperson they may as well be the same thing, but that layperson is still being protected from the former.

This might seem like splitting hairs, but I think it's extremely counterproductive to act like the battle for users' data privacy and sovereignty is lost just because most users can't tell the difference. I see this a lot from the other side: extremely cynical, at least somewhat tech-savvy people reacting to each new corporate abuse with an "old news" mentality as if we've already reached the endgame—if you haven't been using tails linux for at least a decade you may as well just zip up your home directory and email it to every shady tech corp and data broker you can find contact info for. These people ought to know better, and they ought to set a better example for those who don't.

This inculcated helplessness definitely hurts trust, but it also gives people the impression that a better world isn't possible—that there aren't better and worse choices for who to trust with their data and privacy. This Dropbox snafu seems to be that mindset coming back around: users won't give a shit if we imply we're sending their private files to a third party without asking, right? Lunacy.

Incidentally, I've had most of my data out of Dropbox for a while (in favor of a self-hosted solution), but yesterday was the kick in the ass I needed to cancel it for good. Thanks, Dropbox!

This reads a bit naive and “assume good intent”-y for me. Look at what’s happening outside of AI for a decade: everyone’s eating data like a compulsive hoarder. Not just Google and Facebook who are actually using it in their core products, but everyone. Today I discovered that a mini-site of Swedish traditional recipes I’ve been using for Christmas has added autoplaying videos, dark-pattern cookie consent banners, the usual. Almost every new app/site is oriented around this economic axis.. and then the sudden coordinated lockdown of 3p APIs right around when LLMs starts getting strong.

And now we have ChatGPT/OpenAI and their competitors.. if the other players eat data like a secret midnight snack, current-gen AI is like zombies (the fast and twitchy variants) starving for blood and brains. Both because data serves a more direct role in the product, but also because of the typical hype-train-race psychology of tech VCs has been woken from slumber by the first potential paradigm shift in decades.

All circumstances point towards zombie apocalypse/ gold rush / ask for forgiveness later / etc etc. I strongly believe that’s why they’re (all of them) doubling down on the safety/responsibility rhetoric now, before the inevitable reputational PR crises (plural). Get ammo to muddy the waters early.

Meanwhile we techies are lulling around like we didn’t deeply just experience the last 10 years and thinking it’ll be different this time because.. AI is rooted in academia? Flashy new companies? The safety rhetoric? Edgy Twitter takes from “down-to-earth” founders?

I don’t pretend to know exactly what goes on behind the scenes but I’ve been around long enough to know how people work. And they haven’t changed for the better.

I think your article glosses over the fact that we have privacy concerns beyond training on my data.

I'm a working professional and I have clients that are governed by confidentiality agreements and regulations regarding where information goes, and I would just prefer using a service where my data rests on a server instead of having more and more points for a data breach to be introduced.

I don't really understand why my data isn't fully encrypted at all times and only I can view it in the first place, but the idea that they are actively sending it across the internet to be ingested by other companies and processed without my consent or interest is so terrible.

I often use AI features when I opt into them, but having a company just sending my personal files all over the internet without my consent is insanity.

Honestly, OneDrive has a migration tool and I got a trial for dropbox business and moved all my files automatically last night. It's just the last straw in their company constantly doing things I don't ask them to do like introducing crap and popups into my desktop interface and never offering the feature I constantly ask for... end to end encryption.

If you want a couple click migration from Dropbox Business to an Office 365 Onedrive account, it's right here: https://learn.microsoft.com/en-us/sharepointmigration/mm-dro...

> The key issue here is that people are worried that their private files on Dropbox are being passed to OpenAI to use as training data for their models

This is only part of it. I don’t want my data being sent anywhere unless I authorize it, regardless of what it’s being used for.

In this case, we not only have to worry about OpenAI training on our files (I have no reason to doubt that they are truthful when they say they won’t), but we also have to trust that they can securely handle our files.

The microphone trust aspect in the article seems like a red herring distracting from what could be a clearer point.

Facebook literally takes your data from their apps and internet, tracks your behavior on the internet, and feeds this data into models of you. These models are so accurate they can sometimes basically predict what you're thinking. Hence the layman jumping to the conclusion that they must be spying though the mic.

A LLM company like OpenAI, and their partners, employs almost literally this exact model. Grabs data from whatever sources to improve their models, to increase the likelihood you'll keep clicking where they want you to click, to monetize you.

> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical.

Even though I don’t believe Facebook is spying on anyone surreptitiously through their phone’s microphone, I find that specific argument entirely unpersuasive.

Facebook’s reputation is dogshit, at least with regular non-technical people I know. I’m in the US. People know they helped foment the January 6 insurrection in 2021 and saw how they deflected any and all responsibility while not fixing a thing afterwards. The reputational damage they’d absorb from actually doing this thing that so many people already think they’re probably doing pales by comparison.

I certainly don't "trust" OpenAI or any other big company about what they say they did, or will do, or are doing.

Yet I believe OpenAI isn't using data from Dropbox to train their models without users' consent.

BUT I don't think that's the problem here. The problem is data in transit; data sent to third parties who can actually read it, and who may have rogue employees that Dropbox has no control over; data that can appear in logs or subject to different policies, etc.

If I send private data to Dropbox they can't send it to anyone for any reason, including "improving" their product, without my explicit and informed consent. I'm not sure how this is even debatable.

If Dropbox wants to house models and offer RAG search themselves, to consenting users, that's one thing.

If Dropbox sends all data of all users to third parties without telling anyone before the fact, that's another thing. A terrible thing.

Trust Crisis for AI? What after we see the board/CEO of one company fired/changed apparently for allegations of lies or manipulation that nobody is clear on? If Dropbox derives data from user data by scanning said data, then that "derived" data is no longer "users data" it is Dropbox data and can be shared. It may only be statistical in nature and not related directly to individual users but isn't that exactly what training data is? Isn't that how it works? That can be shared to train AI models can't it? Its not lies its hair splitting.

No its called unethical behavior and has become the norm for big tech.

I think you'd find the people who don't trust AI companies also harbor similar feelings about companies in many different industries, or non profits, or even government agencies, depending on who specifically you ask. It seems to me that there is a much bigger trust issue well beyond the scope of AI based companies, so trying to have that specific field fight this distrust against them that really comes from all fronts, seems like an impossible task well out of the scope of these businesses. I'm not sure what the answer to this issue is, or if it even is a real issue, or what we might be marching toward should this pervasive cynicism spread to all things and all people. Perhaps we are simply cursed to live in interesting times.

This article misses the mark. The problem is that, for good reason, we've moved to a post-trust society. The author argues that you're a nutjob if you don't trust what large corporations are saying. He goes so far as to say it's on you to prove that they're doing something wrong.

This doesn't represent reality. Take this quote:

> A society where big companies tell blatant lies about how they are handling our data—and get away with it without consequences—is a very unhealthy society.

So, what are the consequences of lying? If there are any, I'm curious what they are. The author is arguing that "big companies" are going to do the morally correct thing even though it hurts their profit. That's just a silly argument.

In a post-trust society, companies have to prove they're doing what they say. There's no more making a statement and having the masses act as if that's what's going to happen.

Honestly, is there less than a 100% chance that down the road there will be a disclosure that there was a "mistake" where your data was used differently from what they claim? Fool me once, shame on you, fool me 5,872,328 times, shame on me.

People believing the FB was eavesdropping on their conversations might be wrong, but that's mostly a technicality. It's like saying that I passed out because you hit me with a baseball bat, to which you'd reply: naaa...ah! it was a hockey stick!

FB has been/is spying on people and making us collectively more stupid, but using different, more esoteric/harder to explain methods (e.g. cross-device behavioural targeting).

> Those companies need to earn our trust. How can we help them do that?

This is a wicked problem and such a broad question (+ a bit of a weird take imho), that I don't see anything actionable. So, here's a half-baked list off the top of my head (so, poorly expressed, incomplete, incoherent):

- slow down the ill-conceived progress at all cost and understand that moving fast in the wrong direction ≠ progress

- force stricter transparency rules through legislative action

- understand and accept that no-trust should be the default

- invest more in open and offline models

- educate people

- accept that some of those companies in their current shape don't deserve our trust

Edit: This is a better take: https://news.ycombinator.com/reply?id=38643792&goto=item%3Fi...

I'm asking this genuinely and honestly:

How can I trust a company about my data, when their whole technology's performance depends on scraping the whole internet and feeding to a model? Also consider the fact that the data I'm providing them is way cleaner than what they scrape.

Moreover, we (at least the older ones amongst us) have seen what Microsoft has done in the past, and said company is almost one with the same Microsoft, which is ready to do anything for platform dominance.

OpenAI, esp. when combined with Microsoft and the whole A.I. hype paints a very untrustworthy picture in my eyes.

I'm not against the technology, but how it's trained, developed, hyped and how the researchers in the ecosystem behave about the data they handle makes it very off-putting in every sense, esp. in privacy and ethics.

It's like looking into a proverbial sausage factory, but this one is way worse.

I've briefly touched it in another blog post. It is not just trusting the institutions to not use the data for training. Optimizations you apply to making LLM inference fast might present new security landscape that we don't know much about. For example, if your LLM inference engine stores KV cache across user sessions, you can in theory do a timing attack to retrieve what's in the KV cache, potentially leaking other users' requests.

There is no independent monitor or authority that can check what OpenAI does or doesn’t do. So there is no reason to trust them. Simple as that. Tipping the scales further are the numerous instances of tech companies lacking sufficient data security, the latest being 23andme. Case closed.

There’s a deeper connection between concerns about OpenAI and the “Facebook is spying on us via microphones” meme:

While there are perfectly rational, non-hot-mike explanations for the phenomenon of talking to a friend about something and then having it show up in an ad, the experience itself is damn spooky, no matter how it’s occurring.

And hypertargeted ads are just the earliest ripples of the wave of AI-driven applications for predicting human behavior. Those predictions—about where we’ll decide to go on vacation, when we’ll quit our jobs, what we’d like to buy, who we’ll marry and divorce—will be more right than wrong to an incredibly spooky degree.

I have no idea how society will react to that.

I am building an AI content generator and my edge that makes it better is sourcing data from not easily accessible data sources (think research papers) through SEO. I cannot imagine that these companies are not using every bit of confidential data for their advantage. Proprietary data access is the only thing that matters in the future of AI wars

> think this is a combination of confusing wording and the eternal vagueness of what the term “consent” means in a world where everyone agrees to the terms and conditions of everything without reading

I think this is always the case. Unless the companies are forced to actually simplify and limit their TOS -or even divide it by features-. No one is realistically expected to read hundred of pages daily in legalize. I don't think many vendors do it our of necessity but as a workaround consent issue. This needs to be addressed by legislation that ban those practices.

Yes. In general, it would be great if companies threw away the outdated PR-communication playbook, and instead took a leaf from some indie devs and had the builders directly communicate with users as people. Products are built by people, companies are made up of people, but most communication is instead corporate.

In practice, I think the PR playbook is too entrenched, but we can dream.

You can only truly trust someone when your incentives are aligned, and people's incentives aren't aligned with OpenAI, Facebook or Dropbox. That trust simply can't be earned.

> It’s increasing clear to me like people simply don’t believe OpenAI when they’re told that data won’t be used for training.

I certainly don't. I don't trust any of the AI companies at all. Their behavior and statements have given me quite a number of reasons not to, and no reasons why I should.

This isn't really AI specific. The AI angle I suppose is that there is a new way for companies to exploit data they get from you, but like the article says with Facebook, people already believe their info is being exploited. Laws won't change it (not that they shouldn't exist) there's always too much fine print that can be gamed and they'd never be enforced to the benefit of the consumer anyway. Like privacy, it's awareness and not giving companies the chance, which will take a long time but hopefully will be the default instead of just not caring. For AI, local or own-cloud models are a good start, it's actually hard to belive a company like dropbox would go with OpenAI and not see how bad it looks.

There is an ongoing question about whether AI can be trained on copywritten works.

If the AI pioneering companies are able to push back against the entire publishing industry and have nothing more than a conversation about it, then why would I ever expect any company to respect me or my personal data?

In fact, I think there is a large and growing portion of the population who have watched big business scoff at laws and regulation (see Wells Fargo, Uber multiple crypto exchanges, etc) and are realizing that profit is the only motivator.

That's not even talking about the chaotic political landscape.

I think we will soon see a crisis in institutional trust across the board.

I think this article is a little unfair to the lay critics, aka conspiracy theorists. The average user of Facebook has no idea how massive and widespread their personal data collection machine is. They correctly intuit that Facebook has an absurb amount of personal data on them, but don't have any frame of reference for how most of that data is collected. But the concept of a microphone listening in to private conversations is easy grok, and while it's factually wrong, the effect it points to is a lot closer to reality than the corporate speak they here from the company about how they "Have the utmost care for users personal privacy, blah blah blah". Remember a few years ago when Facebook tried to rebrand itself as a privacy company!?

Facebook absolutely has a lot of information that the average person considers personal and private, and big tech companies work hard to mantain the illusion that their data collection isn't as pervasive as it is.

With OpenAI, most non-technical users understand that the company has ingested a LOT of creative works done by individuals, most of which weren't expected to be ingested by a giant AI company and potentially regurgitated to millions of users.

The better corporate messaging this article argues for will help ameliorate concerns to some degree, but won't address the root of the problem. Most people might have the details wrong, but they are directionally right about the massive data being collected and used by these companies and they have every right to be concerned about that.

In what way would AI deserve our trust? To date data hungry companies have proven to NOT have consumers interest in mind. Why should AI companies be any different?

In dubio pro reo? I don't think so.

>One interesting difference here is that in the Facebook example people have personal evidence that makes them believe they understand what’s going on.

>With AI we have almost the complete opposite: AI models are weird black boxes, built in secret and with no way of understanding what the training data was or how it influences the model.

In principle, I think we could have significant visibility into how OpenAI trains its models by doing something like the following:

* Generate a big list of "facts" like: Every fevilsor has a chatterbon. Some have a miralitt too, but that is rare. (Or even: Suzy Q Campbell, a philosopher who specializes in thought experiments, lives in Springfield, Virginia. Her phone number is 555-8302.)

* Generate a big list of places where OpenAI could, in principle, be harvesting data: HN threads, ChatGPT conversations, StackOverflow, Dropbox files, reddit posts, etc.

* Randomly seed facts in various places.

* Check which facts get picked up by the next version of GPT.

If you really don't trust OpenAI, be subtle about where and what you seed (e.g. avoid obvious nonsense words) to make it hard for their engineers to filter out your canaries even if they tried.

It's important to seed some facts in places they're known to scrape, as a control. If those facts aren't getting picked up, then you have methodological issues. You might have to repeat a fact a number of times in the dataset before it actually makes its way into the model.

This is a great piece. I often observe that this really applies at a high level in general as one of the key elements that demarcates successful / advanced economies from lesser ones : how much is trust a limiting factor on what you can achieve?

If you think about modern society and the levels of trust needed to do certain things : I spent nearly a lifetime of earned income on a house and I completely trusted the agent and bank who facilitated that. Without that trust, the whole transaction wouldn't be possible. Why did I have that trust? These are the hidden value of all those laws and regulations that we often hate so much. As much as anything they create the stable platform for high value transactions to take place. The price of not having them is not being able to access the value you can get if you can successfully execute high trust transactions. There's no way I would gamble my whole life's income if I didn't trust the process. I would have to buy a smaller house.

So AI is playing out along these lines. How much of the value we can access from AI is eventually going to be limited by how much trust we can grant it. Which ironically may derive from how much regulation / legal framework is successfully put around it, which may in the short term slow it down. The self-hosted open source model route is a substitute / fallback for not having trust in hosted models. We can't really pretend that it's possible to self-host a model that will be as good as a hosted one, the scale is always going to be a material difference. But we may be to hit the point where for targeted applications there are diminishing returns, and that may be good enough.

A friend told me about the Dropbox setting last night, so I logged in and turned it off. This morning, I went to look for the setting to take a screenshot but it's gone. The setting just...isn't there anymore.

Made me feel like I was going a bit crazy TBQH. Surely I didn't misremember?

Annoyed because it was a convenient file storage solution and now they have proven themselves untrustworthy so I have to set up my own thing. My fault for trusting to begin with, I suppose.

I love convincing people that Facebook isn't spying through the microphone but those relevant ads that relate to what your friend just said are as creepy as you think, just without the microphone. The example I like to give is if your friend with pets bought a new vacuum that they tell you is an absolute godsend for dog and cat hair. Facebook knows you have pets, they know your friend has pets, they know that you and your friend are friends. If you have location data on, they know that you and your friend are together. They know your friend recently bought a vacuum because they have access to a lot of purchase data. So it may seem like they're listening when your friend mentions the vacuum and an ad for it shows up in your feed later that day. They weren't listening, they were tracking. Listening would be spying in one aspect; rather they were following multiple aspects like purchases, relationships, lifestyle. Still creepy, just in a way that's harder for consumers to understand.

Hmm, isn't the whole point of the "Facebook spies on us though our Microphone" story that then people laugh about it move on and use Facebook regardless?

Hundreds of millions of people use ChatGPT, billions Google and Facebook. The value wins. If the service is good enough people just tend to "hope" more than trust that their data is save enough.

For the dropbox backlash people just didn't see the value.

> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical.

This isn’t the disincentive you think it is. VW manufacturing vehicles that purposely deceived emissions testing was textbook corporate malfeasance, perhaps one of the worst examples of it. They took a loss on the ledger and came out of it relatively unscathed

Other than just manually generating a UUID and storing it then doing searches for it, is there any sort of formalized service/application/library for "Air Tag but for your data"?

Generate a unique string, store it along with my data, notify me if it shows up anywhere.

I wonder if it's possible to generate a canary of that sort that can be detected in AI models.

I think local models will be more and more relevant. First of all, the locally available computing power for inference is going up. In addition, inference is much cheaper than training. Finally, there are diminishing returns with more and more parameters.

Local models also solve a lot of the trust issues

Great article overall and agree with local models. If it is connected to the internet, then it is not secure, you cannot trust your data not to be slurped up somewhere.

Threat models yada yada.

I'm in love with how fast models like insanely fast whisper, ggml llama.cpp, Mixtral, YOLO v5 e.t.c have come along.

I very much predict devices in 2024+ that have powerful energy efficient GPUs and do multimodal inference locally without any connection to the internet.

I remember buying the first comma.ai pre-panding which was essentially a modded android LG phone driving the car. No connection to internet, a freaking mobile phone driving the car without any human input on the highway. That totally blew my mind.

I'm awaiting Apple to release a version of Siri that runs totally offline.

Smart phones are the perfect target.

Who in their right mind would trust a company that just had its CEO publicly ousted for allegedly lying to the board, only to have him put back in charge? And we have yet to see any kind of reasonable explanation or statement about what really happened.

Where there's smoke, there's fire. Public trust in large tech companies is at an all-time low. Nobody thinks tech companies want to help anybody but the few rich people running them anymore.

I wouldn't trust a word OpenAI says, because even their name is a lie. The code isn't open, it's not auditable and you have dishonest people running the company.

All it takes is a simple tiny change to the user agreement and then OpenAI will start training on this user data and most people won't notice in time to switch their toggle off

> I don’t think so: I think this is a combination of confusing wording and the eternal vagueness of what the term “consent” means in a world where everyone agrees to the terms and conditions of everything without reading them.

Bullshit. Consent means consent.

There is no confusion here. Nothing is vague. This is explicit fraud.

I feel like every day now, I'm reading an article where the problem is obviously just fraud.

Fraud has always been illegal. When did we stop prosecuting it? Why aren't we talking about that every day?

Isn't there sufficient evidence to conclude that OpenAI is training on things they know they probably shouldn't be training on? Like the full text of novels that are not publicly available on the internet? I don't think it's that unreasonable to extrapolate that they would blur the lines of legal agreements to get access to private user data they shouldn't have.

"A key role of government is to prevent this from happening."

Well, unfortunately, we have a government trust crisis also.

The problem with AI isn't (p)doom it's the 99% probability that the people at companies like Dropbox will continue to treat customers like human chattel.

They'd enslave you and charge you a subscription for oxygen if they could – and insist it was a feature.

People are going to believe the worst about AI for the rest of the decade, for reasons entirely unrelated to what actually happens.

Concluding "if we respect people's feelings and do one weird trick, people will suddenly trust AI" is naive beyond number.

This comes down to a combination of security and provenance. You have to protect the data, of course, but you also have to keep track of the use rights. Identifying all use rights upfront seems tricky, are the Creative Commons licenses sufficient?

If I can be automatically signed up to allow openai to process (but not train on) my data, I absolutely don't trust that the deal will not be altered the moment any counter-party to the “agreement” (besides me) desires it be altered.

Copy-pasta from the article, about a related claim (Facebook spying using phone microphones)

This theory has been floating around for years. From a technical perspective it should be easy to disprove:

   * Mobile phone operating systems don’t allow apps to invisibly access the microphone.
   * Privacy researchers can audit communications between devices and Facebook to confirm if this is happening.
   * Running high quality voice recognition like this at scale is extremely expensive—I had a conversation with a friend who works on server-based machine learning at Apple a few years ago who found the entire idea laughable.

The first point seems believable, but the second and third do not. Obviously a nefarious Facebook would be toggling these features off when they were being inspected, and even if they were not, they could be using sneaky exfiltration techniques. With the rise of app attestation, they would be able to do so in a way that would be 100% undetectable by reverse engineers. The relevant code would only be delivered to the app when it saw that it was running on an L1 trusted phone (with hardware security intact, unrooted). Additionally, they could embed some whisper-tiny on the device and force it to do its own speech recognition and hotword detection, only sending a list of "ad topics" back to their servers.

I don't think Facebook is doing this, and the social reasons hold more weight with me, but I don't think it's technically impossible or easy to prove that they did not _ever_ do this.

The cost/benefit of believing an infotech company about privacy concerns not backed by an explicit contract is firmly on the side of "don't bother believing them".

Restore the trust? Pay us.

I think it’s obscene that Dropbox turns that setting on by default.

> As with many conspiracy theories, too many people would have to be “in the loop” and not blow the whistle.

I used to believe this, but I do not understand how anyone can say this after the Snowden revelations. The vast majority of engineers are not martyrs. Given a choice to blow the whistle and then seek politcal asylum in Russia, most will not blow that whistle. Instead, they will live their lives and work to support their families.

"But this is not about Goverment Surveillance!"

Maybe this is true - I don't want to argue it here but I do think large tech companies are willing accomplices in many surveillance states - but an engineer at Open AI or Dropbox will probably not be facing felony charges for blowing the whistle. They will be facing the end of their career in big tech, and that's enough to dissuade most of them.

I'd guess maybe 200 engineers would have to be in loop before there is an 80% chance that one blows the whistle in any five-year period, and if they are careful they can keep the number of engineers in the know MUCH, MUCH smaller than that.

> ...people are worried that their private files on Dropbox...

I'm not worried, because I use Cryptomator. Great app, acts as a file encryption layer on top of any cloud storage.

Personally, I think "AI" should be required to disclose the origins of the data it bases it's responses on, and if any of those are copyrighted.

Can someone explain the consisten use of “Facebook are” instead of “Facebook is,” is this proper grammar?

"As with so much in AI, people are left with nothing more than “vibes” to go on. And the vibes are bad."

word

This article has good intentions but seems like a giant buildup to promote local inference tech stack

“Data is deleted from third party servers in 30 days.”

No, that’s a thing that should never have happened. Ever.

Reading the post, my first reaction is to remove all docs from my free dropbox account.

Trust is simple. If you can’t trust your own family, who can you trust? Families, that is the answer. Oddly, the trend is in the opposite direction; trust media, trust government, trust apple, OpenAI, etc…. Things will get better when that trend reverses.

The TLDR is "local models are great to have but hopefully we don't lose out on large hosted models due to distrust about data going straight to OA or some other big companies."

The arguments made in favour of bigco trustability are not great though, esp the parallel between AI data ingestion and Instagram's rumored use of mics to capture voice data for ad targeting:

> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical.

FB has already paid the biggest fine in FTC history for privacy breaches. It has no goodwill, it's running on user inertia and indifference.

The internet has fractionated out our cultural references and commonly used language. This has caused the communication of belief to separate out from lanaguage.

The Facebook Spying idea is the common place to agree and complain about phone spying in general. It is the poster boy for communicating an implicit belief about a larger topic.

We have begun to use language to communicate implicit and shared beliefs without any evidence, facts, agreement or qualification.

We come to our own beliefs privately and then choose a poster boy to carry the conversation. There's not much surprising about the continuation of conspiracy theories, or topics that look like conspiracy theories. We shredded the common cultural references apart...

I find the premise that users should trust tech companies by default absolutely ludicrous. The author seems to live in a bubble where tech companies don't commit privacy violations regardless of what their privacy policies say. Yet in the real world, even when they're caught with their hand in the cookie jar, the fines these companies are forced to pay are just the cost of doing business.

Earlier this year Microsoft was fined $20M for collecting PII from children, which it "shared" with advertisers[1]. And this is _Microsoft_, not even an adtech giant. The abuses from those have their own Wikipedia articles[2,3]. FB was famously fined $5B in 2019[4]. It would be beyond naive to believe that these were just "oopsies", and that somehow these companies have changed their ways.

_This_ is why it's not implausible that the software that runs on our most personal computing devices, built by these same companies, is hoovering up our data in the sneakiest ways possible. Equating this to conspiracy theories is dishonest at best.

When Mark Zuckerberg puts tape over his webcam and microphone, it would be foolish to think you know better not to.

[1]: https://www.ftc.gov/news-events/news/press-releases/2023/06/...

[2]: https://en.wikipedia.org/wiki/Privacy_concerns_regarding_Goo...

[3]: https://en.wikipedia.org/wiki/Privacy_concerns_with_Facebook

[4]: https://www.ftc.gov/news-events/news/press-releases/2019/07/...

I've seen this play out on Discord too. When they rolled out the new "summarize conversations with AI" feature, I saw a lot of people concerned their data was going to be stolen for AI training as a result. I pointed out that they explicitly said this won't happen, and also that Discord's ToS is already vague enough that there's no guarantee your data isn't already being used for other purposes. The LLM fearmongering is powerful.

It's a market trust crisis amplified by AI. Everything in the article applies to scummy business practices that were spreading before we all caught the CUDA crazies.

> The non-technical reasons are even stronger: > Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical.

Ah, bullshit. They've lied about other things, repeatedly and constantly. It's a risk, sure - but a calculated one, and one they've taken before.

This article touches on the more critical and flip side of a significant problem, which I think will hurt many of the incumbents, while whoever figures out how to resolve it will gain a lot of credit.

Facebook doesn’t explain how their ad targeting works at the individual level, but they thought about it and tried—barely. It was perceived as an obstacle to black-box ML, even when there were clear opportunities to show customers (and partners): here’s how the algebra works, here are key dimensions that we have identified explicitly and that the advertisers wanted us to target (age, gender), here are more dimensions and the feedback from showing that ads to other tells us (people who like board games respond more to the ad, and we think you do).

That training, in clear, accessible language that anyone would understand exists: it’s part of onboarding.

The result of not having it is a large chunk of ads are shown to people who literally don’t speak that language (something Antonio GM mentions in his book and that was identified, corrected, broken, and re-identified at least three separate times since). But even benefits would be enormous: it would free what looks like 60% of my ad inventory by not showing gambling ads.

But they went the other way, treating users asking questions that their own employees ask on their first days with suspicion—assuming they knew better than the people they are talking about.

OpenAI is less adversarial, but they must explain how training and testing work with user-relevant examples: “Here’s a conversation you had last week. Here’s the feedback you gave. Here’s how we detected it was relevant to refinement. Here are examples of questions whose answers were changed by your contribution.”

Having a full-text search on their training set might be difficult, but something along those lines could be implemented and reviewed by a neutral third party. Those conversations would lead to insights about where to find more relevant data.

As Simon W. writes, this matters, and you want to respect privacy because it’s an inherent good, but you also want to preserve trust because it’s commercially essential (even though people are quick to forget if they get free stuff). I think you should do it because security by obscurity, or at least the ML equivalent of that, isn’t working.

https://xkcd.com/1838/ is funny, but the truth is that we can offer some tools for exploration and feature breakdown. Every time I did, we learned so much from those.

> Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical.

The author refers to the theory that FB is always listening as “laughable”, but this statement is even more laughable. If FB actually got caught doing this, at most there might be a few articles about it, and maybe a class action lawsuit resulting in a few pennies distributed to individual users, and then things would go right back to business as usual.

The only convincing argument that FB isn’t always listening is that it would be too expensive to be worth it. Arguments about “reputation” risk are beyond absurd. They don’t care.

The biggest problem with AI is not the tech but the people. White billionaires worrying about SkyNet but totally overlooking the impact on the everyday "poors".

The individuals who never had to go through a second of adversity in their lives cannot be expected to understand. In fact, their sociopathic tendencies only skyrocketed through the years of isolation from the real world, with COVID compounding the effects.

Casually talking about how robots should replace humans as a species because of "efficiency"...

By the way, did you know that Sam Altman was VERY active in politics until the latest drama?

He was backing a candidate to primary Joe Biden, but now he had to go take care of his own stuff. The outcome of this would be pretty obvious to those who follow American politics:

https://www.theatlantic.com/press-releases/archive/2023/12/a...

My point is, a person that cannot draw a straight line from F---ing Around to Finding Out cannot be trusted with something infinitely more complex.

Just read this. I have had enough weird incidents where you for example chat with somebody about a topic and a day later I suddenly received ads Youtube recommendations about this topic. There is a lot of reason to not believe big companies. I don't see why Facebook should be more ethical than tobacco or oil companies that have been lying and obfuscating for decades.

The only solution I see is to make it illegal to use personal data. Especially make data brokers illegal. When you give a company access to your data, you often agree that they will pass the data to third parties who then pass it on and suddenly everybody has your data because you have permission to use your data.

From article:

> Trust is really important. Companies lying about what they do with your privacy is a very serious allegation.

> A society where big companies tell blatant lies about how they are handling our data—and get away with it without consequences—is a very unhealthy society.

> A key role of government is to prevent this from happening. If OpenAI are training on data that they said they wouldn’t train on, or if Facebook are spying on us through our phone’s microphones, they should be hauled in front of regulators and/or sued into the ground.

I find it very difficult to believe that big technology could be held accountable to anything by legal means by regulators. They have too large legal teams, and too well written legal agreements, and eulas.

I find it impossible to believe that I, as a person, could challenge them in any way though legal means.

It's 2023 let's summarise AI

* My Documents can be trained on

* My Instagram images are legally not mine and can be trained on

* My voice can be cloned without consent

* My complete motion capture data can be replicated to another image

* My analytics and usage data is owned by either US or Chinese corporations who either side will train AI on to feed me more ads

* A green colored hardware giant is leading the AI warfare

* The internet is turning highly anti consumer