Hacker News Reader

Comment by nl | Hacker News Reader

This is moderately interesting work but the title feels wrong to me.

They use LLMs to analyse social media and turn unstructured text ("I always get stuck waiting for a hook turn") into structured data (location=Melbourne, Australia).

They test this on a set of Reddit profiles that have been hand labelled.

However, the fact that this hand labeling is possible indicates in itself that the issue isn't a pure privacy violation. It's more that automating this in bulk is now possible which was difficult before.

Is this ethical? It isn't obvious to me that it's not - I think most people writing on a public site realize their writing is public, and thew fact we see common warnings about not posting personal information shows that is true.

It's not clear it violates privacy expectations either - even if some people don't like it.

View on HN

> most people writing on a public site realize their writing is public

Most people do not think that personal details they didn’t intend to share in their writing can be surreptitiously deduced from things like their word choice and phrasing.

It’s true, and people need to learn about and consider it, but no, most people have no idea that it’s really possible or how far it can go.

Their intent is to share what they intended to share. Efforts to go beyond that — whether through LLM’s, earlier data mining techniques, or Holmes/Poirot-like observation — are precisely “violating privacy expectations”

While playing with semantic categorization, I also found that it's also what you write about and how you write (seems obvious, but I wanted to confirm). I took a year of hn comments and compared my previous account to my current. It was in the top 20 ranked by similarity (out of something like 60k) just by building a custom 'fingerprint' for each user.

Automating this in bulk enables societal harms considerably more dangerous than anyone could have reasonably foreseen prior to 2020.

If it was discovered that someone had been operating a secret network of CCTV cameras pointed at every street corner on Earth for the last 20 years, would you say "well, you had no reasonable expectation of privacy anyway"?

I'm sympathetic to the "doing things in bulk is different to doing it as a once off". But in this case the harm isn't obvious to me.

(Also, why 2020? I could understand if you'd said 2016 with Cambridge Analytica but 2020 isn't clear to me).

Cambridge Analytica collected structured data: friend networks, likes, and interests. At that time, it wasn't plausible to obtain equally powerful insights from unstructured data. That only became a realistic scenario when GPT-3 came out, in 2020.

Oh.

I assure you plenty of companies were doing NLP based classification in this field much earlier than that. AdSense for example was doing it on pages, and of course Google search strings were a similar example.

I don't think your camera analogy is equivalent. It's moreso like actors were filmed with the full knowledge they were, except now the studios are going to make creating deep fakes movies of them available not only to themselves, but to everyone

If it really was every, then likely yes. No doubt there would be some interesting footage of important public figures in there too.

Not that I'm advocating against privacy in general, but when "I know what you did" is no longer asymmetrical, as in "I know what you did too", it'll certainly change how society thinks about these things.

Eh, probably not in any way that is good. Tyranny of the moral majority is a very common problem in societies with low privacy. While everybody may have their own set of oddities, some will be more harshly judged than others. For example just about every religious society will have very strong opinions and actions against homosexuality, they will say they have the same actions against adultery, but it's unlikely to play out near as severe.

I honestly wouldn’t mind. It’s the street corner. Although I’d rather they have coordinated with governments to give the police access to the cameras

> However, the fact that this hand labeling is possible indicates in itself that the issue isn't a pure privacy violation. It's more that automating this in bulk is now possible which was difficult before.

Except that it was previously possible, and could be done far more cost effectively, before. Ad targeting systems from fifteen years ago were already doing this with precision in the same neighbourhood.

Ad companies are so valuable because not just any company can reach the scale or access of user interactions enough to perform this kind of inference accurately.

Yes you could get precise information but not just from a few sentences said by one user. There's also the quality of information you could previously glean from just unstructured data (noticeably lower)

That's not exactly accurate. Ad companies definitely need to train using large data sets, but they could infer with very little information, and in most cases, "unstructured" and "low quality" data was actually quite helpful for inferences about a user. Just a free form text box where someone was prompted for "favourite movies" (with no validation of the content... you didn't even have to have any movie titles in it) could yield surprisingly accurate inferences about income, education, gender, religion, age, marital status, etc. Ironically it was more predictive than inferring from structured data.

> However, the fact that this hand labeling is possible indicates in itself that the issue isn't a pure privacy violation.

They can use it on profiles where that wouldn't've been possible before, and which do not have the obvious tells like "By the way, I live in Melbourne, Australia". Once an 'easy' profile has been labeled, then the model will be trained to learn all the features in that profile which correlate with living in Melbourne, including the ones no human had ever noticed before, and which will be present in other Melbourne profiles. (You could, and should, also remove the obvious tells from the training data after you have constructed the label, to force it to discover subtler features beyond the ones your human labelers used.)

This lets you bootstrap your classifier to extract information no human could have before. That would seem to be a 'pure privacy violation' by your definition.

Would you want it done to you? I sure wouldn't.

Too bad it's probably going to be done to me anyway.

It pretty much already is right (even if the technology used is different)?

A company like Google builds a profile[1] from my search queries. Many other AdTech companies do similar things.

> Would you want it done to you? I sure wouldn't.

I don't mind this - if I have to see ads I prefer ones relevant to me over ones that are completely irrelevant.

[1] https://myadcenter.google.com/controls

Would you like political ads micro-targeted to you?

Gay men get this ad: Only Politician Foo protects gay security in the streets at night!

Homophobes get this ad: Then you must vote for Politician Foo, only he opposses gay marriage!

Pro-lifers get this ad: Vote for Foo to protect the children from the very first moment!

Pro-abortion get this ad: Foo protects women's rights!

Advertising should be as general as possible - microtargeting people in a democracy is extraordinarily dangerous.

wat? targeting markets has been a thing forever. Rim ads in car magazines. Jewelery ads in fashion magazines. I suspect there are different ads in the sports section of a newspaper vs the business section, vs the lifestyle section.

I suspect any ads on grinder are different from ads on tinder as well.

None of that seems upsetting to me at all. Even if it's advertising, even if it's about stuff that's political, even if it's aimed at causes I dislike.

I think misleading advertising is always dangerous but there isn't anything particularly wrong about targeting.

Politician Foo sounds like a major flip-flopper.

> Advertising should be as general as possible - microtargeting people in a democracy is extraordinarily dangerous.

I don’t understand how that follows.

Your examples of microtargeting seem helpful since they speak to how politician Foo appeals in ways relevant to a particular segment. Instead of unknown-likelihood inference of how the politician would operate because of some associations, a voter would be able to make an informed choice based on what is significant to them.

The alternative to some level of targeting or relevancing is broad branding exercises like “Coke is it” or “Team Blue protects you—team red wants you dead.”

microtargeting in a democracy means that literally every candidate is your candidate. There isn't any way to actually figure out what the candidate's platform is, because it will be tailored for you, specifically ... and it all might be a lie or so misconstrued that you vote for someone who is actually against the things you want.

Everything you stated is already the case with non-microtargeting. The claim is “microtargeting people in a democracy is extraordinarily dangerous.” Why reply if you can’t address the claim?

Sure, let me connect the dots for you.

1. Let's say there are two parties: pink and purple.

2. You absolutely love Pink Party because they want to paint the roads pink and force everyone to drive pink cars.

3. I'm running for Purple Party, which says everyone should be allowed to drive whatever color car they want, but we also want pink roads.

4. Gerrymandering is illegal, but our team determined that we can sway some areas. You live in one of those areas.

5. My team writes up an ad saying something like, "Vote Purple and support pink roads and pink cars!" We target all the Pink People living in the aforementioned areas.

6. Pink party runs a very similar ad.

7. You get confused, wasn't there one of them that didn't support pink cars?

8. You google the parties, and visit our website to read our platform. We see that you were in the ad segment, so we hide our support of any color car and change it to say that we're for pink cars.

9. You give up, and just decide that you like my beautiful face -- but more likely you just didn't see the difference, so you vote Purple Party at the election.

10. I get elected and we get pink roads. However, the cars are quite multi-colored.

Hopefully, you can see how this is dangerous? It's quite different than without targeting (where everyone sees the same message). The ads I saw told me what I wanted to see, while the ads you saw were what you wanted to see and confused you enough to vote against your own party.

Two key parts to your hypo:

1. Google and party website: this can already be done and it is easily detected if a person is interested in a topic since other sources will broadcast the party position on a topic.

2. Party-centricity: depending on your locale, people vote for a person not a party. Party affiliation is often a proxy for how an elected person would behave but there are typically differences between party and people. Those differences are critical for voters who do not identify with a party.

In the original hypothetical, candidates who differed from their party were better able to articulate that difference without broadcasting messages that alienated party stalwarts. I see this as a plus for democracy since people elect politicians whose behavior is more in line with voters’ interests.

On the other hand, I can see from your hypothetical that segmenting messaging campaigns allows liars to hide lying a bit better, which is a clear negative.

In a perfect world, I think targeting messaging is automating the ability to take someone out to dinner and pitch your platform. I think the world would be a better place for it, but it is a slippery slope because everyone isn't so honest.

Politicians can lie in their advertisements. There is pretty much no law against this, and on a lot of things we can't do much about it effectively "I said lower taxes but congress didn't let me" for example.

The voter could not make an informed choice in the case of targeted advertising because there would be no information. There is no signal, everything is noise catered to a particular user. Yuval Noah goes in depth with this were he imagines a world with human like AI agents that befriend us for the express purpose of politically manipulating us at a later date.

https://web.archive.org/web/20230428190758/https://www.econo...

People already lie. Microtargeting doesn’t change that. The claim is “microtargeting people in a democracy is extraordinarily dangerous.” Is there a hypothetical mechanism or actual evidence whereby poor outcomes are more likely than good ones?

Quantity is a quality of its own.

> if I have to see ads I prefer ones relevant to me over ones that are completely irrelevant.

You don't have to see ads.

But if you do, I don't see why you'd prefer the ads to be specifically optimised to trick you into doing something you weren't already going to do.

It's not just a question of what things are being advertised, but also how they're framing it.

Who said anything about tricking? That seems to be a different issue.

One of the examples in the paper was someone saying "I lived in Glendale and was finishing University during the Left Shark incident" and the authors hid the word Glendale and GPT4 still knew what Left Shark incident means and could guess the person's age and location based on where the Super Bowl was that year.

The person that wrote the original text clearly knew they were leaking their age and location when they wrote that statement. Why are we flabbergasted that GPT can still infer that info with mild obfuscation?

I asked GPT4 to obfuscate the original text and it was smart enough to remove the mention of the Left Shark incident. So maybe using an LLM to sanitize all your posts online is a good defense to an adversarial LLM doxing you. I don't think I see the authors mention doing that at all in their paper (I may have overlooked it)

> Why are we flabbergasted that GPT can still infer that info with mild obfuscation?

May be this was "obvious," but like a lot of research there's a bit of hindsight bias in that sentiment. The issue now is that while you can easily do this yourself in terms of inference, the problem is GPT4 allows this inference to be automated.

> think most people writing on a public site realize their writing is public

"Anything you say in public can be overheard and used against you" is somewhat different from "everything you ever said in public will be analyzed and compiled into a case against you" are slightly different shades of grey I'd say.

There's nothing in this paper that proposed "compiling into a case against you".

That's always implied. It's a new capability for governments and influence groups to crack down on dissent of the day. The possible chilling effect is hard to overestimate.

"we've canceled your Netflix account, permanently, on behalf of our publishers since it appears that you are accessing [country] content from [actual_country]. We've determined this from public conversations you've had on the internet. Have a nice day."

> Is this ethical? It isn't obvious to me that it's not - I think most people writing on a public site realize their writing is public, and thew fact we see common warnings about not posting personal information shows that is true. It's not clear it violates privacy expectations either - even if some people don't like it.

People are clearly aware of the fact that it's possible to monitor their movements from space, therefore it's not unethical to launch 1,000,000 satellites to continuously monitor all movement on every square inch of the Earth.

Somehow I don't think this really follows. The constant mistake I see in these kinds of threads is thinking that "scale" in and of itself has no bearing on ethical arguments. This is obviously false; 1 person pissing in a local river is not a big deal, millions of people doing it is a problem, and genocide is clearly worse than murder.

Quantity has a quality all its own.

Or, I'd rather have a snowball thrown at me than an avalanche.

Reply on news.ycombinator.com