US Government building AI tech to unmask anonymous writers | Hacker News Reader

US Government building AI tech to unmask anonymous writers

theregister.com

94 pointsBender96 comments

It's becoming a question that if you have any sort of beliefs that may not be looked upon kindly by this government or future governments, maybe you shouldn't be writing anything on the internet at all. Maybe you shouldn't be texting or writing emails, either.

Over the past year, I've certainly been debating not expressing myself at all in writing. It's starting to feel very dangerous.

Expressing certain thoughts is already dangerous in some contexts. Not state-actor-is-coming-to-get-you levels of danger just yet, at least for us common folks, but it's already happening and it ain't gonna get better.

It certainly is, but anonymity and pseudonymity tend to protect you unless you get some sort of celebrity or need to communicate for your job.

Anyway, one expresses one's opinion for the sake of other people. Maybe instead of worrying about what other people think, I should concentrate on my own safety. I can do other things for people.

> if you have any sort of beliefs that may not be looked upon kindly by this government

List them. Which beliefs?

Edit: Not trying to "out" anybody's beliefs, I just don't think the govt cares.

>"List them. Which beliefs?"

I have a feeling the person you're asking won't want to answer this question!

I'm not asking for their beliefs, just an example.

Like I have said before¹², you are, in effect, asking for people to step in the bear trap in order to prove to you that it’s dangerous.

1. https://news.ycombinator.com/item?id=21633388#21659417

2. https://news.ycombinator.com/item?id=26460238#26463899

Part of my worry is that people who reply to me on the internet are looking for reasons to denounce me.

That being said, I've got a 12 year history here, you should search for whatever you need to see in order to tell me that if I weren't such a reprehensible person, I wouldn't have anything to worry about.

I'm not asking for your beliefs, just an example.

And I didn't mean to reply so aggressively, but I don't think they're relevant. Pick one of the common heresies and assume I'm suffering from it.

What heresy? I can't think of any that aren't illegal like violent threats or something but that goes beyond mere beliefs into action territory.

If you'll allow me to speculate on what I think you're getting at, it looks like you are trying gather a list of potential heresies in order to respond and remind us that none of those examples are illegal to express. And, because that speech is not illegal, there is no reason to fear the government having the ability to associate one's speech with one's real identity through these tools.

That being said, people like me worry because the government also has things like the Disposition Matrix. Sure, our speech is perfectly legal and protected by the First Amendment but there's nothing stopping some agency from classifying us as a risk and subjecting us to all sorts of who-knows-what behind the scenes.

https://en.wikipedia.org/wiki/Disposition_Matrix

Even if they are legal nothing promises that they will remain legal with the future governments.

And it doesn't even need to be governmental threats, we can already witness people being cancelled/getting their life ruined publicly for things they have said years if not decades ago on social media, things that were socially acceptable back then too.

>you should search for whatever you need to see in order to tell me that if I weren't such a reprehensible person, I wouldn't have anything to worry about.

I would... but then your username would check out.

Just look at cancel culture.

It can be anything from opposing the war in Ukraine to opposing lockdowns, or mandatory masks, etc.

Cancel culture has nothing to do with the govt. Half the govt was against masks and lockdowns.

Nobody outside of Russia and its allies is getting cancelled for opposing the invasion of Ukraine.

Funny, Tucker Carlson is still being broadcasted to millions of americans most nights. Where is this supposed "cancellation" of people who are critical of the war in Ukraine?

If what you're speaking about is free people exercising their freedom to associate (or not!) with you, everyone has a right to not associate with someone they don't like, barring very specific exceptions, and people may not like you for your opinion, meaning an opinion you have can lead to you being ostracized. For example, if my employer were to hire someone who used to be a local head of the KKK, I would likely be up in arms about that. Lot's of very conservative workplaces often feel the same about other things. Think about what happens when people who work for hobby lobby get upset about their employer's policies.

This isn't "cancel culture", it's literally humans being humans the same way we've done for 100,000 years. Hell, even monkeys make outgroups.

Given enough time, our now-virtuous beliefs might enter government's cross-hairs

So the listing you want is pretty much anything political written by anyone

I'm gonna take this bait and list a few "beliefs" that the current U.S. government disapproves of enough that some branch might dedicate some resources to investigate. I'm not even going to bother with the obligatory "it's not my belief" since you explicitly said that's not relevant.

- The 2020 presidential election was not sufficiently investigated despite unusually and statistically unlikely results and questionable legal/procedural changes and activities at the state and local levels in regions that benefitted the eventual winner.

- White male Republicans who advocate for stricter enforcement of immigration law and internationally accepted asylum processes, seek to reduce or prevent taxpayer-funded education/encouragement about non-generative sexual preferences and practices to prepubescent children, and who support the individual right to defend against a monopoly on violence by a potentially tyrannical government, are NOT threats to democracy nor are they extremists to be likened to domestic terrorists.

- Recognition of extremely strong correlation between prevalent cultural norms of any given socioeconomic demographic and the geographical crime/violence rates in which those same demographic groupings reside is not racism nor xenophobia when the recognition is aimed at addressing the cultural aspect (caveat: I accept that there is also strong correlation between those who believe this and those who apply this belief to everyone within said socioeconomic demographic - eg. "poor Appalachian opioid addicts are all unintelligent, violent thieves")

- The variations of global temperature and weather phenomenon have been in fluctuation, and at more extreme levels, long before human intervention

I'm sure I could think of a few more but the effort to phrase things properly is not worth it for the inevitable [dead].

Those are the most popular outspoken right wing identity politic beliefs that the government absolutely does not care about. They're plastered on 'news' networks and 'news' radio every single day and the FCC does not care.

Wait until you hear about the Speech-to-Text AI they've been building.

That's been part of my anti-censorship arguments for a very long time. There's no technological reason why your phone calls can't be monitored and controlled just like any other medium, such as Facebook, Youtube or Twitter.

Unlike tech companies( Facebook, YouTube or Twitter) phone companies are common carriers. There are a lot of legal reasons in the US.

Also, phone calls are 1:1. Most deplatforming is in the 1:many space

There is a treasure trove of data awaiting us on Zoom, Discord and Skype to name a few.

Do it anyways. Be assertive about your rights. They were paid for with blood.

There's a reason why "live free or die" is an expression. Your fear of death or punishment can be used to virtually enslave you. So you might as well live free, because you're going to die one way or another.

And what is probably even worse: being enslaved by a fear of punishment that never existed in the first place.

Don't be the fully grown elephant restrained by a rope only capable of holding an infant.

-Then again, not having an online presence at all is probably sufficient for you to be paid extra attention by the powers that be, so there's that...

It is already something recuiters are suspicious about. I have had to justify myself and explain that I value my privacy a lot more than I care about likes and thumbs up.

Unfortunately short of "faking" it, I don't think there is a way to not look suspicious... pretty sad.

I guess there's a business opportunity right there - surely, with capitalism being what it is, some enterprising soul must have launched a service grooming social media accounts which you can then purchase or even rent whenever you need an online presence?

That's a shitty way to live your life, especially in period when these things are doing mere baby steps. Glass is always half empty strategy.

Maybe you are uber-important, rich and powerful and you should be actually concerned about this. But chances are high (not only due to your nick 'pessimizer') that you are just making your own paranoia and depression worse for little more than nothing.

Self-censorship is very common in oppressed regimes, I've seen it damn well under soviet/russian oppression during cold era in my own country. Even people with good intentions do pretty horrible things desperately trying to not get any attention they don't want.

If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him. --Cardinal Richelieu

It does not matter what your beliefs are. If they want to hang you, they will find a reason.

For what it's worth there are also some tools out there to mitigate stylometry identification. [1] Some discussion [2][3] I do not know if using such tools would render one's writing less interesting or artistic.

There seem to be many tools on github [4] that work with and against stylometry.

[1] - https://github.com/psal/anonymouth

[2] - https://security.stackexchange.com/questions/198741/what-are...

[3] - https://www.whonix.org/wiki/Stylometry

[4] - https://github.com/topics/stylometry

If the government can do it, so can corporations. And individuals, for that matter.

There's more than just the government to be afraid of. If you're hiding from an abusive ex, they may be able to find you despite a new life under a pseudonym. No matter how many proxies you're behind, an advertiser could skip the tracking cookies and connect the dots between accounts with writing style. If you've written anything under your real name, you could get doxxed by connecting it to your "anonymous" Internet posts.

So indeed: maybe you shouldn't be writing on the Internet at all. If it's possible at all, forbidding the government from doing it will not make you any safer.

I think of Satoshi Nakamoto. He seems to have gotten away with being anonymous. But that was 10 years ago, and he did everything an individual reasonably could in terms of opsec. Even then, he’s still been narrowed down to a pretty short list of known individuals.

These days, someone who was a normal young person using the internet and social media in the 2010s has very little hope if an online mob decides to unmask them, let alone professional investigators.

I wonder if this will turn out to be something of an antipattern (for lack of a better term). The set of vocabulary in a given language is fairly limited by the human mind, enough such that it's relatively uniform enough that subtle alterations are achievable enough to the individual that they could conceivably, of their own volition, intentionally fool emergent AIs leading to false attribution. I mean, if the AI can alter the text subtly enough to fool other AIs, given the limited number of words known to other humans (which is kind of the point of speech - what use is a massive vocabulary if nobody else knows those words?), why couldn't humans do it too?

So, here we have AIs erroneously pinning the author of heretical text X as human Y, thus triggering punishment Z. All the while, it was really written by bastard A, who is really a member of political party B: the opposition of human Y's party, the C's.

Eventually, we'll need an AI that we write through, that then anonymizes our writing to make it unidentifiable.

Depends on who the audience is. If you're writing a persuasive political message, you will have to have your fingerprint on it to generate the human reaction you want.

Although if we are at that point with AIs, it could also just be trying to game other AIs into interpreting writing into what you want before it's relayed to the individual.

I find it absolutely, utterly, and completely implausible that the intelligence community just now started to research this possibility. It's been an obvious idea for decades now, and the tech has visibly obviously been roughly up to the task for decades as well. The last couple of years may have given an incremental improvement but it is completely implausible to me that they don't have workable solutions to this already. It's not like the system is useless to them until it 100% fingers exactly one person.

Don't believe it jerf.

It's obvious that this isn't the cutting edge that USG has to offer. Just a random department re-inventing the wheel.

Disclaimer: Worked on CIA / FBI stuff two decades ago.

I know that someone in the intelligence community was giving a lecture at the University of Florida on using UF’s fancy new donated Nvidia cluster to do this analysis.

This sort of writing style analysis has absolutely been used to ID pseudonymous / anonymous individuals online for over a decade.

The new twist seems to be using "AI" instead of more traditional algorithms. Personally, I would expect that to make the process more error-prone, but it makes sense to try.

> be me

> be anonymous writer

> be fearful of AI de-anonymizing my contrarian self

> idea

> write only greentexts, like other millions

> AI can't single me out

> AI defeated

IFW

Beyond the meme, this seems to be a brilliant idea - just like how we have coding standards, why not make writing standards, several levels beyond what we have currently? Constrain the "syntax" of expression (the way you communicate, as opposed to the ideas you want to communicate) to counteract fingerprinting. Sure, you'd have to restrict your creativity and prose, but you'd get anonymity in return.

All natural languages have fundamental ambiguity built in. This increases "fitness" both for the languages and those using it. Precise disambiguation requires an ungodly amount of context which, notably, we don't all share between our distinct moral communities. This is a feature, not a bug. See (and practice) lojban if you intend to disambiguate. Next, just try to imagine prose or poetry or, hell, even a compelling speech or nuanced opinion...

I agree that disambiguation requires context and that this is a feature.

Wouldn't that mean that you could write in a homogeneous, but still unambiguous, style by manually, laboriously, providing context longform, alongside your main text, written in the same homogenized style?

Instructions unclear, got a society whose language is Orwellian Newsspeak.

Not sure how this is relevant - I didn't say anything about tuning language to restrict thought or avoid anti-government sentiment...

OK, so before you publish something you give it to GPT-3 with the prompt "write the following in the style of $X". Run the resulting text through a few AIs to confirm it says that it was probably written by $X.

So, either to anonymize the text or to implicate someone, it seems like we're just in an arms race now.

"Write $POLITICAL_OPPOSITION_PIECE in the style of Hemmingway".

AI fall down go boom.

More like "Write $POLITICAL_OPPOSITION_PIECE in the style of Q"

Internet fall down go boom

"Write $PRO_ABORTION_ARTICLE in the style of Tucker Carlson followed by $ANTI_GUN_CONTROL_PIECE in the style of Rachel Maddow."

I just broke cable news!

Then BigBrother starts getting logging info from the GPT-3 based SaaS you used to do that translation. Better to run your own AI from home. I hear Nvidia has a glut of inventory you can utilize.

This has been possible if not easy to accomplish for a very long time, so calling it "AI" seems like a bit of a stretch. I suspect most people can be fingerprinted according to a handful of less typical grammatical flourishes and reuse of certain uncommon words and turns of phrase. The only real challenge is scaling that up.

Sure, but there's always room to advance the state of the art. And in the article, they mention using the adversarial AI approach, which I would say is firmly in the world of AI, and not just statistical analysis.

For years I've tried to obfuscate my own style/word choices/idiosyncrasies when writing anonymously. Not because I fear that I'm saying things my government would like to punish me for, but because I fear these tools will become accessible enough for anybody who would like to punish me to identify and harass me.

In the process, you've established another identifiable pattern.

Yeah, you can't just change from one fingerprint to another - you either have to erase your fingerprint (such that you look identical to someone else), or generate random ones and cycle through them. The anti-web-tracking people have been wrestling with this one for a bit.

Well, the point is to keep that fingerprint isolated to anonymous accounts. I don't mind if anonymous accounts are linked. Luckily I don't write much outside HN, and even less under my real name. I am, however, not under the illusion that this practice makes me safe. It's just something I do out of nervousness, since the only other option is not to write, and I'd rather not give it up.

I read about a child porn guy on the dark web once, he was an auzzie I think. He always said some strange greeting on every post. He would say hiyas.

They found Facebook posts where some dude said hiyas, did some police work and got him.

Found an article: https://www.bbc.com/news/uk-36437856

This isn't quite the same, but validates the point you are making for sure.

If they did so behind a backdrop it could yield some results. How long till anonynous writers run their text through some fuzzers or some other AI system to junble the style or emulate other’s styles, replace words with synonyms, change sentence structure and such ending up with AI systems fighting other AI systems in futility.

What you do is have your sister to re-write your posts and you re-write hers. No one will suspect you two are plotting to bring about the first Hegemon.

Easy thwart: Run a document through a couple different languages in Google translate then back to English.

The writing will be understandable if not especially elegant, and my understanding is it's like a hash: after a couple hops in Google Translate it's not possible to "un-translate" it back to the original source document. e.g. English -> Spanish -> Esparanto -> English won't yield the original document if you then run the translations in the reverse order, and it wouldn't really be possible to determine which languages it was run through in the first place using only the end result document.

Of course, I'm sure the government could just get the Google Translate logs if they really wanted to . . .

Language models are becoming more accessible all the time. I wouldn't be surprised if there's a similar system out there right now that can be run locally on a consumer grade GPU.

I think this comment is a good example of why such an AI would be nigh impossible to implement, at least in a way to be meaningful. When I read this comment, at a high level it's very original like something GPT-3 could create. But this comment has meaning that GPT-3 gibberish wouldn't. Basically I think I'm implying that such an AI would effectively be AGI.

Most people won't use a language-mixer of sorts. Do you think Apple or The Google are going to provide this as a feature on their devices? Of course not. The vast majority of people wouldn't even care to want such a feature. The whole point is to use this technology against everyday people, not the criminals who might be savvy enough to obfuscate their identity.

It is only a matter of time before some group starts developing and then expanding a jumbler of incorrect grammars, useless punctuations and poorly substituted adverbs before posting … as a plug-in/service.

Origination still remains the better datapoint class to analyze.

ungrammarly.com /s

indi.an

Damn it, someone owns that domain.

Various research groups have been looking at computational methods for determining authorship attribution over the past thirty years. One of the more valuable applications has nothing to do with anything IARPA would care about.

Unless an author is rehashing the same content over multiple publications then there isn't much that can be done to attribute writing to the same source, and even then if it's publicly available then attributing it to a singular person is dubious.

A great way to freak people out though and potentially stifle freedom of speech.

Yet shockingly there will be some other shrill complaints from the other side when the government fails to identify <future event> because, in retrospect, all the information was online. "They should have known!"

There are all sorts of reasons work on this. Mental health (think force health protection). Intel. Counter-intel. Criminal investigations.

Surprised, stylometry is not mentioned once in the article. [1] Unmasking anonymous writers has a rich history, and there are approaches writers with a large corpus have taken to conceal their writing, as well as approaches folks have taken to deanonymize those same writers.

[1] https://en.wikipedia.org/wiki/Stylometry

things like privacy protection don't apply to the Big Brother

their psycopaths have untied hands, full control on citizens and "tools" to protect themselves against wrongthinkers

I do believe a sizeable percent of all blogs and comments on the big bad internet are already generated by AI / models / etc. Especially so for the anonymous ones. So it's an AI buddy that's going to unmask lots of its AI buddies.

Also, as others have commented, it's going to be hilarious when the AI shall unmask comments written in the style of Obama or Trudeau.

But, as always, it's amazing to see taxpayer dollars hard at useful work.

This certainly feels a natural extension from being able to do it off of code already.

https://media.defcon.org/DEF%20CON%2026/DEF%20CON%2026%20pre...

I've seen this as inevitable for a while, and that's why I'm writing under my real name right here right now.

I wonder how this would fare against an astroturfing campaign that's using something like OpenAI[1] to generate human-readable text

[1] https://beta.openai.com/examples

Technology seems to create more arms races than solutions to problems these days.

Humans don't deserve the technology is classic sci-fi trope. Sadly, it's more art imitating life though. How long does it take for a society to be considered an ancient society? We've only not destroyed the planet with nuclear weapons for less than 100 years. That seems like such a baby step. Every bad thing we've done with tech since the atomic bomb has been misused at an even faster rate which seems to be gaining pace.

But what if those anonymous writers use AI to keep their writing anonymous?

If they're using the same AI for every anonymous identity, maybe that's still identifiable to an extent? But yeah.

Hopefully any such AI used by the government can't be admissible as evidence in a court of law (assuming it's neural based and not a bunch of if statements), or at least is only used in pursuing federal crimes. But with the way things are going, who knows.

I was thinking of those writing AIs that rewrite whatever you write to a consistent voice. The ones that students are using to cheat in schools nowadays.

I wonder how difficult that would need to be in practice. Obfuscation just has to be enough that it creates enough ambiguity. All you'd have to do is make things like basic punctuation, capitalization, rate of typos, advanced word count, etc. consistent. Those are things that can be "statically analyzed" from one's genuine writing and be changed on the fly without needing an AI to know the meaning of a body of text.

So will we live in a world where anonymous and pseudonymous writers hide behind AI generated translations of their original writing? Will this become a new cat and mouse game?

Cant someone also build an AI to mask anonymous writers?

Just translate the text through Google translate a few times.

"I plan on not paying my taxes" becomes "I intend not to pay tax."

Hmmm,

Not sure how effective it will be against the latest NLP tools. You can also generate writings in the same style as others.

This is another reason I wish we had more anonymous forums - truly anonymous, not pseudonymous.

Satoshi here we come

Shocked they're not already doing this.

They probably already deanonymized a large % of the internet and are using this as an excuse to ramp up parallel construction.

BUILDING? The NSA announced they have this tech and used it to identify the real Satoshi like half a decade ago lol

Cant someone also build an ai to unmask anonymous writers?

Reply on news.ycombinator.com