Counteracting abuse of this site is the #1 thing we do behind the scenes to try to prevent the value of HN from eroding. That's actually what I spent the first hour of my morning doing, before I realized that there was $BigDrama happening. (Thank you, bat-signaling emailers.) If you ever see me commenting on how "large HN threads are paged for performance reasons, so click the More link at the bottom, and we'll eventually remove these comments once we turn off pagination", well, the reason that's not done yet is because moderation takes 90% of my time, answering emails takes the other 90% of my time, and counteracting abuse takes the other 90% of my time.
The better HN gets, the more people want to suck its juices for their own purposes. Most haven't figured out that the above-board way to do that is simply to make interesting contributions, so they do other things, and there's probably a power law of how sinister those things are. The majority are relatively innocuous, but lame. (Think startups getting their friends to upvote their blog post, or posting booster comments in their thread.)
Users are good at spotting these innocuous/lame forms of abuse, but when it comes to $BigCo manipulation (or alleged manipulation), user perceptions get wildly inaccurate—far below 0.1%—and when it comes to $NationState manipulation (or alleged manipulation), user perceptions get so inaccurate that...trying to measure how inaccurate they are is not possible with classical physics. Almost everything that people think they're seeing about this is merely imagination and projection, determined by the strong feelings that dominate politics.
How do I know that? Because when we dig into the data of the actual cases, we find is that it's basically all garden-variety internet user behavior.
It's like this: imagine you were digging in your garden for underground surveillance devices. Why? Well, a lot of people are worried about them. So you dig and what do you find? Dirt, roots, and worms. The next time you dig, you find more dirt and more roots and more worms. And so for the next thousand places you dig. Now suppose someone comes along and insists that you dig in this-other-place-over-here because they've convinced themselves—I mean absolutely convinced themselves, to the point that they send distraught emails saying "my continued use of HN depends on how you answer this email"—that here is where the underground device surely must be. You've learned how important it is to be willing to dig; even just somebody-being-worried is a valid reason to dig. So you pick up your shovel and dig in that spot, and you find dirt, roots, and worms.
Still with me? Ok. Now: what are the odds that this thing that looks like a root or a worm is actually a surveillance device? Here my analogy breaks down a bit because we can't actually cut them open to see what's inside—we don't have that data. We do, however, have lots of history about what the "worms" have been doing over the years. And when you look at that, what do you find that they've been up to? They've been commenting about (say) the latest Julia release or parser combinators in Elixir, and they've been on HN for years and some old comment talks about, say, some diner in Wisconsin that used to make the best burgers. And in 2020 they maybe got mad on one side or the other of a flamewar about BLM. (Nobody please get mad that I'm using worms to represent HN users. It's just an analogy, and I like worms.)
Or, maybe the history shows that the person gets involved in arguments about China a lot. Aha! Now we have our Chinese spy! How much are they paying you? Is it still 50 cents? I guess the CCP says inflation doesn't exist in China—is that it, shill? If @dang doesn't ban you, that proves he's a CCP agent too!
But then you look and you see that they've been in other threads too, and a previous comment talks about being a grad student in ML, or about having married someone of Chinese background—obvious human stuff which fully explains why they're commenting the way they are and why they get triggered by what they get triggered by.
This ordinary, garden-variety stuff—dirt, roots, and worms in the analogy—is what essentially all of the data reduces to. And here's the thing: you, or anyone, can check most of this yourself, simply by following the public history of the HN accounts you encounter in the threads. The people jumping to sinister conclusions and angrily accusing others don't tend to do that, because that state of mind doesn't want to look for countervailing information. But if you actually look, what you're going to find in most cases is enough countervailing information to make the accusations appear absurd...and then you'd feel pretty sheepish about making them.
I'm not saying the public record is the entire record; of course it isn't. We can look at voting histories, flagging histories, site access patterns, and plenty of other things that aren't public. What I'm saying is that, with rare exceptions [1], what we find after investigation of the private data is...dirt, roots, and worms. It looks exactly like the public data.
And here's the most important point: the accusations about spying, brigading, shilling, astroturfing, troll farms, and so on, are all exactly the same between the cases where the public data refutes them and the cases where the public data is inconclusive. I realize this is a subtle point, but if you stop and think about it, it's arguably the strongest evidence of all. It proves that whatever mechanism is generating these accusations doesn't vary with the actual data. Moreover, you don't need access to any private data to see this.
There are also trolls and single-purpose accounts that only comment in order to push some agenda. That's against the HN guidelines, of course, and such accounts are easy enough to ban. But even in such cases, it doesn't follow that the account is disingenuous, some sort of foreign agent, etc. It's far more likely that they're simply passionate on that topic. That's how people are.
[1] so rare that it's misleading to even mention them, and which also don't look anything like what people imagine
---
Still, power laws have long tails and one wonders what may lie at the end, beyond our ability to detect it. What if despite all of the above, there is still sinister manipulation happening, only it's clever enough to leave no traces in the data that we know of? You can't prove that's not happening, right? And if anyone is doing that it would probably be state actors, right?
You might think there's nothing much to be said about such cases because what can you say about something you by definition don't know and can't observe? It seems to get epistemological pretty quickly. Actually, though, there's a lot we can say, because the premise in the question is so strong that it implies a lot. The premise is that there's a sort of Cartesian evil genius among us, sowing sinister seeds for evil ends. I call this the Sufficiently Smart Manipulator (SSM): https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so....
There are two interesting things about the SSM scenario. The first is that since, by definition, the SSM is immune to anti-abuse measures, you can't postulate any technical measures for dealing with it. It's beyond the end-of-the-road of technical cleverness.
The second interesting thing is that, if you go in for this way of thinking, then either there already exists an SSM or there eventually will be one. And there's not much difference between those two cases. Either way, we should be thinking about what to do.
What should we do in the presence of an SSM? I can think of two options: either (1) give up, roll over, and accept being manipulated; or (2) develop a robust culture of countering bad arguments with better ones and false claims with true information. Of those options, (2) is better.
If you have such a culture, then the SSM is mitigated because the immune system will dispose of the bad parts of what they're saying. If there are any true bits in what they're saying, well, we shouldn't be rejecting those, just because of who said them. We should be big enough to accommodate everything that's true, regardless of where it comes from—just as we should reject everything that's false, regardless of where it comes from. We might prefer to reject it a little more rudely if we knew that it was coming from an SSM, but that's not a must-have.
The nice thing is that such a culture is exactly what we want on HN anyway, whether an SSM exists or it doesn't. The way to deal with the SSM is to do exactly what we ought to be working at as a community already: rejecting what's false and discovering what's true. Anti-abuse measures won't work forever, but we don't need them to—we only need them to last long enough to develop the right habits as a community. If we can reach a sort of (dare I say it) herd immunity from the viruses of manipulation, we'll be fine. The answer to the Sufficiently Smart Manipulator is the Sufficiently Healthy Community. That's what the site guidelines and moderation here are trying to nurture.
Edit: I should add that I'm not 100% confident that this can work. But it's clear that it's the best we can do in that scenario, and the good part is that it's what we ought to be doing anyway.