Hacker News Reader

Comment by keiferski | Hacker News Reader

Why does it matter if the flagging is automatic? Does that somehow make it more acceptable?

The popularity of this view makes me wish more people read Kafka. A future tyranny might end up not being Orwell's 1984 or Huxley's Brave New World, but a kafkesque nightmare where people are lost in a world of AI giving out absurd punishments. [1] Kafka's book The Castle [2] is essentially about this, although I think his aphorisms and short stories are much better than his novels.

Quoting from Wikipedia:

> The villagers hold the officials and the castle in high regard, even though they do not appear to know what the officials do. The actions of the officials are never explained. The villagers provide assumptions and justification for the officials' actions through lengthy monologues. Everyone appears to have an explanation for the officials' actions, but they often contradict themselves and there is no attempt to hide the ambiguity. Instead, villagers praise it as another action or feature of an official.

Replace officials and castle with AI and it is almost the exact same scenario.

1. This is already happening: https://www.wired.com/2017/04/courts-using-ai-sentence-crimi...

2. https://en.m.wikipedia.org/wiki/The_Castle_(novel)

View on HN

> Why does it matter if the flagging is automatic? Does that somehow make it more acceptable?

Neil Postman discusses this at length in Technopoly: The Surrender of Culture to Technology (1992):

> Naturally, bureaucrats can be expected to embrace a technology that helps to create the illusion that decisions are not under their control. Because of its seeming intelligence and impartiality, a computer has an almost magical tendency to direct attention away from the people in charge of bureaucratic functions and toward itself, as if the computer were the true source of authority. A bureaucrat armed with a computer is the unacknowledged legislator of our age, and a terrible burden to bear.

AI is a new wrinkle in an old trick to distance elected officials from accountability. After the electrical blackouts last week in Texas, the weather-worthiness of power generation facilities was framed as a technical decision that engineers made, instead of a political decision to weigh costs and benefits of regulation. When the decision is framed this way, the public's attention is directed away from the elected officials who are supposed to be accountable, and for which we have created straightforward mechanisms of accountability. Public anger dissipates in a fog of uncertainty of how these unknown, anonymous engineers could be held accountable and what kind of accountability they should actually have. AI plays exactly the same role.

Funnily enough, Das Schloss is often interpreted to be a criticism of the church and organized religion in general. The castle and its workings are seen to be like the clerical teachings, arbitrary, incomprehensible and illogical, but nonetheless highly respected by the plebs.

Eh, I think Kafka was infinitely more subtle than that. Framing the story as a mundane criticism of the Church seems to be missing much of the depth of his writing.

On that note, Kafka was also quite interested in Kabbalah, Jewish mysticism, which is almost inversely related to sociopolitical structures.

https://en.wikipedia.org/wiki/Franz_Kafka_and_Judaism

Here's a great aphorism of his that's related to religion:

“Leopards break into the temple and drink all the sacrificial vessels dry; it keeps happening; in the end, it can be calculated in advance and is incorporated into the ritual.”

How does the specific institution being ridiculed matter at all?

Whether it's the Catholic church, Youtube or the DMV the point is it's a black box of procedural spaghetti that too often produces insane results and is seemingly unaccountable.

It doesn't really matter, but it is a funny coincidence that a cleric, as a representative of one kafkaesque institution is being censored by Twitter, another kafkaesque institution.

There was a time when the clergy was Catholic, but might various "scientific experts" be filling that role today?

Great point. This can be applied to any type of "revealed wisdom" which is not obvious to the layperson. We are expected to accept their conclusions not question them.

Reminds me of the greatest scene to ever appear on TV

https://youtu.be/Zgk8UdV7GQ0?t=60

The role of the church 400 years ago is now similar to that played by the technocracy today.

Why does it matter if the flagging is automatic? Does that somehow make it more acceptable?

The problem is very simple. Unless social media companies either stop moderating content, which will kill their profits because advertisers will run away, or pay people to moderate everything, which will kill their profit because it'd be wildly expensive, they'll continue to do the half-assed version of moderating they do now using AI and actually make a profit but also make stupid mistakes like this.

Computers and people don't need to operate perfectly. Expecting that is silly. So long as companies deal with errors reasonably it's all good.

> So long as companies deal with errors reasonably it's all good.

We all eagerly await this utopia.

Or maybe expecting that is silly, given the many years of empirical evidence?

There’s plenty of counterexamples one who might wish to dismiss the idea that technology can reasonably manage things, but all things considered, given the number of _potential_ instance of mishap that do not regularly occur any the relatively minor consequences of the mishaps that do occur, I’d conclude that on the whole arguing the original poster’s position is not inherently ‘_silly_’.

> There’s plenty of counterexamples one who might wish to dismiss the idea that technology can reasonably manage things

Are there? How many historical precedents are there for "curation" at the scale of today's tech giants: Facebook, Twitter, Google, Apple App Store, Amazon.com? IMO none of them do it well, or even reasonably.

> given the number of _potential_ instance of mishap that do not regularly occur

We only hear about the "notable" cases. Every day, people are accidentally or intentionally banned by Twitter or one of the other tech giants, and nobody ever hears about it, because these people are not famous and have nobody else to speak for them and raise a public fuss.

> any the relatively minor consequences of the mishaps that do occur

Regardless of one's opinion on the matter, you can hardly call the banning of the POTUS, for example, from the world's largest and most important social networks as a "relatively minor consequence". This is very consequential. You may think the consequences are good, or you may think they're bad, but it's consequential.

If these "mishaps" are not consequential, why are we all here talking about them?

>> stop moderating content, which will kill their profits because advertisers will run away,

One common trend for decades has been that "conservative" advertisers quickly acquiesce once they see that their customers do not share such values. Once upon a time major brands wouldn't go anywhere near sexualized content. Now coke/pepsi/ford will pay millions to get their into a Miley Cyrus video. Once upon a time major brands didn't want to be associated with "shock jock" radio hosts. Then Howard Stern made that acceptable and they all jumped on board. Advertisers today don't want to be associated with unmoderated websites. That too will change as such websites become a norm.

Sure, 1922 Coke advertising used women in a swimsuit, that’s about as sexualized as a Miley Cyrus video relative to the time period. If anything their very consistent in targeting various segments over time.

That said, US social norms get more and less restrictive over time. Dueling for example used to be perfectly legal as was harsh physical punishment for children. What we think of as progress is arguably just increasing similarity with our current views. Continue the same mutability and 2200 might seem much worse to us than today is, but they would similarly think of right now as unbearable.

>> 1922 Coke advertising used women in a swimsuit

There was far more sexualized content available in the 20s. A girl in a 20s-style 'swimming costume' was the the advertiser acceptable limit at the time for mass media. A Miley Cyrus video would be the sort of content available at 1920s peep shows in bad neighborhoods, content from which advertisers then stay far away. Today they readily endorse it.

Those peep shows where closer to strip joints/late night cable than Miley Cyrus. Full frontal nudity etc.

She’s a little past prime time TV, but hardly porn.

* Advertisers today don't want to be associated with unmoderated websites. That too will change as such websites become a norm.*

Or, to put that another way, stopping moderation would lead to a few quarters of negative growth, which would cause shareholders to riot, try to sack the board, etc, and that is exactly why it'll never happen.

> Computers and people don't need to operate perfectly. Expecting that is silly. So long as companies deal with errors reasonably it's all good.

That depends a lot on the lead time for dealing with the errors, and the magnitude of the negative impact the error has.

The lead time for Google dealing with their errors seems to average just below infinity. Twitter, admittedly, is quite a bit better.

> or pay people to moderate everything, which will kill their profit because it'd be wildly expensive

This gets repeated ad nauseam whenever excessive automated moderation is questioned on big platforms, but I have yet to see any actual evidence of it. I don't believe for a second that the likes of Google, Facebook, or Twitter can't afford to turn down their algorithms and significantly increase their manual moderation efforts.

Platforms like Reddit and Discord get along alright with limited automated moderation combined with a federated mod structure where most community-specific moderation is handled by unpaid volunteers. Those platforms are gigantic, but they don't make the front page of HN daily for illogical, heavy-handed moderation practices.

Besides, nobody expects all moderation to be 100% manual. A lot of content can be reliably identified and moderated by an automated system. Automated systems can even still help with things which they cannot be certain about. Instead of removing the content and taking punitive action against the user, an automated system can just bring the questionable content to the immediate attention of an organization of humans.

Not to mention we're talking about some of the wealthiest companies in the world here. Do we have any idea how much they currently spend on manual moderation? Professional content moderation isn't exactly a highly compensated position. I suspect they could increase their moderation workforce by a couple orders of magnitude without the slightest concern about breaking the bank.

The big social media platforms complain that significant manual moderation impossibly expensive, but they'll say just about anything to save a dime. That's the real reason they rely so heavily on automated systems - to save as much money as possible. As long as it doesn't cut into their revenue, they'll keep doing it. And if people buy their crap about it being impossible, that suits them just fine.

>Platforms like Reddit and Discord get along alright with limited automated moderation combined with a federated mod structure

Reddit is constantly criticized for the effects of its moderation policies, both in how arbitrary they can seem, and how they allowed genuinely law-breaking content to stay up at the discretion of those "volunteer" moderators. They are practically constantly in the news.

(Discord exclusively has invite-only communities, so it's a totally apples-to-oranges comparison)

While I would say that reddit's behavior is at least somewhat preferable to the topic of discussion, claiming it's without a similar level of criticism just means you don't actually use the platform.

It's also worth pointing out that what others have voiced regarding AI, that such a strategy just serves to deflect blame from the company, is just as true when relying on community volunteers. I suspect that the main reason that they don't make the front page of HN specifically is because "human makes a bad decision" is several orders of magnitude less interesting to this site than "AI makes a bad decision", regardless of the relative social harm.

This is like the negative opposite of the happy coincidences that need to happen for a startup or new idealistic organisation to take off:

The founder(s) have a crazy, innovative, idea to improve the world, and while they're perceived as mad by the masses initially, it's the first followers who give credence and legitimacy to this idea. So the first employees, initial customers, initial investors, and even just random supporters who independently believe in the value what the originators are doing, and that helps propel acceptance within the larger community and eventually the masses.

This is the authoritarian opposite - really terrible decisions and mechanisms have been implemented and are being repeated on a daily basis by powerful companies or organisations, and while the masses find it worrying initially, there are apparently no shortage of apologists online and offline who are falling over themselves to helpfully explain to the rest of us why this crap is ok:

"It's an AI algorithm"

"They're a private company"

"You don't have to use them"

"Well, they're standing against bigotry"

etc

And then suddenly we find ourselves in an environment not unlike the Kafka-esque scenarios referenced. Be wary of "explainers" - often just they're consensus building, but it's not clear whether they're building in the right direction or the wrong one.

Or Gordon Dickson: https://www.atariarchives.org/bcc2/showpage.php?page=133

I remember reading this years ago, and it's still as bonkers as ever.

Slightly related:

https://en.wikipedia.org/wiki/Brazil_(1985_film)

"One day, shortly before Christmas, a fly becomes jammed in a teleprinter, misprinting a copy of an arrest warrant it was receiving resulting in the arrest and accidental death during interrogation of cobbler Archibald Buttle instead of renegade heating engineer and suspected terrorist Archibald Tuttle."

how prescient that it was a bug :)

"Do not fold, spindle or mutilate" was written on IBM punch cards.

One difference is that Twitter, Facebook, Google, are not governments and have no real authority over you. They only have the power you give them by choosing to stay in their ecosystem. Therefore it's a strain to refer to them as officials.

If you're worried about them _becoming_ officials, then act now. Leave these platforms and convince others to do the same.

Well, in a literal sense, no. But they certainly have the ability to:

- Exclude you from the largest (and nearly monopolizing) media channels. Good luck getting an online business off the ground when an AI filter bans you from YouTube, Facebook, Twitter, and Google search results. And good luck contacting Google support to contest it, which is notoriously absent for virtually everyone.

- Have you socially ostracized by labelling you as X bad thing, all without you having any recourse or ability to contest the designation. The fact that stuff like this is legal blows my mind: https://www.cbsnews.com/news/yelp-racist-alert-added-busines...

- Collude to affect government policy and prevent competitors from gaining any footholds. Typical BigCo stuff.

Sticking to the consumer-side of the discussion, the BigCos aren't going to break the cycle. In fact, it's a capitalist crime to do so. Their flywheel requires a steady cattle population to surveil in the fields. When we choose as individuals to remain in their ecosystem, we choose the hidden fees of surveillance capitalism over more straightforward payment models. This is what gives BigCos the muscle to wield against smaller capitalists.

We in our consumer role, by not participating in these business models, are the only truly free actors in the system at this time. The only other way out is for government to get bigger and step in, as it's doing in Australia now.

The government could do plenty of things (using laws written a century ago in the Teddy Roosevelt era) to encourage competition and break up de facto monopolies. They've simply been asleep at the wheel.

> If you're worried about them _becoming_ officials, then act now. Leave these platforms and convince others to do the same.

the assumption that we do still have agency, and can exercise power as a collective "by voting with our feet/wallet" is wrong. we (some isolated heroes and activists who care) might be able to throw some spanner into the works or otherwise help things unravel at a slower speed. But things will still unravel in the same way as Jacques Ellul documented and predicted in "La Technique"[0].

To get rid of them is not possible because the only way out we can imagine is regulation which will further cement their position and status quo. There is no future in which these companies and power structures will not be around. I think it is more likely we see the end of the world than to envision a world where these platforms don't exist (maybe not among everyone my age but certainly among everyone 20 years younger than myself who has never experienced society without the Internet/Web)

[0] The Technological Society https://archive.org/details/JacquesEllulTheTechnologicalSoci...

Yes - but until such a time as we have a healthy ecosystem and interoperability, they are a defacto monopoly and I would assume the "anti-monopoly" people would just do something about it, if only to promote an ecosystem. Instead we live in this weird limbo where we can't criticize these platforms because "it's a private company" whilst still using it as if it's a government/society-level ubiquitous platform.

> One difference is that ... Google ... are not governments and have no real authority over you.

Legally-required quarantine-checking apps or contact-tracing apps in some countries have only been made available through the Apple and Google Play stores. The Android apps often require Google Play Services, which require a Google account.

The same will likely be true of the vaccination-certificate apps that are being planned in some countries (countries intend to allow you to generate limited-time QR codes, not present a permanent paper certificate).

Consequently, staying on the good side of supposedly private platforms is increasingly necessary to be an ordinary, law-abiding citizen.

> If you're worried about them _becoming_ officials, then act now.

If you mean this figuratively, we already have many situations - going back at least to the Arab Spring - of government officials directing policy on social media sites. Secretary Clinton of the Obama Administration spoke about this many times.

If you mean this literally, check out how may high level policy types from social media companies came from and have returned to high ranking spokesman+policy roles in Biden Administration.

But fundamentally, it doesn't require being an official. Even filtering and shaping search results has an impact.. and this has been measured repeatedly. Here's a study from five years ago (aka pre-Trump) which demonstrated it:

https://aeon.co/essays/how-the-internet-flips-elections-and-...

We don't live in isolation, and most of us couldn't. If we want to be part of the social fabric, we're often forced to use these services to some extent, so they have real authority over us.

> future tyranny might end up not being Orwell's 1984 or Huxley's Brave New World, but a kafkesque nightmare where people are lost in a world of AI giving out absurd punishments.

I think the present for some people is already a synthesis of all three and if we're not careful it will be for everyone.

Automatic flagging just means their algorithm is crap. Which is bad, but it's a fixable technical problem in the medium term and the post can be restored in the short term.

If this were deliberate flagging it would mean they were way outside of any reasonable interpretations of their guidelines and users really could no longer post anything and be confident it wouldn't run afoul of those interpretations. This is much harder to fix, and the confusion it wod cause is significantly higher.

Ah... I'm sorry to say, ihsw, but you've been shadowbanned.

The technical problem isn't fixable without AGI.

The vagueness and ambiguity are features, not bugs -- it permits arbitrary punishment. Personally I'd rather they skip the appearance of reasoned arguments and go straight to "Twitter does not approve of this message."

No muss, no fuss.

Alright, let's say you get appointed at Twitter to clean up this mess. You get full authority to do whatever needs to be done.

How would you solve this? Your solution will be criticized by the other HN-ers here :D.

>Alright, let's say you get appointed at Twitter to clean up this mess.

Announce a sale of intellectual and physical assets and close down. I would realize the company is a negative drain on society.

So basically some other platform will take over?

No, break up all the tech giants, and prevent monopolistic platforms from ever arising.

A platform so gigantic that it's impossible for humans to intelligently manage it should simply not exist.

You do realize the network effect right?

You mean that a bunch of your friends is on one platform, and a bunch of others on another platform, and your family on still another one.

But then you have to be on all platforms, so this means everyone else also needs to be on all platforms.

So how exactly do you see this working when reality pushes everyone to use a single platform? because you know, it's a social network.

1) My family is on Facebook. I am not. ;-) It's already the case that Facebook, Instagram, and Twitter for example have different "crowds", though there's some overlap.

2) Somehow humanity survived without these networks pre-2007ish.

3) I have no problem with the existence of many smaller special-interest social networks. Those are manageable, and we had them before the rise of the BigCos. The problem is with the existence of a few all-encompassing, world-consuming, general-purpose social networks. They have proven to be completely unmanageable and toxic.

"Abolish Gmail", for instance, doesn't mean abolish email providers. Not at all. And it doesn't mean that "the next gmail" will replace it. Healthy competition is good, unhealthy monopolies are bad.

Your point 1 contradicts your monopoly theory.

Your point 2 says you really want to join the Amish.

You mean smaller platforms like Parler or 4chan?

> Your point 1 contradicts your monopoly theory.

What is my "monopoly theory"? I clarified what I meant: "A platform so gigantic that it's impossible for humans to intelligently manage it". We don't really have a good term for a corporation that has undue, oversized market power. Pedants like to argue "so-and-so is not literally a monopoly!" but this is just wordplay and doesn't really address the serious social issues involved.

> Your point 2 says you really want to join the Amish.

Maybe!

> You mean smaller platforms like Parler or 4chan?

I mean smaller platforms. It would be ridiculous to suggest that your cherrypicked examples are representative. You can make anything look bad by cherrypicking.

Hmm. Obviously I don't have a perfect solution, but I've often thought that a "tagging" system would work better than a "moderation" system.

It would work like this: by default, nothing is moderated. Instead, every post/tweet/etc. has to be labeled by the user. There can be an auto-suggestion feature to make it a quick process.

Then, other users can choose which tags they want to view or hide. If you don't want to see content tagged with X, no problem. If you do, you can.

In my mind, this would solve a few problems:

1. Justifiably ban people that try to circumvent the tagging system. You aren't determining whether their speech is allowed or not, merely ensuring that they follow the very basic rules of the site. This is much simpler than relying on an AI to interpret a nuanced comment and then moderate it.

2. Allow everyone to say whatever they want. No one will feel silenced.

Again, I'm sure there are holes in this idea, but I think it might be a better approach.

Tumblr's tagging system is quite similar to this - you can "follow" tags or "blacklist" them and most people do tag pretty extensively - and it allows people to get away with some things that would be banned on twitter, like gore blogs. On the other hand, Twitter allows nudity, while Tumblr does not, and they both still ban for "hate speech."

Right, it seems pretty implementable from a technical perspective. The hard part is having a company put I disagree with what you say, but will defend your right to say it in their company philosophy.

It should be criticized, why do you imply that is a negative? AI is just a tool, and it can be used incorrectly. In this case we have a policy that seems to be applied incorrectly. That is a simple problem to solve, require more review, or the ability to escalate. Don't make such decisions "final".

Criticing is easy when you don't have to provide any solutions.

Expecting Twitter not to have their hands full with these kinds of issues is very naive. It's always a question how to spend your very limited resources.

Migrate to a federated network, put up a large barrier for the ability to post and have employees moderate it. Facilitate administrators creating their own user moderated nodes and force users to opt into someone else's node if they want to read or post in them.

document the filtering/ranking widgets and give the users their own knobs to control them.

> Why does it matter if the flagging is automatic? Does that somehow make it more acceptable?

Since the action (for “encouraging or promoting suicide”, bizarrely) was upheld on human review, whether the flagging was automatic or not is obviously irrelevant.

Clearly, this action is unwarranted and the Bishop should be entitled to a full refund of monies paid to Twitter for the privilege of having messages distributed on the platform, at a minimum from the point of the unjustified ban.

It allows those responsible for these events to feign lack of agency. The Third Reich was enabled by convincing people that they had no agency and demanded that they adhere to the will of the bureaucracy - even when that meant genocide. Not saying that Twitter's censorship activities are comparable in terms of outcome, but rather that the same underlying mindset is at play here... of course the big tech censorship is likely to result in much more troubling outcomes in the future.

I think the argument that private business is going to be the death of free speech is a strawman argument of the right. Typically this demographic goes out of its way to ensure a healthy free market with little obstructions.

However they seem to lack the technical experience to launch a (successful and secure) competing social network of their own. So instead of achieving technological parity and becoming valid competition, they instead aim to impose regulations on the entire industry. It's ironic and disappointing.

>Typically this demographic goes out of its way to ensure a healthy free market with little obstructions.

You are kidding right ? Private businesses goes out of their way to obstruct the market. Google being exemplary proof of this.

How does AWS caving to social pressure to fuck over Parler fit your narrative?

If parler knew what the were up against they would have had on prem infra. Parler did not have technological parity with Facebook.

It's embarrassing to say you're going to beat Facebook at their own game and then create a business model that's unsustainable from the very beginning.

Uh, wat? Parler's directly competing with Twitter.

Why are you commenting when you have no idea what you're talking about?

Have you forgotten about Parler?

They obviously did not have technological parity with Facebook, did they?

Reply on news.ycombinator.com