Hacker News Reader

Comment by rossdavidh | Hacker News Reader

I think FB has a fundamental issue here, which is that:

1) they don't feel they can NOT police content

2) they are wayyyyy too big to police content manually

3) any AI they train to police content, is not going to be able to state clearly why a given post/account was banned, and anyway...

4) if you state objective, clear rules that are automated, it will be pretty easy to game them (e.g. using euphemisms or code words for objectionable content)

I don't see how FB can get out of this situation. Any social network that is as large as it is, which relies on advertising (and thus cannot be as content-neutral as e.g. phone companies are regarding what you talk about on the phone), is going to face the same situation.

Of course, if FB ends up getting way smaller as a result, that would be fine by me. But I'm not holding my breath...

View on HN

> 1) they don't feel they can NOT police content

True.

But we should be explicit here-- FB isn't some neat, nerdy social graph software that just happened to get really popular, like git for regular people or something. They have a business model to shake up their social graph to maximize engagement, and the implementation achieves this by keeping everyone maximally mad as hell at each other. It spreads conspiracy theories efficiently. It and its users are easily exploited, and they continue to be exploited by any nation-state, nefarious actors, or even just some dude looking to make a buck spreading fake news. That all feeds into the same destructive behaviors for the users as the system desperately tries to keep them engaging more and more. Facebook's own software and UI exacerbates these problems.

With that starting point in mind, I agree with your 1, 2, 3, and 4 above. But this is less like email having to deal with new sets of problems as it scaled to the entire world. It's more like a drug cartel trying to figure out the maximum amount of their fundamentally destructive product they can sell before the government finally decides to comes in and break up their enterprise.

Censoring content rather than refuting its claims is counterproductive. If skeptics are not able to be argumentatively refute a claim, they will not succeed in changing minds. The current strategy of trying to silence view points will eventually fail miserably.

The issue is that when a platform outsources its censorship (“moderation”) to a third party that it still controls, rather than a third party that is actually perceived to be neutral by most people, then you create distrust which further fuels conspiracy theories. Also a conspiracy theory may actually be true. Jeffrey Epstein was not discussed during the 2016 election aside from a few fringe articles. It was too toxic for mainstream to touch. And everyone in Hollywood knew about Harvey Weinstein and even joked about it. These topics were covered up only for so long. Today there are lots of insane and farfetched theories being floated. But the theories being censored today that are indeed based in reality, will eventually become common knowledge and the censors will be implicated in the coverup thereby harming their trust.

Discussion, debate, argument, evidence, analysis lead us towards truth. Mute only appears to work and can be a useful political tool for only so long.

Today our public forum is virtual, but still a public forum. Trump wasn’t allowed to block certain followers because of their right to be in the forum and hear from their president.

So the question is whether governments will acknowledge this fact and limit the power of our big tech group communication platforms and enforce transparency and non-biased appeals process. Or will they cede total control of our public forums to Big Tech out of short term political convenience.

I'm talking about lead in the gasoline and you're talking about more speech.

I don't disagree with your point but it isn't relevant to mine.

Totally fair

I'll take that at face value.

I'll only add that it's going to be difficult as hell to stay on topic as FAANG pushes back against attempts to deal with their monopoly practices. Uber and Lyft effortlessly destroyed the original meaning of "ride-sharing" for their own interests. Expect Google, Facebook, and Twitter to put all their weight behind narrowing the meaning of free speech to simply mean ensuring everybody has equal access to be manipulated by their commercial blackboxes.

> will eventually become common knowledge and the censors will be implicated in the coverup thereby harming their trust.

But by that time almost no-one would care. And this is probably what they're counting on.

Probably true. And can always spin it later as well, especially when people subliminally need to agree with the spin so as not to explode their own head by challenging the very basis of all their thinking.

I think your point is valid, and relevant. Well stated.

Having said that, I think if FB were to stop doing that, some other social network would, and would replace FB the way they replaced MySpace. So again, I think the current "we want one social network for everyone" situation, inevitably produces this kind of behavior.

But, I agree with your point, and it is a good one.

Thank you for stating these points. I believe that FB is going to be cut up due to these issues and the interplay with politics, just like Bell was.

Per my views on the situation, Zuck is cooked. When he went before congress to testify, he was being given the classic robber-baron deal that the US has used a few times now. As an analogy to actual robber-baron, at first when the railroads were new, Congress didn't care much about them. Then when all the businesses and politicians started to use them, they started to notice that the railroads were very important. When beef suspiciously didn't get to Chicago on time, or when trains delayed representatives to important votes up in Albany, then the politicians got legislative. A similar scaling of importance has occurred with big tech.

The US then typically takes these newly vital sectors and then gives them the deal: Fix it or face jail time. Compliance means that your family for countless generations is never poor again. You go kite-surfing for the rest of your days. Congress works with you to write the laws in such a way that you get a monopoly on the sector with a nice 3% above inflation growth baked in. Congress and the US get a sector that isn't scammy and just works (albeit poorly at times). Congress gets you to testify for a few days, ensures that you have lied to congress in that time, and holds that jail time over you as leverage to comply.

Zuck was given this deal.

Per the points OP outlines, Zuck cannot follow through on his end of the bargain. He cannot get FB to work the way Congress is demanding that he do. Zuck may not be facing jail time (yet), but may when he gets in front of Congress again.

After Dec. 37th / Jan. 6th (because 2020 never ends), Congress likely sees Big Tech as a direct personal threat (not existential) and is likely to move directly against that threat with due haste.

While I think Mark Zuckerberg is unlikely to face jail time, there are some points to your post I agree with. Especially the analogy with railroads.

The other point I would add, is that it's not just Congress. I have heard government leaders in Mexico, Poland, Hungary, and Turkey all voicing similar concerns, and in some cases actually passing legislation already. I think more will come.

> 2) they are wayyyyy too big to police content manually

To be more exact: they do not want to spend money on competent policing. They probably have capability to employ ten times as many human moderators as they do now, thus reducing the blatant error rate (at least in the case of highly influential pages and people), but that would eat into their profits.

Assuming a user:moderator ratio of 10,000:1, they would need to hire 200,000 moderators to cover their 2 billion users. Then at that scale, they’d need to hire thousands of meta-moderators and meta-meta-moderators too.

To muse a bit: a city of 2 billion would need a lot more administrators than just 200 000.

Ofc, Facebook does not provide nowhere near as many services as a city would, but ... it still seems to me that current user:moderator ratio is very obviously insufficient.

In the early 2010s, this sorta worked, because the effects on real world politics were insignificant. Now everyone looks at their fingers and they have a very underinvested infrastructure and no clear idea what to do.

Taking a random large city as an example, New York City has over 300k employees [1], which is a 25:1 ratio with population. The Police department has 55k employees [2], which is a 151:1 ratio with population.

[1] https://en.wikipedia.org/wiki/Government_of_New_York_City

[2] https://en.wikipedia.org/wiki/New_York_City_Police_Departmen...

This is the incisive observation on this topic.

No society with numbers north of somewhere in the two digits works without some kind of rules, administration, and enforcement. I'll bet you there are at least half a million people staffing courts in the US, maybe even a full million, and probably an equal number of law enforcement officers, and that doesn't get to public and private attorneys which is probably another million.

Which means something like 1% of the population of the US is probably involved in just administering the laws of society.

Now, maybe for a society where the limits of how people interact is communication you could get away with lower overhead. But that's going to be a constant factor of the same underlying dynamic.

Also, we've done the same damn thing with freedom of discourse that we have with almost every other liberty in the US: we turn it into a thought-stopping cliche and glide on treating it as an indulgence rather than investing in thinking about the limits, edge cases, and responsibilities.

There’s no need for a specific class of meta-mods. Just use the same system as Slashdot: periodically throw a random moderator’s decision in front of another mod. Ask them if the decision was fair. A portion of the bad decisions will get reported and screened.

If all mods are routinely screening each other’s work, everyone has an incentive to be fair. Biased mod decisions are thus a risk to your continued employment because you know your work is spot-checked.

If meta moderators are at the same scale then that’s only 200 meta moderators and only 1 meta meta. The base of a log scaling pyramid is approximately as large as the entire pyramid.

Wouldn't that be 20 meta moderators (10k:1 scale), rather than 200?

You’re right, apologies!

Where's the problem in that? FB makes 18.5 billion $ a year in profit. 250k employees @ 30k would work out to 7.5 billion $ in cost a year, it would hardly leave them bankrupt.

Also, Facebook could offset a part of that cost by charging users a token amount - say, five dollars or something like that.

First, it’s $7.5 billion freaking dollars. Of OpEx.

Second, it’s 250k human moderators who will feel obligated to, you know, moderate stuff. They’re not paying $7.5 bil to just allow people to post whatever content they want now! Very quickly they will run out obviously clear-cut cases and will be moderating anything and everything under the sun.

How anyone could think 250,000 people running around with mod powers on people’s Facebook walls is not deeply, deeply problematic... I cannot imagine.

> First, it’s $7.5 billion freaking dollars. Of OpEx.

So what? If that cost is necessary to run a platform that is safe for those who are on it (= from spammers, catfishers, threats, pedophiles or right-wing extremists) and that does not serve as a breeding ground for threats for society (= "freemen"/"Reichsbürger" groups, militias, neo-Nazis, antisemites, Qanon, ...), then this cost should be paid by the ones making the profit (=Facebook), not society.

> Second, it’s 250k human moderators who will feel obligated to, you know, moderate stuff.

So what? The digital equivalent of police(wo)men on patrol. Just because it's The Internet, it should not be a free-for-all land.

> How anyone could think 250,000 people running around with mod powers on people’s Facebook walls is not deeply, deeply problematic

The US alone has nearly 700k police officers for 300M people, that are running around with guns and regularly kill people, and yet society accepts this.

If FB charged money, it would totally change their business model, and I believe that could solve some of the issues. However, I have to wonder if people would actually be willing to pay money for FB? It's easier to quit something when you just have to stop paying them. They might get a lot smaller if they charged money.

I think it's far worse than even this. Facebook reportedly receives over 50000 posts per second on average. With 200000 moderators, each one would have to read a post every 4 seconds every hour of every day. I would place the number of human moderators at least an order of magnitude higher.

I don't want strangers looking at the content I post to my friends. Full stop.

Snowden made it pretty clear that they already are.

> already were

Fixed that. Maybe some nefarious three-letter agency found another way, maybe not. Even if they did, they have a powerful incentive never to let it be known. And it's still almost entirely algorithmic.

On the other hand, human moderators are inherently going to be entry-level individuals in a call center in a low-wage country. These are not the same things.

How would you feel about human moderation of your email?

Seems logical if you're using their stuff, they get to do anything they want with what you put there. Expect them to read every word, and use it to make money.

You are right that this is the status quo.

But I do have to say that it is neither how the law works in many places, not is it ethically correct.

I agree with you, which is why I don't use platforms like Facebook at all.

3 and 4 are easy - have a manual review of automatically flagged content. You get clear reasons, better resolution of false positives, and it’s not simple to game. And, in another plus, it can feed back into the AI training.

I think part of the problem at FB's scale is getting any level of consistency across human moderation.

If you ask 10 good, experienced, human moderators whether a controversial post is allowed, you'll probably get at least 15 different answers and justifications.

FB would probably find it harder to train from this, as each moderator will be using different justifications, and reaching different decisions.

At FB's scale, they want something cheap and straightforward which they can implement automatically and go Kafka on the lack of appeals. The reality people need to wake up to is that when a business is at FB scale, people who fall through the cracks due to a broken algorithm simply don't matter. Google doesn't care if you get locked out of your Gmail, in the same way FB doesn't care if you get spuriously banned.

When users place more value on a service than the service places on them as a customer, it feels like a market failure scenario that leads to these kinds of outcomes. When the go-to price is $0 though, it isn't clear how you resolve this.

Any justice system also has a variety of different judges. Of course in an ideal world there would be perfect consistency, but imperfect consistency would still be a lot better than just letting an algorithm decide. Of course they want something cheap and straightforward, so that's where regulation should step in to make them go against their own interests, in the interest of population at large.

Facebook has nothing to gain from transparency. Inevitably, they are going to be inconsistent when it comes to edge cases, which will be used against them.

> 3 and 4 are easy

But 2 (scale) makes this solution to 3 and 4 expensive / impractical.

Facebook has money, and automated filtering of content (rather than manually scanning everything) can make it scalable. Given their monopoly it should be possible to hold them accountable / there should be transparency.

While this certainly makes it much less likely that costly massive human moderation would actually be implemented, I don't think that really qualifies as an argument for why it shouldn't be done.

If you think rossdavidh is right about those issues Facebook is having (and I think he's right), then that's the reality in which the company has to operate. If a business model leads to infeasible costs due to fundamental issues, maybe that business model shouldn't exist as such.

Obviously Facebook (or any other company) doesn't want to do anything that significantly increases their costs without providing revenue if they can avoid it, but that doesn't mean it shouldn't be expected of them.

Or, perhaps, these problems are reasons why large, centralized social networks shouldn’t be allowed to exist.

They exist because they're popular, you can't dictate what is popular among the masses without authoritarianism.

They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages.

There's definitely an anti-trust dimension here, but it's not related to first mover advantage; Facebook wasn't nearly the first mover, or due to capital; Facebook was somewhat widely thought to have over-paid for Instgram, it was Zuck's foresight (or perhaps risk aversion) that gave him the conviction to proceed with the transaction.

Can you give some examples?

Haha, are you serious? Instagram, WhatsApp, Snapchat... Instagram's Systrom explicitly said that he sold because otherwise Facebook would snuff him away.

Facebook never acquired Snapchat so I'm not sure what you're talking about there. Instagram and WhatsApp have continued to thrive since their acquisition, so from a user perspective nothing really changed, it's not like you need a FB account to use insta or whatsapp.

GP: > They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages. Facebook literally copied Snapchat features into Instagram rightaway - Systrom even openly admitted it. That fits the bill of "strangled". Snapchat is in stagnation now, because everything it has is there on Insta.

Instagram and WhatsApp thrived only because Facebook acquired them, else both companies new they would be smothered to death. That pretty much fits the bill of "acquired... Using investment capital and money from first mover advantages".

They don't need you to use your FB account because they cross share data anyways. It's not like it's a painstaking operation to link different accounts, once you have all sorts of device data (which Facebook had, even on Apple devices up until recently).

That may be the case now, but Facebook did start requiring Facebook accounts for their Oculus VR headsets.

Much to the dismay of many. Maybe it's the test case FB is monitoring to look how to move ahead with their other acquisitions. Last I heard Quest 2 sales are good..

whatsapp. It's been drastically changed by facebook

You don’t have to dictate popularity: just create legal liability frameworks that make a solution like Mastodon more appealing than a solution like Facebook or Twitter

People can still be banned on Mastadon, it changes absolutely nothing. In a world where Mastadon becomes more popular than FB people will complain about being banned from the most popular node instead of the most popular http website.

Mastadon is federated, so you can be banned from someone else's instance but you can always start your own instance, or join one where you're not banned.

Right, just like how you can start your own website if you get banned from someone else's, it's servers all the way down. The people complaining about being banned don't care that they can create their own instance, they care about losing access to the popular instance.

The advantage of mastodon is that banning is not monolithic: so, if I have instances A, B, C, D: B can decide (through whatever process it uses for such things) they don’t like A’s content and ban it without affecting users on C and D. But, the problem it really solves is moderation: if I start an instance with 10 friends, moderation of posts on the instance is almost a non-issue that can be handled by a couple admins in their spare time. Handling off-instance posts is more difficult, but you can either maintain a ban list to block a list of bad instances or you can adopt a default deny policy and only federate with instances that have content your local community wants to see. Either way, the Mastodon model seems better because it allows policy to be specified at the “local” level rather than requiring a uniform policy across the network.

> if I have instances A, B, C, D: B can decide (through whatever process it uses for such things) they don’t like A’s content and ban it without affecting users on C and D.

Yes, but this isn't really analogous to the complaint being presented. In the Mastadon world this article would have been titled "A Disabled My Account After I Criticized Them". The author is still free to use B, C, D etc, just like with http websites.

> Either way, the Mastodon model seems better because it allows policy to be specified at the “local” level rather than requiring a uniform policy across the network.

To be clear, I do think an ActivityPub-like architecture is preferable for social networking, but federation doesn't obviate the concept of popularity, it only seems that way now because the fediverse isn't popular, if it ever did become popular then we'd see the exact same complaints when people get banned from popular instances.

The only reason people care about Facebook disabling their account is because Facebook has all the users and their data in its silo. In Mastodon, A disabling federation with B isn’t really the same sort of problem because it’s not a global action across the network.

What A defederating from B is like a website blocking your IP. Facebook banning an account is more like your ISP turning off your internet access.

In theory, absolutely. In practice, I've heard of major instances using their power to dictate "If you federate with $out-group, we won't federate with you". Most admins want to federate with other major (mostly good, popular) federations, so they capitulate.

As such, being kicked from one instance leads you to sign up elsewhere, until a major instance notices and tells your new server admin to kick you out. When you create your own instance, the major instances use their soft power to ensure almost nobody else federates with you.

Sure you can, just require interoperability with similar services and prevent them from straight up buying the competition and watch their "popularity" fade away.

I think preventing FB from buying competitors is perfectly reasonable, but that doesn't really mean much with respect to FB's popularity. It's not as if users have to choose "should I use FB or instagram" they can and do use both, and that would still be possible whether FB owned instagram or not.

How about the other part of the comment that you've ignored? If people could use Alternative X without losing their Facebook/Instagram connections, Facebook would fade into irrelevance. There's absolutely no technical reason for everyone to have a separate Facebook/Instagram/Twitter account with dozens of other IM services instead of picking one and communicating with others regardless of what they've picked.

That future is only one regulation away. The most common reason for staying on Facebook ("all my friends are there") disappears.

> If people could use Alternative X without losing their Facebook/Instagram connections

They can already do that. Using an alternative does not cause someone to lose access to FB.

> There's absolutely no technical reason for everyone to have a separate Facebook/Instagram/Twitter account with dozens of other IM services instead of picking one and communicating with others regardless of what they've picked

This is already a thing.

https://www.pidgin.im/plugins/?publisher=all&query=&type=

I feel like you're just trolling at this point.

> Using an alternative does not cause someone to lose access to FB.

Deleting Facebook causes you to lose access to your Facebook connections. Why can't you just use your Twitter account to communicate with someone on Facebook? There's zero technical reason for this not to be possible.

> This is already a thing. https://www.pidgin.im/plugins/?publisher=all&query=&type=

It's not. It's a third party workaround that Facebook could kill at any point if it gave a shit. Do I really need to explain to you how that's different than first party support for something so basic? Just like with email, you should be able to enter a username and a IM provider and just talk to them as if you're using the same platform. It's really not rocket science.

Fediverse (Mastodon and compatible providers) already does that without any hiccups. It's possible, we know it works for decades (since we all use email), and it should be a requirement for every social network / IM service in existence.

They exist because they’re profitable. They can become unprofitable through democratic legislation.

Twitter is only barley profitable as of a year ago, yet it has continued to exist as a large social network during that time, so that explanation doesn't fit.

We split up Standard Oil, and many other trusts, without resorting to authoritarianism. I'm not saying it's trivial, but it's probably not harder than breaking up AT&T (also a network to connect people) was.

Just don't use them. And try to convince people you know not to. A ban isn't even necessary if enough people decide a platform works against their interests and cease to pay attention to it.

This argument is basically “I don’t need privacy because I have nothing to hide”

I think it's more, "don't consider any part of it private." I always ask people for their email password when they say "I've got nothing to hide" I think it's rubbish.. At the same time, I treat facebook (all of it) as if it were public. If I wouldn't wear a t-shirt with my DM's on it, then I don't write them.

Not at all, I would very much like to protect personal privacy rights. I'm saying a small number of businesses have convinced the general public to invest a massive amount of time, money, and attention in virtual spaces. These organizations intrude in our lives because we use their platforms. We need to quit using their platforms.

Sorry, I was trying to characterize the sort of argument given: "Just don't use it" only works if the people you want to interact with are available some other way in a nearly equivalent fashion: if one's friend group is mostly on Facebook, you sort of have to pick between using FB and missing out on a certain set of social interactions.

Oh, then I agree, but I would suggest it's a worthwhile inconvenience. The more people quit these services, the less incentive other users will have to remain. It's surprising sometimes how even one person's decision can generate mimicry.

> 2) they are wayyyyy too big to police content manually

Oh hell no they aren't too big, they just don't want to reduce their profits from the current 18.5 billion US $ (per https://www.statista.com/statistics/277229/facebooks-annual-...).

If Facebook wanted, they could massively ramp up staffing, hire them themselves, and pay them a livable wage instead of outsourcing the cost that their business occurs on society (who has to pick up the slack from conspiracy myths, fascism and other shit), on the users who are mistakenly banned and lose their primary contact e.g. to overseas family, and on the moderation staff that is hired via third parties and exploited.

I would shed no tears if FB profits went down, but...I think you're underestimating how big they are. Two BILLION users, in ~100 countries (each with its own standards on what is acceptable, legal, repulsive, etc.). If you had one moderator per 100 users, that would 20 million moderators. I don't think they could hire 20 million high quality moderators (there's only one dang, and we're not giving him up), and certainly not with 18.5 billion.

They need to be forced, by government regulations, to have to talk to people who have had their accounts banned and to provide them some kind of legal recourse. The law needs to recognize that for massive social networks like Facebook and Twitter and for the Apple and Google App stores that people are actually entitled to those services, and have a right to use them, and shouldn't be kicked off of them without any form of recourse. They've grown too big to just consider them like the corner coffee shop and allow them to kick anyone off of their platform for whatever reasons they cook up.

And that means that they will need to staff up and that will come out of their profit margins, and that's okay. That's why they need to be forced to do it by regulations.

Which is why social media networks should make the far easier rule that they will only ban illegal content. If people want something censored, make it illegal. If people don't like they things are being censored, take it up with their local representative.

They certainly generate enough revenue to moderate content manually.

But they may not be able to do it well enough that it will be notably better than an AI. Scaling manual labor can't work indefinitely due to the scarcity of talent.

Keep in mind that while a human being is very good at categorizing things, they're terrible at doing it over and over again, so the scarce talents are likely consistency and conscientiousness.

You should expect that, overall, the (n+1)th person you hire is marginally less talented than the nth. And while people are constantly entering and exiting the labor pool, this is negated by the fact that you're trying to expand your share of it.

So while there's a lot of randomness in recruiting, if you enforce any kind of standard, you'll hit a point where not enough candidates meet that standard, so you have to drop it. Once it is below the threshold of "better than AI" you have no reason to hire more people.

And you'll try to expand your eligible pool by raising wages, but everyone else wants consistent, conscientious people, too, so you're liable to run into the economic constraint: the amount you have to pay someone will exceed the value they're adding as a moderator.

They can solve this issue by adequately providing for self-moderation. Avoiding this approach reveals an intention to impose. Users leave once they realise the violation.

It's inherent to centralized power that opposing interests will attack it. Principle agent problem strikes again

Reply on news.ycombinator.com