Meta in Myanmar, Part III. The Inside View
erinkissane.com
erinkissane.com
Its really hard to contextualize these numbers. What are the comparable rates for other media?
What % of hate speech on Mastodon is taken down? Twitter? YouTube? Discord? Comparing with old-school mob formation technology, what about leaflets handed out? Political rallies?
Meta is an obvious target because they have such a high % of total online speech, and by virtue of this make a potentially impactful single intervention point to make things better. And these articles do make it seem they have room at the margin to improve. But imagine a word with multiple social networks per country - do we think they would be better or worse at policing this stuff (and crucially, the places where it matters are the developing less-regulated countries, often with governments that are at least disinterested in preventing the ethnic conflicts, if not actively promoting them)? I can see hand-wavey arguments in both directions but what I really want is data analyzing the question.
> And this method works. In Myanmar, “reshare depth demotion” reduced “viral inflammatory prevalence” by 25% and cut “photo misinformation” almost in half.
> In a reasonable world, I think Meta would have decided to broaden use of this method and work on refining it to make it even more effective. What they did, though, was decide to roll it back within Myanmar as soon as the upcoming elections were over.
It's still the same people making the profits though. Regardless of what political/economic system you have, sociopaths will always gravitate towards whatever position allows them to acquire the most wealth and power. The only difference is whether they have the legal right to send men with guns to lock you up or kill you if you disobey them.
In any society composed of humans, the shitburgers will find a way to shit on the largest groups of people they can using whatever instrument of power is available. In the US it’s dollars, in the USSR, its influence and power.
First there would be the delay of the rest of the community understanding what was going on in an unusual language. Second there is the difficulty of deciding what to do and implementing an effective filtering policy. Third since there are multiple instances this can't be done in a consistent way across the network. Fourth they can't stop the Myanmar government from setting up its own Mastodon instances, blocking access to outside instances, etc.
You'd better believe that if Modi keeps going down the road he is going on that India will kick out all western social media companies if they try to block his campaign of dehumanizing non-hindus.
From my perspective there is a lot of harmful talk that goes on social media that falls short of hate speech. In terms of my own feed I have keyword blocks against most Republican primary candidates (not Nikki Haley though) and there are many other topics that I think people are not capable of having a civil and productive conversation on or that I just don't want to be exposed to. A lot of stuff on the slippery slope to "hate speech" is blocked by this and it is OK when I do it because it is my own feed, but you'd better believe there are lots of people who think they have something so important to say that it would be crime to "censor" their shit (for lack of a better word) if the block affected more than just me.
Unfortunately the discussion around A.I. is dominated by positions like: (1) ChatGPT is the God at the end of the universe that Teilhard de Chardin told us about, or (2) It was just fine that Google stole everybody's content for years and years but all of a sudden ChatGPT came out and be outraged all of a sudden at this rip-off.
The truth about it is that machine learning algorithms over text have terrible performance, even the good ones. The performance curves look something like
https://scikit-learn.org/0.23/auto_examples/model_selection/...
and for my RSS reader I am happy to approach an AUC of 0.8, an AUC of > 0.95 for most tasks is getting into science fiction territory. And that's nowhere near being able to clean up hate speech at the rate that it should be cleared.
The short of it is that if you want to get rid of a lot more than "5%" of the outright "hate speech" you will also get rid of a lot of "dislike speech" and "angry speech" and "annoyed speech" as well as some "innocent speech".
To be really frank my filter deletes anything about "cis" and "trans" because it I think maybe 10% of what people say on the subject is outright hateful, 80% of it is emotionally negative, maybe 10% of the time somebody has something neutral or positive to say. My take is that losing a small amount of content I might want to see for being free of a lot of negativity is a bargain I am willing to make. A platform that imposed that kind of judgement on the community could be a much more emotionally positive place than anything you've seen but it is also going to get all sorts of criticism from people for doing so.
> First there would be the delay of the rest of the community understanding what was going on in an unusual language. Second there is the difficulty of deciding what to do and implementing an effective filtering policy. Third since there are multiple instances this can't be done in a consistent way across the network.
This (except the last bit) is missing the point about what Mastodon is for.
Why would "the rest of the community" would even be entitled to decide what to do about it ? Not that there would be a lot of overlap with the Burmese community anyway, so that wouldn't have much of an effect.
We already had an example BTW, with Gab splitting out with Mastodon, but then in a heavily English-speaking context.
What would have made at least a symbolic difference, is that some of these Mastodons would have been operated by the anti-hatred groups described in part I, and they would not have needed to unsuccessfully beg Facebook about doing moderation (with 0-4 moderators from other continents).
But in the end this is mostly spurious discussion : hate speech only effectively gets stomped out when a judge hands out sentences to the haters, but when the law enforcement that should handle them are the ones preparing genocide...
(Not that Facebook is not condamnable for heating it up.)
Certainly some kinds of right-wing hate are suppressed but there is very little pushback about widespread misuse of the word "fascist" to describe your local police department, or people in women's sports who use period trackers to prevent ACL tears, mainstream milquetoast politicians, or just about anyone.
On the other hand a lot of it seems like BS, there are two Mastodon instances that seem to be at war with Hacker News, I think I understand why one of them is mad at HN, I don't know if the second is mad out of sympathy with the first or they have some other grievance.
Many mastodonsters were concerned that Threads would be poorly moderated (might allow posts about how period tracking prevents ACL tears in female athletes I guess) but Threads pretty much dropped everybody's radar.
---
One angle is that a system like this should have "defense in depth", like you try to create a culture where hateful speech is not welcome, where people don't try it because they know they'll get disapproval, where people know they'll get confronted if they call Keir Starmer a fascist, where people know you just don't boost hateful things, etc.
In that sense you might just want to cultivate a lot of positive and neutral stuff and encourage people seeking negativity to go elsewhere. I think this is what Instagram was going for with Threads and it might be one reason why people should be asking "whatever happened to Threads?" but they don't because they've forgotten all about it.
Mastodon offerts a possibility of having truly disconnected communities who do their own thing though sharing a protocol. What's acceptable in one community is not in another and that's fine because users can pick and choose what they participate in and what they see. However, this model is not perfect in Mastodon either as people who believe crusading is a virtue actively seek to undermine communities they don't participate in.
With Facebook however, the profit structure of the company and the way the website works pushes it to categorize users to receive content not based on user preference but based on algorithmic optimization. I may intentionally not join any motorist group, I still get shown content that gets a lot of engagement from idiotic suburbanites.
You could get an approximation - For resourced languages though.
Meta launched Hindi classifiers in 2021 and Bengali in 2022, covering 0.6bn and 0.24bn people, respectively. I know how poor the lexicons I have seen in the wild are, and how hard it is to stay ahead of terminology.
Meta is probably doing more to stay ahead than others. However the ARPU for a user in the US vs India is stark.
Platforms aren’t going to release the necessary data for even a rough approximation. Reddit was a potential data source to learn from, but now we have the API rules, so that is also closed off. Lucky us.
Not to mention how supremely unprepared society is to actually understand or discuss the data without making it worse.
Meta has more resources and frankly they regained my trust years ago. However, business is business. Twitter and Reddit have shown that in this current cycle, there are are no consequences for neglecting Trust and Safety, as shown by other companies. T&S teams have been reduced globally. T&S teams are being targeted and vilified.
Next year, there are ~50 elections happening, including the USA and India. The network suits certain narratives - the more insular, fear driven ones. Trust and Safety teams are themselves being exposed.
Its already looking like a perfect storm.
The question, of course, is whether there is a political will to "police hate speech". In reality this already dubiously noble goal is as flexible as the definition of "terrorist organisation".
> We proactively detect 99 percent of the hate speech removed from Facebook in Myanmar
If the accusation is correct, they are not saying that they capture 99% of all Myanmar hate speech on Facebook. I remember that the big news agencies reported it that way. Meta is actually saying "of the 5% that we do catch, 99% was caught automatically." The key phrasing is "99% of the hate speech removed".
In other words, this number actually means, "We do almost no non-automated detection of hate speech."
Smart spin, huh...
Do they have good estimates or is this an unknown amount?
> …we’re deleting less than 5% of all of the hate speech posted to Facebook. This is actually an optimistic estimate—previous (and more rigorous) iterations of this estimation exercise have put it closer to 3%, and on V&I [violence and incitement] we’re deleting somewhere around 0.6%…we miss 95% of violating hate speech.
https://facebookpapers.com/wp-content/uploads/2021/11/Hate-s...
Their internal teams say they could be missing 95% of it.
The reality is that Facebook has no idea how much hate speech they have. They remove 99% of whatever they detect.
Facebook also said they removed 99% of terrorist/isis posting. Of which the author claims the total amount is unknown. But we know the % of hate speech because of facebooks ability to guess (as stated by the author of which the author was confident). Now we are lead to believe that terrorist content can't be estimated by facebook.
Either we accept they can estimate and take the 99% of terrorist content removal in total at face value (of the total amount) or we must ignore facebooks hate content estimates
Not an unexpected decision overall, but it did surprise me how low Facebook can stoop. Like, they didn't even bother to fix their stuff in Myanmar after all these years with Myanmar being the poster child of their hate speech problem!
> In 2022, Global Witness came back for one more look at Meta’s operations in Myanmar, this time with eight examples of real hate speech aimed at the Rohingya—actual posts from the period of the genocide, all taken from the UN Human Rights Council findings I’ve been linking to so frequently in this series. They submitted these real-life examples of hate speech to Meta as Burmese-language Facebook advertisements.
> Meta accepted all eight ads.
(Keep in mind after this, Facebook hired 20,000+ content moderators at great expense who they still employ to monitor for stuff like this).
According to the article, Facebook did make the deliberate decision not to dedicate resources to Myanmar and other non-western countries, so I don't think it's negligence. "Greed" is more appropriate.
> Guy Rosen, Facebook’s VP of Integrity, told Sophie Zhang that the only coordinated fake networks Facebook would take down were the ones that affected the US, Western Europe, and “foreign adversaries.”
There's no obligation to operate everywhere. Instead, as discussed in previous parts, they invested heavily to get a monopoly in third world countries.
But they didn't. The situation was as you put it mostly unmonitored. In other words, the spread of genocidal content was organic. This is the equivalent of blaming the phone company or the ISP for what the users do with the communication channels.
I promise you if they tried that people would complain even harder.
No one expect Facebook to remove all misinformation/hate speech/etc. That is unreasonable. But they have literally 1 data scientist to monitor the whole non-western world.
In a year where their annual profit is $23.9bn.
Assuming 1 data scientist is paid $500k/year, surely it is sensible to dedicate $20mn, less than 0.1% of their annual profit, to hire 40 more such data scientist and literally increase their effectiveness on this topic by 4000%? At the very least, this is what I expect from a company that actually have a speck of moral.
> A strategic response manager told me that the world outside the US/Europe was basically like the wild west with me as the part-time dictator in my spare time. He considered that to be a positive development because to his knowledge it wasn’t covered by anyone before he learned of the work I was doing.
The integrity org, back when I was there, had thousands of folks and was aggressively hiring. They were not a lone island and were also supported by other orgs like data infra, FAIR, security, privacy, product teams and more.
Site integrity, Sophie’s team, was a small piece of the work done there. And they, like any team, relied heavily on other teams. You have all this unbelievable tooling internally, and internal teams are incentivized to get you to use it.
These issues were not from want of trying. The reality is that problems like this at scale are incredibly difficult. In my opinion, Frances trivialized a lot of this in her whistleblowing, but it makes a good news story so what can you do.
It pains me when people trivialize it as “just do the machine learnings to fix it all” or “if these tech companies actually gave a damn this wouldn’t be an issue”. It’s an incredibly hard problem, and much like security it will never be “solved”.
That doesn’t excuse when technology has a bad effect on the world. We need to be better, and for the most part as a society we are trying hard as hell. It’s also why the work is interesting and impactful. More should get into it.
I talked to FB people (recruiters, managers) in the site integrity team back in 2018. Admittedly I had some reservations up front, but during the on-site day I made up my mind to never even consider FB as a potential employer again. During a chat session between interviews, I asked about the prospect of doing proactive education - essentially, detecting users who had been caught in the influence operations and then surfacing them a note that they had been targeted by such activities, so that they could themselves make educated decisions.
The senior manager I was talking with at the time was visibly taken aback by the very idea. "We don't do that!"
From that experience I drew the inference that FB are fundamentally, as an organisation, incapable of doing the right thing. I suspect it's less about the cost, and more about the prospect of openly accepting accountability for what their platform is really used for.
Insisting that it's hard and requires advanced ML is complete BS. Self delusion by FB'ers to give themselves and excuse for their complicity.
If an airline doesn’t have enough pilots to service 20 routes they reduce their routes, not let passengers fly the plane.
Look at fsociety's comment in this same thread to see just how much misinformation is out there (which he helpfully corrects)
It just so happens a lot of misinformation is anti-corporate since many people want to believe every bad thing they hear about them. So I end up being one of the few commenters who say, actually, the big company did try here.
This is important because if every big company is supposed to be equally bad, there's no reputational penalty for the actually bad ones.
The point people are making here is that Facebook shouldn't have done the impossible and created a perfect tech solution to moderate content, they should not have operated in these countries at all if they couldn't do it safely. They did not even try, instead they ignored internal reports on these problems for years and actively decided to make the problem worse with their single focus of growth. And for as you put it, only a few $ in advertisements, to fuel the fires of genocide.
Of course it's greed, and of course it's reasonable. Apple is directly responsible for the work conditions of its workers and the contractors they use, absolutely. If they can't be responsible then they can close shop. There is no obligation for them to have a business.
They did certainly care about the growth of daily active users even if it meant they expanded to markets where they couldn't responsibly moderate their platform.
A company as big as Facebook is going to have different groups with different objectives that often do not see eye to eye or even know about each other. A company as profitable as Facebook is going to have projects that are motivated by things other than profits.
There is most likely a team at Facebook whose goal it is to get as many users as possible in developing markets. These markets do not generate profits from online ads. Facebook will never disclose its cost per user but in general online ads only make money from affluent users. I have heard that even mid income countries like Mexico or Romania are not profitable for ads. A place where people make an average of $3 a day is wildly unprofitable for Facebook and even the most optimistic development projections will not make them profitable for several decades. The OP is spot on that the amount of money that Facebook will ever receive from Myanmar is negligible.
Based on conversations with people at Facebook I generally get the sense that people there (especially at the top) have a strong sense of mission and a belief that their product makes the world a better place (I disagree with this but that doesn't change the fact). I would guess that these the project tasked with increasing users in developing countries is viewed more as philanthropy than business.
Part II of this series addresses this stat. During the peak genocidal years it seems there were only a handful of mods assigned to Myanmar, and none of them lived in Myanmar. To this day it sounds like there still may not be more than a hundred people moderating Myanmar content. Which is inline with the 20K number: 195 countries * 100 moderators per country = 19.5K mods total
I guess my main idea here is that when I look at the back-of-envelope numbers it's easy to see how even 100 moderators would have trouble keeping up with the content that millions of people produce. So you really do need those automated moderation tools to be effective.
Can someone please continue the thought experiment by estimating how much content each mod can review in a day, then extrapolating up to a week or month, and comparing to an estimate of how much content was actually posted on Facebook in the similar timeframe? I believe Part I or II provides an estimate like that.
Of course Facebook didn't fix anything in Myanmar - the new junta government is both the one running the troll farms and responsible for the previous genocide !
Calling out the simple statistic slight of hand would have been a sure fire way of establishing control and the impression of hard on big tech they wanted. It was delivered on a platter.
Instead of that we got “Senator we run ads” and stupid gotcha games trying to prove FB was biased against conservatives when it was painfully clear that Joel Kaplan had fully taken over the bias department in favor of his friends (preventing misinformation action on the news surface because, shocker, it came 90% from conservative staples like Drudge report)
https://www.reuters.com/article/us-myanmar-rohingya/myanmars...
> "“Though not confirmed, there are indications he went to Pakistan and possibly elsewhere, and that he received practical training in modern guerrilla warfare,” the group said. It noted that Ata Ullah was one of 20 Rohingya from Saudi Arabia leading the group’s operations in Rakhine State. Separately, a committee of 20 senior Rohingya emigres oversees the group, which has headquarters in Mecca, the ICG said."
https://en.wikipedia.org/wiki/Ataullah_abu_Ammar_Jununi
That certainly doesn't justify the response of the Myanmar government, but it helps explain it to some degree - imagine the US political response if, say, US border facilities were attacked by an Iran-based group that killed a dozen US border guards.
Entirely neglecting this background is just bad journalism, and also distorts the role of Meta. It's highly unlikely, given this information, that a rigorous censorship strategy on Meta would have changed the outcome much if at all. A more rational conclusion is that the Myanmar government viewed the Rohingya people as a potential base for Saudi/Pakistani-backed terror groups to launch attacks within Myanmar, and that was the root cause of the expulsion, and that social media-based propaganda efforts were not that important in understanding what happened.
I'm also in favor of free speech over censorship - it's very easy to demonstrate in an open conversation that mindless hatred of others based on their ethnic/religious/racial background is simply the product of ignorance and immaturity.
At issue here is the PROMOTION of misinformation, lies and genocidal speech (in users' feeds), not the fact that they haven't deleted it.
Demonstrating that fact is a really poor substitute for taking some kind of action against an ongoing genocide.
Kind of like the best way to have avoided the war in Ukraine would have been for the USA to NOT have engaged in a regime change op in 2014, and the best way to avoid the current Libyan disaster would have been for NATO to have NOT engaged in a regime change operation in 2011, and so on.
Clearly it's not; this myth needs to stop being pushed. Everything we have observed from the internet age is that such hatred metastasizes even easier because of the internet and social media, and that almost all discourse is worse.
It's true there are problems with Facebook. But without Facebook the Tatmadaw would still have laws against Rohingya, still have a policy of driving them out, still would engage in fighting ARSA, still would fund and promote groups like Ma Ba Tha, etc.
The Myanmar (Bamar), Karen, Shan, etc populations were all there before the British drew the borders of 'Burma.' The conflict goes back way farther than WWII. The main difference now is that the Myanmar are winning.
Settlers and natives has nothing to do with it. All the major armed ethnic peoples existed in Myanmar since before WWII. The “settlers” were the British and they left.
I am not convinced this ever happened in any meaningful sense. As you note, it turned into armed conflict almost immediately. This was a British-induced transient state that was unable to sustain itself in the region.
> The “settlers” were the British and they left.
The Myanmar state/people are also colonizers/conquerors. You don't need to come across an ocean to see territory that you dont control and decide to try to take control by force. Facebook and ethno-nationalism are simply new tools in a long-running conflict.
The whole reason Facebook is a useful tool for hate groups is because it allows them to cultivate a fake justification for hate that goes beyond "mindless". Inventing crimes, doctoring photos, promoting false narratives and conspiracy theories, etc. It radicalizes moderate people by making it seem like the out group is an existential threat to the in group. We don't hate them because of their skin color or religion, we hate them because they're trying to kill us first.
Open dialogue can't fix this because it's much harder to disprove bullshit than to generate it.
With that being said, are we really supposed to be holding entities like Meta/Facebook accountable for the actions of governments? You can read through my comments here on HN and I'm sure you'll notice I'm not one to generally have nice or even neutral things to say about megacorp entities like Meta, but I just fail to see how any of this is really on them.
If it wasn't Facebook where misinfo was being spread, it would've been twitter, or mastodon, or whatever other social media platform. And even if there were no social media platforms, disinformation, propaganda and unfortunately even genocides have all been happening for a lot longer than Meta's been a thing, and I seriously doubt that the lack of channels like Facebook would've prevented or even changed anything.
Why are we suddenly deciding that Meta/Twitter/Google/Pick your poison are to be the arbiters and keepers of truth and justice?
They are being held responsible for their claims. For their product decisions. For their choices to ignore their own researchers.
No one is “suddenly” anything. People have been crying out about this for a decade+.
Please read the article. The answers are directly there.
It was them who first adopted most of the dynamics that lead to these things
The articles present a lot of evidence to support the notion that Facebook was well aware of clear-cut incitement to violence years before its escalation into large-scale genocide in late 2016. It isn't spelled out explicitly, but I think there are two things that make Facebook's initial apparent non-response easier to understand as incompetence rather than evil:
* In the aftermath of the Arab spring, technology and social networks enjoyed a honeymoon in public perception that is hard to remember over a decade later. People, including (especially?) people in tech, could genuinely believe that making it easier for people to talk to each other was basically good, full stop. I'd suggest that this honeymoon only really ended after the 2016 US presidential election, which caused a lot of soul searching about information and echo chambers, and got even worse after the Cambridge Analytica scandal [3, this is a graph image from NBC news]
* As part I of the article notes, "[t]he information landscape in Myanmar is so unstable that accounts of any given incident conflict, often in major ways". This dynamic seems to appear in a lot of places where well-off westerners interact with much poorer and more chaotic countries (Rwanda is a clear parallel). Part I points out that the "Burmese government" itself seems to make statements both fanning anger against the Rohingya and then trying to walk them back when it boils over into disorder
So I think it was possible to be a basically well-intentioned (naive) Facebook employee in the leadup to 2016 and hear of strange, almost genocidal sounding posts in another language in a place you don't understand and still not think "this is a break-glass moment and I need to escalate and make a new path inside my corporation for this to be taken seriously". In hindsight, this was wrong, but I think we're forgetting that what happened next didn't seem possible, because we still thought of the internet as an exceptional new dawn in mass communication. A lot is necessary for people to process something as a novel, serious emergency.
However, it's hard to maintain this judgment over the ensuing years, as senior figures at Facebook are repeatedly presented with evidence that people are deliberately distributing misinformation (e.g., making fake accounts, and other clear-cut evidence that doesn't require a fact checker and research) and mostly wave it away unless it's big enough to provoke media and investor consternation. By the end of the 2010s, I think Facebook was well past the "this sort of thing wasn't something we could imagine" stage of a crisis, and into a less defensible position along the lines of "this is reality, it cannot really be moderated, but we can pretend to moderate".
One last point re: moderation, I have no idea how Meta hires its Burmese content moderators, but it must be tricky, since as far as I can tell most people in Burma are anti-Rohingya. I would like to be wrong about this, but what should be factually trustworthy sources like the Brookings institution write things like:
> Most of the Myanmar population, especially the Buddhist majority, feels that the Rohingya don’t belong in their country ... [t]he straightforward solution would be to help the Rohingya return to their homes in Myanmar and live in peace and freedom. Sadly, this solution looks impossible in the near term because of the nationalist sentiment of the Buddhist majority
[1] https://erinkissane.com/meta-in-myanmar-part-i-the-setup
[2] https://erinkissane.com/meta-in-myanmar-part-ii-the-crisis
[3] https://media-cldnry.s-nbcnews.com/image/upload/t_fit-560w,f...
[4] https://www.brookings.edu/articles/on-the-ground-in-myanmar-...
(That quote comes from the description of the mechanics of the military's coordinated propaganda campaign.)
I would like to share a PSA in the truest sense of that acronym. I suggest building an automatic bullshit indicator in your brain whenever you see any content along the lines of "they're harming our women" from anyone. It may literally be the oldest trick in the propaganda book. Both sides used it extensively in WWI [1]. It pops up a lot in the history of American slavery / racism as the idea that often directly sparked lynchings. I'm sure you can think up lots of other examples. Sorry if this is already obvious but over the years I keep seeing more and more examples of it and IMO this tactic doesn't seem to be as well-known as the technique of depicting the victims as an infestation of animal pests.
[1] Example of USA propaganda from WWI https://firstamendment.mtsu.edu/article/committee-on-public-...
The reference to Nuremberg was more for an example of an international trial than for the nazis, although faciliting a smaller scale genocide does somehow seem to have a relationship with the nazis...
Given the havoc they willingly caused to nations worldwide, an international trial for Zuckerberg and the other responsibles does not seem so inconceivable to me...
And that's not considering a lot of other probably less important things they did
Is your argument that Facebook as a whole is specifically engineered to incite tribal violence, and so should be banned on the same basis as radar scanners? Because in that case I think you have an argument to make which isn't completely incoherent.