Facebook approved paid ads inciting violence against the Rohingya
apnews.com
apnews.com
Maybe the answer is that these things can’t ever be automated and if you can’t staff the process then you (and therefore nobody) should be working at this scale.
However, I would add, that also non-automated processes have imperfection. There can be internal incentives to allow such ads. The FB culture is rich of processes which are profit-oriented rather than ethics-oriented. Who is to say, that such ads would not have been approved by some spineless group working at FB? I would not deem it impossible. Maybe after the 1000th ad you approve, you might become sort of blunt. Add to that a small signal from above, that this add would bring a lot of profit and you might wave it through as "Just another ad, how bad can it truly be?". I am unconvinced, that many people working at FB have high ethical standards and even less convinced, that FB puts people with high ethical standards in the position to approve ads.
So in total, I am not convinced, that FB is actually capable of completely preventing such ads, even if they do not automate. This however, is not to say, that they should not be held accountable. They should. In full force. It is rather to say, that their business model is not appropriate and they cannot guarantee it to be ethically OK. (Ethically OK apart from the already ethically very questionable act of targetting ads and spying on everyone they can in the first place and doing that for their own gain / money.)
We would like corporations profiting off the algorithms to be held accountable, but this is hard today and lawmakers have moved too slowly on a solution.
One alternative is to outlaw algorithms operating at a scale larger than our current accountability framework can handle. The reason this works is because it forces a human back into the chain of accountability. The corp will pass the potato and fire the employee who approved the ad. The employee's incentives are slightly more nuanced than you wrote - they want to maximize dollars while avoiding being fired. Since the employee is a regular human and not a corp with superlawyers and deep pockets, there's a better chance of criminal or civil followup targeting the former employee.
With this accountable-scale bottleneck in place, we now have incentive for lawmakers and corporations to legislate the original problem and remove the bottleneck. That's why I think GP's proposal is a step forward. Even though the net harm may be the same at automation-scale and human-scale, the accountability envelope is closer to the desired end state.
Why do I care how Facebook implements their moderation? Their liability should be independent of their processes.
From TFA:
>The group pulled the ads before they were posted or paid for
I would not expect actual approval to come until the ads were paid for or posted. Otherwise, bad actors can test the automated filters for free until they find a way around them.
Maybe the bad ads still get through, but IMHO the article is misleading.
Maybe, just reaaaally maybe, this is not something that should be allowed to be automated.
Could it be?
There are other things in society we do not allow to be automated yet. And this might just be one of them.
This is so disgusting. Made FB already a statement?
Also, I dont think FB would let ads go unchecked in the USA: waaaaay too much hassle in case an "illlegal one" slips through. It's just that FB does not see the need to do that in Myanmar, as that govt can do them little harm.
I find it inexcusable to be honest.
Wouldn't the direct alternative be a localized advertising business based in Myanmar? That would seem to be much worse, since the leadership would then be people who likely condone genocide (and are out of reach of Western laws/norms).
(We're a bit too implicit about the fact that Silicon Valley corporations are American corporations, global projections of American power. If we neutralize them, we lose some things we haven't examined carefully).
It *does* look double-edged to me: if you pull Western social media out of places like Myanmar, their replacements could be more destructive things. Their reliance on Western social media, for now, gives the West additional influence and levers in that country. I do agree Meta is using that influence badly, that their algorithms seem to be amplifying calls to violence rather than countering them.
And when you consider that by pushing these negative externalities onto society, they save tons of money which goes into the pockets of their executives and investors, it begins to look like these companies have just invented a new form of theft.
By "moving fast and breaking things" they hope to stay one step ahead of the regulators for long enough to get rich before society can react and close these loopholes.
I cannot offer any solutions for tackling this issue internationally, especially in cases such as genocide. The issue seems hopeless and not particularly unique to the internet (see: previous genocides and their use of new mass media such as newspapers in Germany, radio in Darfur).
However, in the United States, I think the solution is ultimately local. Making specific and credible threats against someone is crime in all jurisdictions. Harassment using a telecommunications device is also a crime in most jurisdictions. Combining harassment with either vague threats or hate speech is sometimes its own crime.
I've found, to my surprise, that the legal system is interested in internet harassment/threats even if the harassment/threat is "obviously trolling" by internet standards. The bad news is that there isn't a bright line and the bar is high. The good news is that the type of internet behavior that's been normalized in the past couple decades is so far beyond the pale that a bright line isn't even needed in many cases. E.g.,
- sending dozens of unrequited and extremely angry/hateful DMs to multiple unlinked accounts over weeks/months, even after requests to stop contacting the person, is clearly a criminal act called harassment, punishable by fines and jail time.
- Doxxing a person who one has previously made threats against is clearly a criminal act called stalking, punishable by fines and jail time.
- Telling someone that you think you know who they are, that you will figure out their IRL identity, and that you will then harm them in a specific way is clearly a credible threat. Again, a criminal act punishable by fines and jail time.
These are all very common types of trolling. I would have thought that the lines were fuzzier from my long tenure on the internet, and that courts/police would respond with their equivalent of "ignore it and don't feed the trolls". But it turns out that these anti-social behaviors are not normal outside of very specific internet subcultures, and judges/detectives/DAs have spent approximately zero hours in internet cesspools. To them, all of the above are alarming, obviously criminal, and not examples of cases where one should simply ignore the behavior and go on with life.
I've found that even respected moderators of respected forums will disregard as "just trolling" behavior that my DA's office and local detectives describe as "obviously criminal". Facebook and Twitter will also refuse to ban accounts that are engaged in harassment/threats/stalking behavior that the local DA is interested in prosecuting.
I'm also working a few attorney friends -- and spending some change -- on developing a repeatable, low-cost, DIY-as-possible civil strategy for unmasking and obtaining restraining orders against John Doe harassers.
The days of "threatening and harassing people on the internet" being "normal troll behavior" are going to come to a rude close for US residents over the next ten years, mostly on the back of serious misdemeanors and innovation in the use of civil lawsuits.
Consumers can't do much.
Citizens can leverage their local government to prioritize investigation and enforcement of laws against harassment, stalking, and threats.
Persons with sufficient resources can try to use civil suits, but the path here remains opaque, expensive, and is specific to your state of residence.
You are a consumer, but you are also a citizen of a country and a state, both of which have strong rule of law and more-or-less free&fair elections.
> What’s the primary work that you’re doing in this area?
I am:
1. leaning on friends with legal training and supporting their time with my money, and
2. keeping pressure on my DA and Sheriff's office to prioritize enforcement of online harassment, stalking, and threats. Particularly in cases where there is compelling public interest (e.g., local teachers).
> What needs to be done to make this a reality?
Small nudges in culture within every law enforcement organization and every prosecutor's office. Focus on your home turf, and don't over-think it. Participate in relevant elections. Show up to candidate forums and ask relevant questions, then donate and volunteer to candidates who give good answers and let them know why you're donating/volunteering. If there is no candidate forum, or if the forum is held too close to election day to make a difference, send individual emails.
"Prosecute internet trolls who are clearly harassing, stalking, and making violent threats against normal private citizens" is not a hard sell, so it's more a matter of being squeaky than convincing anyone.
Again, acknowledging this fact does not need to be understood as a defense of Meta.
we can’t possibly review all ads to ensure we don’t encourage violence, so between the choices: a) review all ads, charge per request or just operate slower until our automation is 100% accurate OR b) accept that 0.005% of the time, ads promoting violence and worse will get posted, and cause harm.
It doesn’t have to be all-or-nothing. I’m sure Facebook has plenty of intelligent people who are more than capable of creating a risk-based review process that balances the effort vs the outcome better than whatever they’re currently doing.
it has to be a culture issue inside fb. So much of this is just wrong, to go on without much scrutiny and get to this point is just mind boggling.
The problem is, mistakes in this domain have greater world wide consequences than one company losing some money.
They need to step away from this if it's not managed better. This is now an ethics problem, and not a financial one, even though some Facebook execs may not see it this way.
Or maybe more precisely: "we don't check them all because our business model wouldn't support it." The arrogant (and false) implication there is that making their business model work is the highest priority, and all of society's other priorities should take a back seat to it.
If your business model only works with shitty automation that doesn't actually do its job, then you need to find a different business model.
That's exactly the argument. Social media's gross margins are artificially inflated because they don't take into consideration the external costs associated with them. Traditional media businesses never got to scale to FB level and enterprise value for exactly this reason.
Your comment reads like this: https://www.newyorker.com/cartoon/a16995
To be clear: I know the answer, in economic terms - economies of scale, network effects, etc. But also I don't care about that, because none of that matters to me. And the downsides (and externalities) of having a single giant service functioning at this scale are massive.
The costs are large, and the benefit to me is small. That's why "we just need to accept it" is an absurd argument to me - I don't personally benefit from Facebook being as large as it is, and I would experience no great personal or professional loss if Facebook were far, far smaller.
I really don't see how a tiny fraction of bad ads slipping through and immediately being caught and making the news for being so rare means we should throw the whole thing out.
Those things need to be bundled because something has to pay for it and poor people shouldn't have to decide between groceries and being connected to others online.
That said of course, what’s good for the gander might not be what’s good for the goose.
So something may be legal in Timbuktu but not in Shangri-La. Moreover one can’t aim ti please everyone. How do you meet local regulation in Russia or China but also abide by second or third party laws? Or contend with public opinion (cancel culture, boycotts, moral outrage, etc).
An individual user does not bring any revenue. If they are horrified by an ad they cannot just vote with their wallet and switch to a competitor (network effect together with free offer guarantee no honest business can really compete). Meanwhile, the only metric that matters to actual paying customers (advertisers) is eyeball count—they are not going to leave in droves if this platform is where the users are.
Thus, letting the number of actual human staff grow to address some concern that involves end user satisfaction (including lofty goals like ethics, which is higher up the pyramid of end user satisfaction) is financially irrational to them, and automation rules.
Sadly, this creates a situation where many people call for FB to be regulated. In my view, it will only make the situation worse by entrenching FB’s monopoly—rather, what we need is a general regulation that makes these types of businesses infeasible, so that we have a choice of smaller providers who charge their users (and are incentivized to make a product that benefits them) and interoperate with each other (and have to worry about users leaving). Instead of giving up on free market, we should make it work.
The whole thing never happened! Complete and total hoax. A pump and dump scheme that Snopes got wise to more than a week ago. And it showed up in my feed 4 times in scrolling five minutes. I reported every single one of them as a scam.
I swear, no one is at the helm over there.
Or, more accurately, Facebook has a lot of rules that they impose on the third parties and few first party ad reviewers, especially in the area of medical claims.
The entire service was a scam and all the comments interacting with it were thus guaranteed to be fake.
I've reported both the page itself and a handful of comments - this should've been a jackpot for FB to be able to catch and ban all these fake accounts conveniently rounded up for them.
The response? That none of these violated their rules and were allowed to remain.
I say, in all brutal honesty, I give zero fucks about a company going from a $900 billion market cap to just a mere $600 billion. Cry us a river already.
I wonder what the end goal is for whoever is running those ads? To get people worked up at Mayim Bialik hosting Jeopardy!? To sell CBD gummies? Or just to piss off people on both sides and stir up American pop-culture controversy?
To generate traffic which creates ad impressions.
As far as I can tell, that program started in 2016, not "shortly after... 2000"
“Facebook confirmed to TechCrunch that Free Basics did indeed end in Myanmar in September 2017, a little over a year since its June 2016 launch in the country.”
https://techcrunch.com/2018/05/01/facebook-free-basics-endin...
Last year it was reported that Facebook failed to remove anti Muslim content in India, another country rife with ethnic tensions. India also happens to have Facebook's largest user base.
Ah, here: https://multilingual.com/facebook-linguistic-gaps-allow-hate...
greed breeds negligence
> The group pulled the ads before they were posted or paid for
This is just a way to create buzz about an upcoming study/report at Facebook's expense. I don't like Facebook, but if they/AP were really serious, we could see the report and look at the way they executed the whole experiment. The other data point is that how many did they try, cos as far as I know, they use AI to detect text on images. Probably not that effective in Thailand's local language.
One other aspect here, while Facebook deserves no sympathy, let's not forget misses happen even with editorial oversights. Let's not forget how NYT exclusive about nuclear weapons in Iraq was the final trigger for the war, and that report was false.
There are 7,100 official languages on planet earth, presumably all of them include language which can used to incite violence. If we hold Meta to a 'zero hate' standard, the only solution would be have native language speakers in every language, on call and ready to deploy in order vet every single advert before publication. Maybe Meta should indeed do this, but we can at least acknowledge that there are significant practical constraints involve to maintain a zero hate standard
If they're capable of launching the platform in every language, they're capable of moderating it in every language.
Uhhhhh this does not follow at all. "If you're capable of launching feature set A, you're capable of launching expanded feature set B" is asinine.
Said differently, if they aren't capable of moderating in every language, they they aren't capable of launching in every language. At least not ethically.
When you are Facebook, it must be table stakes.
Yes, you should have to vet your publications before they are released. That is a thing that should absolutely be part of your processes. If you cannot service a language, do not offer anything in that language.
My opinion is that huge tech companies have always wanted their cake while taking the last bite - they want everyone in the world to be their customer, but don't want to actually have to do anything to ensure those customers receive a quality service.
So block messages in any languages other than the ones you officially support? I could imagine people getting up in arms about that.
I'm confused by what your actual argument is here.
People would rightfully call it linguistic suppression.
Spoiler - it's our current world. They have the resources. I cannot be convinced that native language support for all languages in the world isn't an option for companies that make literally hundreds of Billions of dollars annually.
If you want to make money on a language via selling ads, you should be able to moderate that content in that language. I have no idea why you're fighting against that argument.
If Facebook sells ad space, they 100% are responsible to vet that the ad meets their standards before publishing. Not being able to read it because it's in a language Facebook is incapable of reading means they either need to decline to publish it (and not take any money) or hire appropriate staff to vet it (and pay them using the money you get from selling the ad).
The alternative is that they don't serve markets where they aren't willing to understand the language. Those markets are then served by companies who do and might actually put some effort into vetting what they're being paid to display.
I feel like most of the blame should be with the people who created the content, not with the people hosting it. I can go staple a call to violence on telephone poles, that doesn't make the phone company responsible imo.
They might very well be, if it weren't for Section 230. That can change. https://archive.ph/A5PEw https://en.wikipedia.org/wiki/Section_230
> Or is it only ads?
Advertising has been regulated for a long time now, in print, radio & television, so it wouldn't be odd to treat them as special on the internet.
Barbie said it best, "Math is hard, let's go shopping!"
It's like when you call 911 to report a murder in progress. They say "oh wow that sounds hard, sorry" and hang up.
Or you go to a doctor complaining of an irregular heart beat, and they say "that's going to be hard to fix, sorry."
Or you're writing an operating system, and you learn that scheduling is an NP-complete problem, so you stick with single-tasking.
Hard problems require investment of effort and resources. If facebook is profiting from its participation in genocide, it should suffer consequences for that. If they don't want to accept that responsibility, they should actively work to resolve the issue or stop offering the offending services.
The native language speakers don’t need to be on call either. Having a one business day turnaround for new advert approval is fine. They have longer turnaround times for similar things like WhatsApp message sending, so this is clearly acceptable responsiveness for them.
So Facebook’s decision is now: is there enough profit in publishing adverts in a particular language that it would justify hiring a native speaker of that language to review them? That seems like a perfectly reasonable cost of doing business, especially when the alternative is providing a platform for inciting genocide.
That's not how Facebook will see it though. The alternative for them really is slower growth and therefore lower profits. Facebook makes more money by jumping into foreign markets and spreading genocidal hate speech than otherwise. That's the cold hard truth. Because your excellent suggestion is really just common sense and being responsible. Why didn't they do your suggestion in the first place? Is it because no one at Facebook was smart enough think of your idea? No, of course they are smart enough. They just don't care.
Can we also acknowledge that not every business idea is destined for success, and that enabling any business to be successful, regardless of consequences, just isn't one of the things that can be legitimately asked of our society?
If a company has a brilliant idea for a solution that enables you to convert nitroglycerin to food, thus ending world hunger, but it just doesn't have the capability to implement safe handling of explosives, then it's not allowed to operate until it can.
Similarly, if a company has a brilliant idea for bringing people together, but it just can't moderate it well enough to avoid frickin' ethnic cleansing, then maybe it shouldn't be allowed to operate until it can.
Sucks to be Mark but some things really just are too dangerous to be allowed without some safeguards in place. It's literally up to Meta to figure out how to place them. If they can't, well, tough luck, but I'm sure they can think of other ways to make money.
You can't convince me that they were completely unaware that hate speech was proliferating on their platform until 2017, and here we are, 5 years, and not much has changed. We should absolutely expect and demand better.
No, it's that they want to become the platform and medium for all of humanity's communications and they want to digest the entire social graph and mine every human interaction for a place to inject advertising in a bid to get even bigger.
They want to swallow the world, let them sit on the hot seat. Forgive them nothing, and point out their gross incompetence and hypocrisy at enforcing their moral standard while trying to push their top end numbers up and up and up.
It’s like saying “there’s X death because of cars in the US, let’s prevent all cars from driving”
There are roughly that many languages known to science, according to Ethnologue.
Does anyone happen to know if the advertiser has to tell Meta what languages are used in an advertisement?
[1] https://www.globalwitness.org/en/campaigns/digital-threats/r...
Simply put social media should not be monitoring content other than graphic ones where it is clearly a violation.
It is only expected to take down content based on government or court orders.
One of the reasons why I have a big problem with social media moderating content is that it provides perverse incentives for the stronger party to stifle freedom of speech. A lot can be left to interpretation.
That doesn't seem like a particularly stark finding. Eight ads submitted by the same organization getting approved just tells us that they probably had a high reputation. Now, eight ads submitted by a mix of different actors all getting approved would make for a much stronger case, what we got here was somebody running the exact same experiment eight times and pretending to be shocked that it gave the same result each time.
- Facebook subsidised the internet in places that had never had it before — effectively making Facebook free in those areas and making it the main source of news for many people.
- They have very little content moderation in these languages.
- The algorithm optimises for engagement. Anger is the easiest way of creating engagement, and lies are an easy way to spark anger — hence the inciting of violence and genocide.
- Facebook profits
Mark Zuckerberg and his staff are responsible for this, and they should face consequences — but they wont.
Is this surprising to anyone who has spent time on the platform? Even outside of advertising, the platform is filled with violence and hate.
Many pockets are equivalent to the content that can be found on 4chan, with thinly veiled racism and LTGBTQ+phobia hidden behind emojis and jargon that is clear to humans but opaque to algorithms.
Reports against groups such "[Black circle] should unalive" or "[same sex couple emoji] are [slur]" where the content is 100% hate speech are routinely flagged as "It doesn't go against one of our specifit community standards". Even when they contain nothing but photos of other Facebook users and links to people's profiles with invitations to brigade and spam hate.
To my eyes, that's one of the main factors of the migration away from Facebook yet it is so rarely covered.
No. "Your house is still on fire" may not be news, but it's still relevant to keep talking about until the problem is solved.
Yes, those 8 paid ads should absolutely get taken down and Facebook should be held responsible for it. But the remaining hate groups will still breed genocide, especially when the users of those groups are well aware that their actions are protected by the platform and will not be taken down. Worst of it, they are often put back online after the group managers ask for a second scan of the reports.
https://news.ycombinator.com/newsguidelines.html "Please use the original title, unless it is misleading or linkbait; don't editorialize."
The title of the AP article is "‘Kill more’: Facebook fails to detect hate against Rohingya." The submission title is "Facebook approved 8 paid ads inciting violence and genocide against the Rohingya." The editorialized version intentionally misrepresents Facebook as endorsing violence and genocide rather than automated systems failing an explicit test (these weren't real ads).
Can we please do better?
So while such ads/posts might not make you want to kill the "others" immediately, there is significant population which will get radicalized over a period. Then one trigger event is all it takes for a mob to turn into murderers. You can read about how Radio Rwanda fueled the genocide.
IMO the difference between who gets radicalized and who doesn't is based on how much exposure one has to different cultures. But then my sister who has visited Auschwitz supports genocide, so there must be more to it.