In Screening for Suicide Risk, Facebook Takes on Tricky Public Health Role
nytimes.com
nytimes.com
The cops advised me I shouldn't be walking in the bike lane that everyone walked in because there was no sidewalk on that side of the road. Having advised me of this, had they stopped me again, they would have ticketed me. I then had to routinely expend extra energy to go out of my way going "home" every single day.
No, the cops absolutely weren't going to help me solve any of my actual problems. No, they absolutely didn't care about me, no.
Calling the cops on a stranger because you supposedly care about their welfare is asshole behavior. I don't care at all about your supposed justification or algorithms etc.
If you care, you check on them yourself or maybe call an ambulance. If you don't know them well enough to do that without it going weird and shitty places, then mind your own damn business.
Also, the remark about ambulances wasn't in my original comment. I added it after the fact, perhaps a mistake.
I can see few situations where they would send only an ambulance. The closest is if you said you simply overdosed on pills. However, at this point you need to go to the hospital. The ambulance has no ability to force you if you change your mind. Police would likely be sent along to make sure you didn't change your mind.
At least in Germany, there historically is/was (even though this is not always the case anymore as far as I am aware). 110 for police and 112 for ambulance service and fire department.
There is a good reason for this: if you and a group of friends did some experiments with, let us call it, substances that did not go well, you want to call an ambulance to get them medical help. But under no circumstances you want that the police to be informed of this accident.
Our institutions are failing us. https://www.ted.com/talks/barry_schwartz_using_our_practical...
No, seriously. Facebook should host content and serve ads. That's it.
They don't need to worry about being the arbiter of truthful news.
They don't need to stop "hate".
They don't need to make calls about who needs a psych eval.
They just need to host content. But if they truly want to do this, what happens when someone sues them for not making the call?
I made the new year's resolution to delete my account yesterday. I almost chickened out, but then this article comes up.
I would agree with you that Facebook should not be an arbiter of truthful news or stop hate.
But they are indeed an arbiter. They continue to push content to users to keep them consuming their product. They promote content that makes people engaged and that usually means content that makes people angry.
If they stop inserting posts from sources users don't follow and not rank stories from friends, I would agree with you.
Once day, their researchers decide to test how accurate the model is by setting real time flags, but take no action. And let's say, it turns out, of 100 people, 75 people did end up taking their life.
What would you propose they do?
For sone folks, like me and the GO, there are no “buts” about it.
# of deaths reduced is not the sole factor we should use to determine whether a particular course of action is good
A free society is also often a risky society, but the rewards of freedom outweight said risks.
Without really telling anyone, they install a webcam and point it at the driver to see whether they might speed on a certain day, based on facial expression and skin color. Why those? Just a weird hunch! They find that the person actually did speed. (Turns out, they didn't really tell anyone that all your driving data was uploaded to their servers, too!)
The problem is two fold. First, they shouldn't have been collecting this data on their users in the first place. The manufactures job is to build a car, not ensure that the car is being used responsibly. This is simply an impossible task to charge anyone with, and the implications are frightening.
Second, I'm not a believer in using algorithms to predict behavior. Too much black box "mangle the numbers until it works". You could make an argument that there's a correlation between skin color and speeding. You might even be able to make pretty charts, but I think this is simply bad science.
You make it sound like this model would actually continue to work at 75% perpetually. I think that it's far more likely it worked this one test, and performance would trend sharply downwards. However it makes this decision, I think it's about as absurd and wrong as the crazy example above.
On the other side of the coin, people are putting too much trust in magic algorithms they don't properly understand, and are far too complacent about egregious invasion of privacy.
For example, if Facebook can identify suicide risks in Ohio, then they will be pressured to identify gays in Saudi Arabia and Muslims in Xinjiang. Or weed smokers, underage drinkers, and a bunch of everyone-does-it-but-nobody-cares crimes, where the punishments are technically quite harsh.
The only effective standard that differentiates suicide from the others is "what Facebook's executive thinks is morally acceptable". It is certain that Facebooks executive are not the right people to arbitrate moral issues, not matter how black and white.
There best layer of protection we have here is Facebook taking an official position of "we don't judge, we just sell ads". It isn't perfect and it isn't necessarily logically consistent, but if Facebook sees itself as an arbiter then sooner or later it may well disagree with something we do and take action to stop us doing it.
I've had two friends who went to take advantage of university counseling services for anxiety and depression that they wanted help with, only to be committed to a local hospital involuntarily for saying the wrong thing until their families intervened to have them discharged. The episode would have merely been stressful for them, but they were each restricted from returning to their classes until the next semester. Neither of them were at any level of risk for suicide or self-harm, but the universities they went to naturally were obligated to err on the side of caution.
Such events are relatively rare, but keep many students who are suffering from mental illness from taking the risk of using available resources.
I'd honestly hate to be committed because I was a little too sarcastic in a Facebook comment, and much like public universities I fear Facebook is incentivized to make the least risky call.
Even more: Exactly these kinds of rules lead to a strong incentive to simply commit the suicide because talking about it, will lead to worse consequences.
In recent decades, psych-laws have become more strict because as a society we have decided that it is better to involuntarily detain 20 people for a few days rather than let one person end their life. That seems like a pretty fair tradeoff to me, although one that reasonable people could easily disagree on. If you disagree, you should be lobbying your state psychological association, not complaining about random therapists at colleges.
There’s little to no evidence required to keep someone for days or weeks under “observation”. Some states setup entirely separate court systems without due process, since commitment is a “civil matter”.
Some amount of blame falls with states allowing this type of behavior.
The burden is on psychologists to demonstrate the practices work and hospitals are properly equipped.
But I'd rather that they did do these things. They have an opportunity to make some people's lives better by doing things like this and I think it would be nice for them to do that.
It’s an imperfect predictor and Facebook will make a mistake which will eventually be followed by a police shooting. After all, how many stories have been published of police called for mental health issues that resulted in the shooting of the person the caller was worried about.
I hope the programmers are ready to explain how it worked because I get the feeling the family’s attorney is going to have an easy time with black boxes, machine learning, and no true tracing of why.
The only legal issue I could see is violation of privacy, but I bet their EULA covers them there.
Now, if they screw up bad enough, I could see it precipitating some new legislation that they might not like. Until then, I think they're in the clear.
It's another one of those situations where everyone is incentivized to "do something" whether it's actually helpful or not. "Student dies after symptoms of suicidal depression went ignored" is a splashy headline. "Student unable to graduate after involuntary psychiatric detention" is not.
Facebook isn’t exactly the most trusted and liked company right now. So they are exactly the type of entity a lawyer would love to get in a court room.
Yes, you can try to sue for anything. Also, yes, because the lawyer will try to show it was not "good faith" but a failure of software that caused the tragedy. I'm sure the discovery stage will yield enough to make it painful for Facebook. The current feelings about Facebook are going to come up.
A EULA will not protect you from the families of dead people.
Point taken.
> the lawyer will try to show it was not "good faith" but a failure of software
AFAIK, the phone calls are being placed by Facebook employees, not software. Presumably, these employees have to look at the actual post content to even be able to make a coherent phone call. You would have to argue either that the employee was incompetent, or that the employee was coerced into making an inappropriate call.
Proper training of police (years, not weeks) and a policy of deescalation have at least as big an effect as rates of gun ownership
Has it occurred to you that cause and effect are non-obvious?
AIUI part of the reason most UK police does not carry a gun is precisely a policy of de-escalation — if the police doesn’t carry a gun, criminals don’t feel the need to match their firepower. Inversely, the fact that American law inforcement treats their beat like a warzone probably contributes to an escalation of violence on the other side too.
Same concept different application: escaping prison isn’t a crime in Germany. You will still be chased by law enforcement to serve the rest of your sentence but the act of running away isn’t a crime unto itself. This means an escapee is likely to avoid committing further crimes on their way out to avoid worsening their sentence, while an American escapee, facing a large penalty for escape itself, is likelier to take an in for a penny, in for a pound approach.
I'm not convinced. Committing crimes, esp. robberies, is much easier with guns. Not to mention that a lot of criminals are involved in intra- an intergang relations and in such environment, a drug dealer without a gun (for example) wouldn't last very long.
It seems reasonable then that this might weigh in as an occupation stressor for police officers.
Especially considering unregistered firearms outnumber registered firearms by 392:1.
In 2013 ... there were 33,636 deaths due to "injury by firearms" (10.6 deaths per 100,000 persons). These deaths consisted of 11,208 homicides, 21,175 suicides.[1]
Approximately 1.4 million people have died from firearms in the U.S. between 1968 and 2011.[2]
That’s approximately ten times the number of soldiers who died in the American War of Independence.[3]
1. https://en.wikipedia.org/wiki/Estimated_number_of_civilian_g...
2. https://en.wikipedia.org/wiki/Gun_violence_in_the_United_Sta...
I’m not convinced that someone that’s willing to shoot a police officer would be unwilling to smuggle weapons internationally.
Guns also get stolen all the time in Europe, yet we still manage.
https://en.wikipedia.org/wiki/Political_abuse_of_psychiatry_...
A few of us were following what he was doing, and at the end of the quiz he got the result effectively saying seek immediate professional help. Shocked, he exclaimed "What?! I'm having the time of my life!"
Turns out there's an overlap between certain symptoms of depression and being madly in love... and he just got a new GF a week earlier.
Such as? I'm having a hard time coming up with any.
My guess is the wording in the quiz was poor.
I recall the that the questions made sense given the goal of the quiz, but the questions were probably too broad and, which was kinda my point, they lacked control questions to rule out other causes for those effects.
So now platform that arguably actively participates in making people depressed has launched an human assisted AI to report people to authorities. How Orwellian is that.
This is not "Orwellian". And the keyword in the claim about Facebook making people depressed is "arguable".
Times was giving Facebook crap that Facebook Live promoted/publicized suicides, so doesn’t quite compute that their attempt to do something about it is also bad.
If Facebook's algorithms were promoting suicide-related posts, then doing something about it might be good, but it does not follow that they should also take an active role in suicide prevention.
You are creating a false dichotomy, both things can be bad at the same time. E.g.: being a thief is bad, being a vigilante is also bad.
Safety/effectiveness studies certainly seem like a good idea, though I'd be really hesitant to prohibit Facebook from doing this, at least not from worry about false positives (the privacy implications are a different issue).
Facebook is just the first in a line of common-sense checks: a Facebook algorithm detects that someone seems to be suicidal, this escalates to a Facebook employee who reviews it and in extreme cases escalates to law enforcement, law enforcement gets the information from Facebook and decides how to respond, and may end up pulling in psychiatric medical help, which of course involves trained people making judgement calls.
Facebook's role is to escalate, not to intervene, and even that escalation process is done by a human. It seems very unlikely that it would ever result in a situation where someone gets involuntarily hospitalized because some machine learning algorithm had a bug.
Per TFA they don't actually track the outcomes of their calls because that would be too much of a privacy violation. I'd expect the resulting lack of feedback to be problematic.
Also, I'd say that escalating to the people authorized to use force is a form of intervening.
They would like to cover their asses in case something bad happens.
If they didn't act, they would be susceptible to lawsuits.
they would be susceptible to lawsuits
Perhaps, but people being involuntarily detained because people misunderstand communications those users thought to be private seems much more likely. I have a group of friends who share memes about suicide and make jokes about it constantly. This is actually pretty common. If some uninitiated outsider read some of my DMs, I’d expect some people I know to be on their way to hospital in handcuffs.
More importantly, why does Facebook think this is their job, and why would anybody else agree with them about it? The idea of the postal service sending the police to my house because they were concerned about the contents of a letter I wrote is horrifying, or the phone company because they were worried about a call I had. I’m not any less horrified about facebook doing this, especially considering how much less I trust them than the postal service or Verizon.
I'm no longer on facebook, but it would be really useful if something I open daily would tell me "hey dude, you're overtalkative and online at odd hours and maybe you should get it checked" -- but give me the agency to do so. I understand perfectly well that there are mental illness scenarios where the capacity of a person to help himself is lost (making involuntary intervention the humane course of action) but even in mania this is the exception rather than the norm.
I can't think of a model of illness severity where machine learning is able to determine a score but the time course of this score is expected to be discontinuous. Before any noise of escalation without my knowledge, I would expect tools to be developed that would help me realize I'm getting ill before losing agency. Everything I just said works for depression too.
But I agree that it would be great if there were more tools to help with self-awareness and guide people to get help. I've had plenty of mental health problems in the past, and any source of better self-awareness seems useful, even outside the context of seeking professional help. Still, at least with my Facebook usage, it seems very unlikely that it ever would be able to know when I'm depressed except when I'm already in the process of getting help from a friend.
This sort of thing already exists a little bit. A much tamer variant I've seen is Nintendo games that tell you to take a break when you've been playing for a while. And Google searches involving suicide show the message "You're not alone. Confidential help is available for free." and direct you to the suicide prevention hotline.
Please do not play with people's lives based on an algorithm whose classification output we have no way of explaining.
Data collection may be bad, but suicide prevention is a beneficial byproduct. Data collection may be bad, but the New York Times won't be accused of leading by example. The article is packaged to ping Google, Amazon, and yes Facebook among it's 32 scripts. It's unlikely any of this there to serve a higher common interest than suicide prevention.
The examples in the coverage show how far Facebook outclasses the New York Times.
Facebook said its suicide risk scoring system worked
worldwide in English, Spanish, Portuguese and Arabic
It's reporting is parochial. Interviewing two experts in New York, four police forces on the east coast and Ohio. To me as a non-absolutist, judgment about the initiative's tradeoffs ought to consider stories from Brazil, Morocco, Mexico, and Canada. Those people don't enter the parochial conversation at the New York Times.
Preventing suicides is the kind of difference journalism is supposed to make in people's lives. The New York Times is driven of "weapons of mass destruction" reporting on Facebook's suicide prevention initiative because it is so completely outclassed as an instrument for journalistic impact on the lives of individuals.
You need strong proof to claim "Facebook has become the most dangerous online company". Recently, Facebook groups were used to organize the yellow vests movement in France. For non-techies, Facebook does provide a new way to communicate.
You could always argue "oh yeah, but it already existed 15 years ago with X", but Facebook made it available to the masses.
The security/social concerns happening recently create new opportunities for Facebook's competition, so I wouldn't be that alarmed.
Google, on the other hand...
Amazing this is considered bad by the commenters here.
It's easy to feel like what you say online doesn't really count. People feel that way all the time and that fact often causes problems. Usually, though, those problems don't involve someone calling the cops on you "for your own good." At most, you get banned from some forum.
Indeed. Imagine if Facebook called CPS/police every time a user wrote "my dad/mom/partner/boss is gonna kill me when he/she finds out."
They do have context given how much of a person’s internet life they can “see”. While you can argue that is bad, if they have enough signal a person is likely to do harm and the person says something like that, it seems like it’s a net positive for Facebook to alert someone to help the person out.
So my experience really doesn't fit with what you are saying at all. My experience is that what people say online is the tip of the iceberg of their life and the world imagines it has greater context than it really has and leaps to weird and inaccurate conclusions on a pretty regular basis.
Trying to figure out how to express myself so I feel like I'm actually understood takes an inordinate amount of my time. I have reason to believe most other people don't spend anywhere near as much time and effort as I do on trying to sort that out.
Most of the time, most people don't seem to get as wildly and horrifically misunderstood as I seem to so often end up being. Or if they are misunderstood, it isn't that big of a deal in many cases.
But I remain aghast at the idea of someone calling the cops on me because of something I have said online that people imagine they understood that was not a "threat" against anyone but me.
On top of being skeptical that Facebook understands you as well as you imagine they do, I am also pro right to die.
In a nutshell, if people give a damn about my sorry ass, they could choose to do something meaningful in the here and now to help me make my life work instead of being part of the problem, making my life harder than it already is and then pretending to themselves they are Good People by calling the god-damned cops on me for leaving a fucking courtesy note somewhere online if I decide to finally check the fuck out of here.
Note to self: Don't bother to leave a courtesy note online informing anyone of fuck all. They really don't deserve such courtesies. Geez.
You can argue on the term "mixed result", but I think it's reasonable to assume they're indicating that non-success is a negative result, even when Facebook is going entirely above and beyond here to begin with, so there really isn't any "negative result" to be had.
Do people dislike big tech companies so much that reporting immediately concerning messages and suicide attempts to the police must be a bad thing?
This is common practice anyway. If you write that you're about to kill yourself on Reddit and a moderator notices it, I sure hope the police are contacted. Nothing I see to criticize here at all.