A machine learning approach to identifying thought markers of suicidal subjects | Hacker News Reader

A machine learning approach to identifying thought markers of suicidal subjects

onlinelibrary.wiley.com

142 pointsmisnamed81 comments

A discussed version of the paper[1] says:

>A new study shows that computer technology known as machine learning is up to 93 percent accurate in correctly classifying a suicidal person

P(+|suicidal) = .93

I want to know P(suicidal|+) [2]

[1]http://neurosciencenews.com/suicide-machine-learning-5448/ [2]https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing

Yes! That is what is scary as fuck with these "magic classification" techniques. The more reliable they are, the more people dismiss false positives. This, applied to criminality is totally 1984.

For those that don't know, "this applied to criminality" already exists and has some problems.

Propublica wrote a story criticizing it's use: https://www.propublica.org/article/machine-bias-risk-assessm...

The company responded with a rebuttal: https://www.documentcloud.org/documents/2998391-ProPublica-C...

And Propublica has a counter-rebuttal: https://www.propublica.org/article/propublica-responds-to-co...

Propublica has an "interesting" definition of "fairness". It turns out there is a beautiful mathematical result that shows that the different definitions of "fairness" are inherently incompatible with each other and cannot be satisfied at the same time.

This is the best article that summarizes the controversy on this subject:

https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1...

Paper by Jon Kleinberg and Sendhil M. and Manish Raghavan on this topic https://arxiv.org/abs/1609.05807

What happens where these kinds of models are the reason for suicidal thoughts or criminality?

Quite right. I would have expected any paper to include both.

This is quite a good article on the subject: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

> P(+|suicidal) = .93 > I want to know P(suicidal|+)

Can someone explain what the difference between these 2 is? I'm not versed on statistics.

I believe you read the | as "Therefore" (possibly going for ⊢ ?), as in "The probability that someone who scored highly on this test is also suicidal is .93" They want to know that the probability that someone who is suicidal scores highly on the test. The test could be good at identifying suicidal thoughts in only a small portion of the population, since they have demonstrated a good false-positive rate (i.e. 93% of subjects that it identifies as suicidal are); but not good at identifying which members of a population are suicidal - that is, if given to all of a population, only find some small percentage of people with suicidal thoughts.

Thank you! I think I understand now, this clarifies a lot!

Read the pipe as "given" (or "when"). P(A|B) = "probability of A given B".

You've got people who are actually suicidal. You've got people for whom the test comes back positive. There are two subtly yet crucially different metrics. The paper reports the answer to: if someone is positive, will the test come back positive? The flipside is very important for customers of the test: if the test comes back positive, does that mean I am suicidal?

P(+|suicidal) = the probability that the test comes back positive given the patient is actually suicidal = 0.93

P(suicidal|+) = the probability that the patient is suicidal, given that the test came back positive.

As a degenerate case, imagine Nat's Suicidal Tendency Detector.

10 PRINT "SUICIDAL"

It will correctly detect every single suicidal patient put in front of it. P(+|suicidal) = 1. IT'S A MIRACLE BREAKTHROUGH!

That test's critical failure is P(suicidal|+) ... it's identical to the rate in the population. As you could have guessed by reading the source code, taking the test doesn't give you any more information about the patient's suicidal tendencies.

Whew. Hope that was coherent! Google "conditional probability" if you want the math of how to work with these.

The linked article contains exactly this information, though. Figure 1 showed the Receiver Operating Characteristic curve, which they called something else, but it's the standard way of discussing this. It plots the classifier's true positive rate versus the false positive rate at various settings thusly: the y-axis plots the "sensitivity", which is the true positive rate. The x-axis plots the "specificity", which is equal to (1 - false positive rate)... but because everyone is thinking of the false positive rate anyway, you'll notice that the specificity goes from 1 to 0.

Incidentally, these are plotted as a curve because it's trivial to build a classifier with a 100% true positive rate (call them all suicidal) or a 0% false positive rate (call none of them suicidal), but just how "good" your model / algorithm is is a function of how it performs with both.

I can appreciate the effort made in the research, but classifying possible human behaviors is exactly a field where I don't want machine learning to go.

There is behind machine learning both a phantasm - artificial intelligences that can guess what humans can't - and a reality - it's just statistical models that are never 100% accurate because, well, that's an attribute of statistics.

Those two elements combined and applied to behavior classification sounds like a scary thing, not unlike the kind of errors eugenics made, over trusting their science to apply it on social facts, totally discarding empathy and individual context.

> classifying possible human behaviors is exactly a field where I don't want machine learning to go

Advertisers are already doing this on a massive scale.

If they make an error, it's their own problem, so I'm fine with that. I'm more concerned about researching such models to decide policies or act upon as administratives. I don't think classifying suicidal behavior is something most advertisers are interested about :)

The content you're presented with on the net is more and more controlled by algorithms. They provide you with stuff they think you'll like. This has an influence on your opinions. On a large enough scale this can very much influence policies and societal developments.

>I don't think classifying suicidal behavior is something most advertisers are interested about :)

Facebook has shown an interest in manipulating depressive states of it's users.

http://www.pnas.org/content/111/24/8788.full.pdf

And I hate it

Are you saying that the cost of doing this is low because of advertisers' experience? Or are you saying that it's ok because we already have a slightly less intrusive version of this in place?

But humans already make those judgements, and when they make errors, it's tragic too. If a psychologist erroneously decides that you're a menace to yourself and should be institutionalized, would it make you feel better that it was a human that made this decision? Even if AI would make the same error with a much lower probability?

This is a good point. I guess the difference here will be "how much do we trust human judgement?" and "how much do we trust AI predictions?". If AIs make 10% less errors than humans but we have 50% more trust in them, that's a problem. Eugenics were not a problem because science was wrong, but because people blindly trusted it to apply where it should not.

There are a lot of things to consider, here, those are exciting times for thinking.

The concern isn't the lack of humanity, it's technology allowing for massive scale. The potential damage of a buggy machine learning classifier is much higher than that of an unskilled psychologist.

I can't speak for OP, but that is one reason I want to end all involuntary treatments. Getting locked up in a psych facility is a nightmare to me.

But you can at least sue a doctor and get his license revoked. It is much harder to sue an algorithm.

> I want to end all involuntary treatments

Are you sure you have thought through all implications? I mean, getting locked up sounds pretty scary, but there are a lot of other scary stories that can happen if you really ban the whole concept of involuntary treatments.

Meh, this is a bit alarmist. We've been using machines and statistics for classifying human behaviors from the get-go. It's one of the most important things in analytics, mainly because money comes from people.

So you're saying we're used to it, now.

You're saying it's OK because it's profitable?

If both treatment and non-treatment could have dramatic consequences, I would agree with you. Criminal justice is a classic case of that.

In this case, such a tool can be used to flag behaviour automatically (Facebook does that, for instance) to start a process, i.e. have a conversation. “Depression” does not have hard-set limits, and caring about someone who had a bad day is not problematic, talking about how to deal with rejection is appropriate. Forcing them to take mind-altering substance is not ideal, but I can’t imagine any licensed doctor doing that just because some patient’s score is high, even if they do not qualify otherwise; they receive a decade of training to teach them nuance. As someone who lives with a psychiatrist, I can confirm: no one is skeptical of classification any more than the people doing the rating.

Having measuring tools (imperfect as they may be initially) is what allows science to try opposable theories, and psychiatry needs this (and plenty more tools).

But doctors getting massive amounts of training is exactly the kind of problem computer science seeks to solve with a machine learning approach. There aren't enough doctors around to monitor every patient, so machine learning takes over. Now doctors can exactly focus on a score, and say, hey we didn't even see this.

> no one is skeptical of classification any more than the people doing the rating.

And perhaps the people being rated.

> classifying possible human behaviors is exactly a field where I don't want machine learning to go.

What do you think doing A/B testing on cohorts of users is?

This is subject specific journal reporting a "machine learning" study with just ~400 odd patients dataset. I would not take these results seriously. The accuracy number cited is meaningless, and other than being a bullet point on someone's CV it does not have much of a value.

85% Accuracy or 0.8 AUC score is pointless, unless compared with current state of art e.g. having psychologist give an opinion and comparing against the correct population, e.g. all patients who get interviewed as opposed to a balanced set.

Based on [1], I think I can build a model that outperforms this:

    return False

[1] "An estimated 9.3 million adults (3.9% of the adult U.S. population) reported having suicidal thoughts in the past year." https://www.cdc.gov/violenceprevention/pdf/suicide-datasheet...

But the patient pool was drawn largely from psych units, where your (very efficient!) classifier might not work as well. The real headline should probably be "identifying suicidality in patients already admitted to psychiatric institutions".

That 85% 'accuracy' is ambiguous.

I couldn't get into the article, but if that number includes false positives, that's not really practical. Suicide is really rare, so it could just be that it picks a 15% of the population that includes al suicidal people. That means the vast majority of those positives are false positives.

Basically, its the precision and recall that matters.

I think machine learning people use a combination (specifically the harmonic mean) of both when evaluating:

https://en.wikipedia.org/wiki/F1_score

They should use it, but articles often drop that (or similar) numbers. It's easy to get headlines if you report 85% accuracy as it sounds good and is easy to understand. Reporting the proper numbers is harder to understand for people who are not working in the field and you'd quickly discover that the model doesn't actually work.

I'm not even sure why people report ONLY accuracy as an evaluation metric. It means nothing - appropriately weighted F scores, precision, recall give more insight into the system.

Completely agree. Although if your binary classes are perfectly balanced then accuracy is a valid metric, but that's rarely the case.

I guess the problem is that an ROC curve wouldn't create flashy headlines.

> I guess the problem is that an ROC curve wouldn't create flashy headlines.

You can, you just need to put in a little bit of work. We don't need to specify the exact accuracy in the title.

I wouldn't call suicide really rare, it's the 10th most common cause of death in the US

http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm

Although nowadays you have to view these statistics carefully, for X, Y, and Z causes of death inevitably rise as I, J, and K get knocked down by modern medicine, all we've done to reduce many types of accidental death, from 5 gallon buckets to cars, etc.

But, yeah, in this case I agree it's a big problem in the US.

"classify 379 subjects recruited from two academic medical centers and a rural community hospital"

So they recruit people from hospitals and check if they ever thought about suicide/self death? Hmm okay nice test group....

Everybody, healthy or not healthy, in life will think about how it would be or how they would end themselves, it's human nature. Though people in hospitals, potentially sick, generally older, for sure will have had those thoughts, because they often had issues that makes them lets say less enthusiastic about life?

Surely it is more normal to fantasize about a Tom Sawyer funeral than actual suicide?

How it would be/how to do it - completely different thought processes.

I don't know, i'm no expert, and certainly not a judge of what is normal and what is not. I do know that 'normal' isn't the same for everybody.

But if you get a serious event in life (example, death of family member or other loved one), that makes you doubt if you still enjoy life or whats the use of 'finishing' it till your body gives up.. those thoughts will occur, sometimes swiftly sometimes occupying your brain for longer. Do keep in mind, having thoughts and actually acting upon those thoughts are two different things.

Though pondering if tom sawyer has brass handles on his coffin.... Maybe if i was reading tom sawyer, yes that would probably be a thought that came before others. :) Have to admit, i never read that book, so not entirely sure why one would fantasise about his funeral.

Tom Sawyer is sitting in the rafters watching his own funeral. There's more than one sequel, and he survives all of them.

I guess that people doing research into suicidal ideation know that it comes in different forms - from well defined plans with means and method available, to abstract thoughts of "things would be better if I wasn't here", and that they will have taken different forms if ideation into account when preparing the research.

Suicide is a big taboo in the 'western' christian society, but there are (and have been) lots of cultures were suicide is looked at differently - as something honorable, good, etc.

Even our culture has different interpretations for it - eg suicide vs sacrifice.

We morally condemn the former but elate the latter, even though the outcome is the same - a person dies.

Metaphysically, the meaning of suicide is given by the perceived meaning of death and the [lack of] belief in some sort of afterlife.

Is it a sin or not ? If there's an afterlife, will you be punished for committing suicide? What about the loved ones ? They will judge..

But for example, if you knew that this life is actually a realistic VR simulation that you've entered into, then suicide would be perceived as a sort of 'ESC' key - a way out of the simulation. Like exiting a game.

If a person's circumstance in life is such that the person is bound to suffer until death (eg. disease, mutilation or loss of everyone), then suicide might be looked at as a sort of release - a good thing.. There's the whole controversy regarding assisted suicide..

Then there's the sacrifice - going into battle screaming is a form of attempted suicide combined with attempted murder. At the end of the day, the battlefield participants are eventually split into killers and those who committed suicide. From this perspective, going to war is collective [attempted] suicide.

War is a form of temporarily suspending the moral rules we obey by (do not kill others or self) and people gladly participate in both killing others and themselves.

I guess my point is that tfa is looking at a very narrow spectrum of 'suicidal people' - suicide is a lot more prevalent than that and it's practiced not just by people with mental illness..

> suicide is a lot more prevalent than

If you are talking about war, then no, that's not suicide. The aim is not to die, even though it might be likely.

Even suicide bombers don't have death as their primary aim - if they achieve the same result without dying, they would.

> Even suicide bombers don't have death as their primary aim - if they achieve the same result without dying, they would."

That depends on how you define "the same result". The assumed (in some cultures) moral superiority of the suicide bomber regardless of the legitimacy of the target can have considerable PR value. In a way that somewhat resembles a ponzi scheme, those at the very top may reap higher rewards, and so choose to elevate the tactic into a strategy.

Consider "the result" to include religious goals based on a belief in the afterlife. If they could obtain piety some other way they could - as opposed to wanting to end life as it is no longer enjoyable.

And you now know why some groups (religions, sects, etc.) use (or have used) martyrdom (including the suicide bomb variety) as a tactic, and others have not:

More than one goal exists for each group, many of these goals are not mutually compatible, and different groups arrive at different mixes of tactics in pursuit of their goals and to resolve (or ignore) the various contradictions and incompatibilities in different ways, and these mixes change over time in reaction to a changing environment, including the S&T employed by other groups.

IOW, don't expect the strategies or tactics that groups use to settle on any sort of Nash equilibrium.

They are measuring suicidality in patients of depression. This is a legitimate medical issue: disease leads to death.

The problem of suicide isn't so much whether killing oneself is "wrong" or not, or the possible impact of someone's death on their social circle, but that it's extremely unlikely that any boundedly rational agent acting under imperfect information can truly commit to an irreversible choice. The "permanent solution to a transient problem" quip is very much true: way too many who wake after a suicide attempt take it as an opportunity to regret what they've done, and try to turn their lifes around.

Great, now society has an excuse to put scary people away with essentially no due process thanks to the magical label of "suicidal". For those living with mental illness, reasons to despair of safety and fair treatment are up, up, up. Seeking help will never have seemed so dangerous.

Great, now society has an excuse to put scary people away with essentially no due process thanks to the magical label of "suicidal".

Only saving grace is that in the US since the '60s-'70s we've been too cheap and too "enlightened" to do this unless the threat is truly dire. I wouldn't worry about it here, although all responsible mental health workers will during a visit ask you about suicide issues and I assume you should answer truthfully (weasel word since I've never been suicidal).

As for needing an excuse, many examples like the Soviet one show no real excuse is needed for true persecution.

Not to mention that even if you're admitted voluntarily, you aren't necessarily allowed to leave when you want, even beyond the oft-quoted 72-hour hold. Legally you might be entitled to leave, but the facility can basically force you to stay by denying the resources needed to fill out the discharge paperwork.

So does this mean it misses 14 suicidal people for every one it finds. Or that it classifies 14 people suicidal who aren't for each one it finds? Not super great either way!

If it is what it claims, it's probably better than any human

Is it?

Suicide risk assessments are mostly worthless. They're done because you need to show you've done something to try to identify people at risk of suicide and to have tried to prevent that death.

And how accurate are traditional methods? Seems to be behind a paywall.

Exciting, whatever the answer is. It's awful when someone commits suicide, and if there's some automated way to test it, that would help a lot. Also less intrusive. Who knows how many people need help but are too shy to reach out.

This is also one of those perfect applications where it's quite okay to have a false positive.

Is it really?

Everyone, sudhirj is suicidal! Please contact everyone who knows him and ask them to reach out. I'm sure this is not going to damage his or her interpersonal relationships or career prospects at all.

Holy , you can see why I wrote my top-level comment. Even here people are already jumping on the "who cares if it's accurate, take away their freedom and their dignity at the first sign of trouble!" bandwagon.

"Hi this is an automated message from Facebook alerting you that the police have been called to your husband's/friends/boss's location to prevent him from killing himself"

"Actually just kidding, we had a little SNAFU. Thank you for being a loyal Facebook user, we assume this had a neutral effect on your life"

Actually Instagram does send a message to the user if enough people mark the user in danger of being harmed. Similar with Google where if someone searches for methods of committing suicide the user will get a number for helpline as first result in bold letter.

https://techcrunch.com/2016/10/19/instagram-tackles-self-har...

I don't know, the first application I can think of is raising life insurance fees for people who may be suicidal.

"The thought of suicide is a great consolation: by means of it one gets through many a dark night."

-- Friedrich Nietzsche

(Not exactly the most balanced bike in the shed, but some people like it that way ...)

I second this.. The idea that there's an escape route is a relief, when you look at a big problem(that you have to solve) and your brain panics(it definitely puts the problem in perspective). However, i'm still divided as if having this is a good habit for self-growth or just an excuse to ignore the problem and work around it.

It might be healthier to not hold on to suicide as a last escape from failure, and just accept the possibility of life after failure. The thought process that you're illustrating here seems like part of what's driving family annihilators: the fathers who would rather kill you than fail you.

My friend was suicidal and we tried to get help from the authorities but got none. None at all. He later committed suicide for real and this makes me wonder why it matters to discover it if you're not even helping people that is asking for it?

I don't understand your viewpoint. This work isn't being done by your local authorities and isn't an alternative action to their providing help.

It provides clear potential benefits (if it performs as well as implied) for numerous people at risk in the world, which is a substantial goal that can be at least be applied in places that do have better forms of assistance. Perhaps if your local authorities do eventually implement better protocols, this will be of use for future suicidal residents there as well. I understand that you're trying to raise awareness for the lack of help from authorities, but it seems awfully dismissive of work that seemingly has nothing to do with the first problem.

What do you test against to see if someone is indeed suicidal? Self reporting, or do you wait around till they die and see if it was due to suicide or something else?

Think they're just looking at self reporting. Otherwise n=400 gonna be small.

You send them a suicide pill and see who takes it.

Just kidding, that wouldn't be scientific. Half of the pills are actually placebos.

From Figure 1 (which illustrates some ROC curves):

> The gray line is the AROC curve for a baseline (random) classifier

The AROC is the area under the ROC curve, not the curve itself.

Also, the scale on Figure 1 for the x-axis (corresponding to the False Positive Rate, or sensitivity) go above zero and below one, which doesn't make any sense.

Suicide is a very serious health issue that affects an astonishing number of people worldwide. Any advance that helps detection and treatment is to be welcomed.

I'd love to take this test.

I don't really know the answer to if I am or not.

For instance, since everyone dies, I'd certainly like to die in a fantastic and wonderfully improbable way. Or if a doctor told me I was going to die soon, I'd accept it and start doing really irresponsible and dangerous things...

I wouldn't be like "oh dearest me, I must clasp on to life longer." Is that suicidal?

No.

Not even close to suicidal.

Thought conformity at our doorsteps.

Reply on news.ycombinator.com