Hacker News Reader

Comment by skissane | Hacker News Reader

You can also write things which sound violent (when read literally) but are actually innocent.

My wife's cousin recently had a baby, and posted a photo on Facebook. My wife commented (something like) "She's so cute I'll have to kidnap her". Facebook locked her account for 24 hours for making a "violent threat". I am sure the mother wasn't threatened by the remark in the slightest. It just added to my wife's anger at Facebook for repeatedly giving her "warnings" over trivial or innocent things.

Not online, in person, but one of the teachers at our son's school sometimes "threatens" to "steal him" from us – her remarks don't worry us, because we know she would never actually do that, it is just a colloquial way of expressing affection.

View on HN

> Facebook locked her account for 24 hours for making a "violent threat".

This is due to years of old media digging through Facebook posts to write stories like "Child kidnapped after threat on Facebook and Facebook DID NOTHING". This led to more and more calls for FB to "do something" to "keep people safe online". So they started to run all posts through the AI prescreener and well, these are the results.

But hey, they get to at least say that they take down X billions of posts before any other human sees it.

What if a random old guy she didn’t know commented that exact message?

I don’t think the mother would be as accepting then, so it also matters who posts the comment to who

In that case, if it's a genuine threat, then freezing the account isn't the right action, notifying local law enforcement is.

Beyond this, by hiding the comment/post, the mother is now unaware of the threat and Facebook didn't contact local law enforcement in fact worse than doing nothing at all.

Maybe it should be up to the mother to report this as a threat, rather than being up to a mindless drone.

To Facebook they are “friends” and have many mutual “friends” as well. You’d think they could take that into account in evaluating the nature of the remark, but it does not appear they have done so.

You cant conclude "this is not a threat" from "being friends and having mutual friends". Most of violence by large is between people who know each other.

If you know that A and B are “friends with many mutual friends”, and A posts baby photos, and B comments “She's so cute I'll have to kidnap her", that’s enough information to conclude that the remark is more likely to be experienced as innocent than threatening.

Which loops back to original question: what if it is a man. What if we remove the "women are inherently unthreatening" assumption.

Would depend on their personal relationship.

Which is to say: Often it's not reasonable to act without first knowing if the recipient actually took offence or felt threatened, before even considering anything else, because you can't know. A model aimed at judging utterances needs to also take into account "how close is this group? could this be in-group language?" and it needs sufficient context to judge that. Even then it'll have huge potential for error.

Incidentally, a lot of this wouldn't be nearly as much of an issue if we could trust that these kinds of tools would be used appropriately. E.g. instead of banning things, even just asking a "are you sure this will be taken the right way?" or similar is a whole lot more benign. The dating app Bumble uses a filter that checks for possibly explicit pictures, for example, but instead of blocking them it shows you a short message asking if you're sure the recipient wants to receive what you're sending, and lets you choose. Depending on use there might well be several levels of severity in messaging justified, from just a mild "are you sure?" marker to forcing you to explicitly acknowledge that you're on notice that you're responsible for the content.

My wife is quite convinced - and I think she is probably right - that nobody reported her comment, some automatic system at Facebook picked it up purely on the basis of the keyword “kidnap”.

If a male relative or friend had made the same remark, would my wife’s cousin have felt threatened? My wife tells me she would not have been bothered by such a remark, in that context, from one of her male relatives or friends.

And, if someone was really bothered by it, they could report it. But, it doesn’t seem like that is what has happened here, it appears to just be the work of some automated system. I’m not saying that if the mother reports the remark Facebook shouldn’t take the report seriously - cultures differ after all, in some cultures those kind of remarks are normal in that context, in others they would seem bizarre and offensive - but it shouldn’t just assume the remark is threatening because it contains the word “kidnap”

To be clear: I 100% believe there was no intent of threat and no person read it as such in your situation. I agree it was likely automated something. It is way more likely then report, because if you had report eager people on facebook, you would know.

I was more interested in hypothetical "can we conclude it is ok from friend status on Facebook without actually knowing involve people" question.

Facebook removes the offending posts in this case, yes? If someone on my friend's list says something offensive on one of my posts, I think I'd rather see it and address it myself rather than have Facebook sweep it away.

So what you're saying is that women never kidnap children?

Violence is one thing, but I think harassment and threats in public forums may not usually be between people who "are friends and have mutual friends".

Reply on news.ycombinator.com