Hacker News Reader

Comment by WesternWind | Hacker News Reader

I think the issue is that they could have taken steps to lower their rate, but didn't. There are steps even privacy-conscious companies can take.

Like it doesn't even have to be E2E messaging, they could check the birthday of people, train an LLM to differentiate appropriate vs inappropriate comments on children's Instagram posts, and use that to help surface problematic comments. They could bring up a popup asking any child to report the conversation which would be triggered by client-side javascript based on a hashed word list of words commonly used by groomers. They could warn the kids whenever they upload images to share directly in DMs rather than publicly on their insta or fb.

In a perfect world, the rate of grooming would be zero, but at Meta's scale, just creating even a little more friction that makes it harder to groom kids should be seen as worthwhile.

View on HN

> I think the issue is that they could have taken steps to lower their rate, but didn't.

This is where it gets tough really fast. For arguemnt's sake, let's say semantic analysis can properly identify patterns well at 95% level (oooooffff already can see problems).

Public/wall comments on any platform are relatively easy to police. But, I argue the vast majority of problematic grooming comes from DMs. Policing DMs seems worse than trying to do hash analysis of private photos.

I don't think there's a win scenario here for anyone involved.

[edit]

> they could check the birthday of people,

yeah because this is answered honestly by predators.

> Policing DMs seems worse than trying to do hash analysis of private photos.

When it comes to DMs between adults and unrelated children, does it really seem worse? Because really what these platforms are enabling is a situation that's never really been common or tolerated before in society, unlimited unsupervised contact and rich (media-wise) communication between unrelated adults and children.

These platforms categorically do know when a user is an abuser, potential abuser or has in general been making inappropriate contacts with children and they could easily do something about it.

> When it comes to DMs between adults and unrelated children, does it really seem worse?

I should clarify when I say "worse" i mean "significantly more invasive". Checking if a photo's hash matches another photo's hash, at the very dumbest levels, straight forward, hash == hash or hash is inside larger hash.

But for DMs matching words isn't good enough because there are so different ways to say the same things. The system needs to understand when "I really like that top you're wearing" is a cute message between friends vs a predator because who as well as the lines right after really change the conversation. If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!?

The machine needs a lot of context to get this right versus photo hashing analysis.

THat's why to mean analysis of chats is way "worse" than analysis of photos.

> If you don't know the "who" then now we're asking an LLM does this conversation sound predatory!?

But you always will know the 'who' - so, you (the platform) knows that it's a conversation between an adult and a child unrelated to them and yes, at this stage, simply asking a LLM 'hey is this conversation sus' will give you results that are more than enough to trigger a human level analysis of the interaction.

Also you have to consider that adults and unrelated children generally do not interact at all. So, if you wanted to legitimately try and screen off abusers, 'adults that initiated more than 5 conversations with children unrelated to them' then feeding all those interactions into a LLM classifier would get you 99% of the way there.

Wait how do you know who? Whatsapp you got phone number, Facebook you've got.... an email address[0] and possibly a phone number....

[0] https://www.facebook.com/help/188157731232424

> Facebook you've got.... an email address[0] and possibly a phone number....

This doesn't seem like a conversation that's being had in good faith.

The purpose of a system is what it does[1].

In this case, one of the purposes - there are others - of Meta is the sexual harassment of children.

Meta could self-dissolve, or cease operations, but rather than that, they continue their purpose. Implicitly, the sexual of harassment is worth it.

1. Beer, S. (2002). What is cybernetics? Kybernetes, 31(2), 209–219. doi:10.1108/03684920210417283 https://sci-hub.ru/10.1108/03684920210417283

I’m not able to read the cited reference at that link, so I’ll try to take the statement at face value.

I could name or describe a number of systems right now that have outcomes that we could all agree are contrary or misaligned with their purpose, so unless the author is equivocating things like “purpose”, or “system”, I don’t think your premise is well founded, nor do I think your conclusion has merit.

The conclusion is that systems should not be judged deontologically, and instead should be judged consequentially.

The point is that if you could describe systems that are contrary to their purpose and have negative outcomes, one is justified in referring to them as systems whose purpose is that negative outcome. Doing otherwise is arguing from conclusions.

> The purpose of a system is what it does[1].

> In this case, one of the purposes - there are others - of Meta is the sexual harassment of children.

I disagree as well. It's too naive to say "the purpose of meta is sexual harassment of children".

Purpose and result are not hard linkable in so many cases.

An obtuse example is a kitchen knife. So easy to say a knife's purpose is murder thus knives should "cease-operations".

But reread what I wrote. One of the purposes. Please, if you're going to argue, argue with what I wrote.

It's too naive to say "*one of* the purpose of meta is sexual harassment of children".

So easy to say *one of* a knife's purpose is murder thus knives should "cease-operations".

----

I did what you asked, but fail to see how that changes anything. Please if you're going to ask me to change something, have that change be meaningful.

It's naive not to. Believing that Facebook's purpose is what it intends to do is like believing that North Korea is a democratic people's republic.

I can rephrase it in terms of intentionality if you wish: Meta has a choice every day to exist or not and the not. When choice to exist results in the sexual harassment of children then transitively Meta is choosing that consequence.

To address your knife analogy, a knife alone is not a system. A knife plus a human plus another human being stabbed by the knife is. In that scenario, the person creating the system, the knife-wielder, should(like Meta), choose to dissolve the system. I think, at least for the knife+human+human stabbing system, that's quite easy to agree with.

> They could check the birthday of people, train an LLM to differentiate appropriate vs inappropriate comments on children's Instagram posts, and use that to help surface problematic comments. They could bring up a popup asking any child to report the conversation which would be triggered by client-side javascript based on a hashed word list of words commonly used by groomers. They could warn the kids whenever they upload images to share directly in DMs rather than publicly on their insta or fb.

All this reduces engagement.

That is the bottom line. Facebook can do more, but they won't unless something makes them, because it potentially costs billions.

In fact, they seem to lobby for this kind of thing... But only if someone makes all their competitors do that thing as well.

> train an LLM to differentiate appropriate vs inappropriate comments > popup asking ... which would be triggered by client-side javascript based on a hashed word

Yeah, this is the feature that I think I should build my own social media for. Owning "the algorithm" and using it to indoctrinate users en masse with my own view. Are you sure you want to insist it's in any way or form even remotely ethical thing to do?

for me the issue is: why? followed by a few more "why?" and once the answer becomes "because." it becomes time to solve those intermediary questions. maybe that happens and nobody tells me, idk

Reply on news.ycombinator.com