cipher129

Interesting! That’s something we will definitely adjust for in upcoming models, though practically speaking, the confidence is low enough that in mod systems they would be treated the same

We define hate here: https://moderatehatespeech.com/framework/

As:

“any unreasonably rude or hateful content, constituting as, "threats, extreme obscenity, insults, and identity-based hate"”

Yeah -- the latter is obviously much easier to identify because it's much more explicit/clear cut.

The first are of course, transphobic given the clear intent, but it there is definitely some ambiguity w/o context. Ie, if something just posted, "He will never be a woman" there's definitely many contexts where something like that is not malicious. Ie, if someone else is correcting an accidental mis-gender, they might say, "No, she is actually a he" which isn't malicious.

Since the state of our API (and literally every other similar system out there, lol) can't capture the context of the conversation -- what "He" is referring to, the overall topic of the conversations, etc -- we basically have to make the best calls given the available context...which unfortunately does create false negatives in certain cases (and false positives in others).

But, context is something we're working on, because its definitely a big issue, and as you can see, plays a very prominent role in informing better moderation decisions!

> What is the max qps and latency this supports?

We generally aim for <400ms in the continental US. The backend is hosted in us-central-1 on GCP. Obviously, we're always trying to expand, though we do rely on grant funding as a non profit so that comes into play too.

> Some moderation platforms I've worked with are too slow for messaging, especially if users are in different countries.

Typically it works pretty well on the message platforms we're on -- we've got a Chrome extension for Twitter + discord integrations we're testing right now.

> Do you have plans for image moderation?

We do! Though that's further down the line.

User retention is much better on apps -- if someone hasn't engaged with your app in a while, the app can send a notification/has more avenues for re-engaging the user compared to a website.

I think it's highly dependent on the culture of the workplace/environment and if its more on the laid-back side of things. In general though, I'd say too many emojis is never a good thing.