CSAM content on Bluesky has risen ten times in just a week

Well, either way it’s useful to be real about the kinds of challenges that come with social media. It’s not hypothetical. And we’re still at a very small scale compared to most networks (9mm users)

Like every other app we’ll constantly battle with it. It’s the job. And then disagreements about how we handle it will create controversies, and then that’s where the protocol comes in, where other apps show up and provide viewpoint diversity and a check on the decisions we make

As just a normal user, I can't speak to their motivations, but the sandcastle kickers are much more present now.

There is a surge of new users I've seen who are there just to spread racial slurs, and get on the opposite side of whatever argument is being raised in that particular thread.

There are so many of these suddenly that it feels coordinated to me. But I suppose that all it takes is one /pol/ post.

I was an early Lemmy instance host, and particular communities in bigger servers were getting hit with CSAM bots that were using it as an attack vector. It caused the meme community on Lemmy.world to completely shut down. Not only that, but Lemmy didn't properly remove images from storage when the attacks happened. They weren't accessible on the frontend anymore at least, but it required additional work on the command line to locate and remove the content from storage.

Wouldn't surprise me in the least if something similar wasn't also happening here, partially masked by the increased attention.

Bsky has a revenue model?

I'm sympathetic, of course. I was part of a startup that iterated for a couple of years on a social thing looking for revenue, so I know the semi-delusional level of hope that any startup requires before you find product-market fit.

I agree that's one thing investment is used for. But another thing it's used for is to prop up things that never find a revenue model until the lack of money forces people to swallow some hard pills.

Bluesky's been going quite a while for not having a proven revenue model. And it's not like it's the only social media company ever; this is a space that has been well explored over decades. So the lack of it in any social media startup these days seems pretty glaring to me.

I get it though. If you have money, then the easier solution is the one you can just throw money at to get instant results; mental health of your employees be damned.

When your platform gets large enough you have to choose harder. Facebook tried to choose smarter in Myanmar, and look at how well it worked out. There’s simply no algorithmic solution to the problem where your users genuinely want to see the content you’re trying to moderate.

Even if you build flawless tools that ensure the algorithm won’t promote anyone talking about killing, your users will invent a new word “unaliving” and use that instead.

I think you underestimate the extent to which companies try to do both.

I know Twitter and Facebook both spent big for years and years on increasingly sophisticated ways to automatically recognize problematic content. Even they couldn't afford to just throw labor at the problem; a few years back when I ran the numbers Twitter's revenue was about $1/user/month, or about ¢1 per tweet. No matter where you outsource too, that doesn't buy much labor.

If you look at the manuals for the outsourced moderators [1], it's clear they were being asked to work at a pretty sophisticated level. That level certainly wasn't available from ML systems before a couple of years ago, and I'm skeptical that it's there now, given that many of the hard moderation decisions come down to contextual judgements, and given how good awful people are at innovating their way around automated detection systems.

[1] which you can do here among other places: https://www.theguardian.com/news/series/facebook-files

Two decades ago

Perhaps they mean it's not a useful indicator of anything. If abuse material on Bluesky has increased, but overall on the web it has reduced (because for example Bluesky were better than other channels at policing content) then that, for example, would actually be good.

A media channel that increases in popularity will have increases of all types of content. Whether it increases beyond levels on other media is probably the metric to watch?

It's a problem everywhere, and here it's simply scaling up along with the rest of their traffic in general which spiked this week due to another Twitter subcommunity exodus event.

Hello from 4 days in the future: https://bsky.social/about/blog/09-11-2024-video (Bluesky supports video now)

> Hopefully the bad actors can be blocked out of existence.

This gave me 2011 Twitter vibes.

>BlueSky isn’t going anywhere, especially with the clearly eventual demise of Twitter.

According to the media and on Reddit, since Musk bought Twitter, Mastadon, Bluesky and Threads have collectively gained 70 bazillion users on seventeen different occasions, and have caused Twitter to collapse and disappear eleven times.