Hacker News Reader

Comment by consumer451 | Hacker News Reader

https://github.com/theosanderson/firehose

I just love that the open nature of Bsky is allowing people to hack things like this.

Now that their growth is crazy, let's hope that the work they did on the protocol and corp structure keeps it this way.

A long time ago in a galaxy far, far away ... there was a twitter firehose and people loved how the open nature of twitter is allowing people to hack things ...

And then they decided to not-so-openly collaborate with government regimes who sought multiple times to implement a ministry of truth and censor everyone they didn't like. So now we have to settle for poor decision making.

Yeah I was looking into the firehose as a potential way to source to discover new domains for my search engine. Even though it didn't pan out, I really appreciate how accessible the data is.

Why didn't it work out?

I decided agains it because it had an incredibly bad signal to noise ratio. Almost all links I saw were either to big websites like newspapers, patreon, onlyfans; or behind url shorteners.

Dunno, I may explore it further down the line, but for now the juice didn't seem worth the squeeze.

I created a website like 10 years ago called birdmine that indexed every link you or one of your followers shared on Twitter, in a Solr search engine so you could search stuff that had been curated to an extent. It was pretty cool, I think I’m the only person that ever used it though.

BSky could use a better search, or a better curation of feeds. I'm certain there is more room in the discovery play.

I might have asked this before, but did you look at adding this type of ingestion for ActivityPub?

Briefly, but I've come to learn there's a contingent of aggressively search-engine hostile people that has made a home on the fediverse. The federated nature of it makes it somewhat tricky to untangle the search engine friendly people from the hostile.

I don't need the inevitable DDOS:es and death threats you get when upsetting a clique of mentally ill people online.

You're right, I forgot about the backlash that this type of efforts got from people that don't understand the technology and just make assumptions about element visibility in the network, though maybe calling them mentally ill is a bit much.

The sorts threatening physical violence almost certainly are. But it's a minority to be sure.

Reply on news.ycombinator.com