Show HN: Orac – Filter news from social media using AI

I think most people do something similar to you, and block themselves from social media. I know I do. People resort to deleting their accounts, temporarily blocking access using browser extensions, or removing the apps completely. The problem I've found is that there is interesting work-related content that I do want to know about that gets shared through social media (and even more messaging). And I get work messages through facebook and twitter, and then half my day is blown. I think that's worse for people on a maker's schedule. You need the important content (say new deep learning research) but you blink and you've shot half a day after glancing at facebook. The direction I'm heading with Orac is to use the topic modelling and a doc2vec approach to match up clusters of concepts between content, and your todo list. That's still a way off but this is intended to be a step in that direction.

The deep learning models are still in progress, but they're already getting pretty good at ranking content and identifying topics etc. One friend described them as "scary" and I don't think they are quite there but they are improving.

Adding "build your own" filters is definitely on the road map - so something like the fantasy football tracking is a great use of that!

Another Terry Nation fan I'm guessing.

There's a feature being worked on that is some way down the track, but will make sense of the name. If you'll forgive the teaser, but it's related to having conversational control.

I'm glad I'm not the only one! ;)

For this preview release, you can't control the sources. But when it gets released for real, you will be able to create an account an add your own social accounts (twitter, facebook, linkedin, reddit) to monitor, and also add your own news sources (RSS feeds, websites to spider etc).

Based on the sources you add, Orac will then try to find other interesting similar sources for content to include in the feed.

On the back-end, we're basically building a database of media content with quality and attribute predictions designed for searching and filtering based on topic model matching.

There are some interesting ideas we're working on with de-duplication (establishing canonical sources, effectively). But that's super early.

A mood or a persona is really an algorithm representing a batch of those. A lot of them you can get pretty close to just by using the Power Filter directly. But there is some secret sauce.

But broadly speaking, you can think of a Hacker persona as being someone interested in Science and Tech and Education as key broad topic areas, wanting more serious content, and wanting content from credible science-based sources. They prioritise useful content, with more in-depth coverage.

An Activist might be someone with a center-left to left stance, interested in social fields, politics and psychology, but also influenced by trends and what is currently generating a lot of "heat".

If you have representations of what good quality content "within that community of personas" looks like, then you can train models and test scoring systems that try to capture them.

Content quality is a big topic. But our working thesis is that there are canonical examples with domains of expertise of what great content looks like - PG's essays, Pulitzer Prize Winning Journalism, award winning research papers. And there are similar examples of what very bad content looks like - hate speech, extremism etc.

So that's been the broad basis for the models that underly the predictions.

[edits for typos]

There are biases in all algorithms because they represent an encoding of either judgement or bias in training data selection.

So our aim here is to control for bias, and take some steps towards giving the user more control, compared to an approach like Facebook's where the algorithm is entirely a black box.

The approach we've taken is, effectively, to try to codify editorial judgement and professional journalistic best practices into the system, and the selection of training data. As well as being a programmer / computer scientist, I'm also a professionally trained journalist and was editor of Australia's leading computer magazine.

The knobs aren't direct representations of model predictions themselves, but weighted summary scores based on algorithms using a number of lower-level predictions (using statistical models, ML/DL models, ensembles). The personas/moods/filter bubbles use more of the individual attribute predictions.

Having said that, in practice it's all a bit of an experimental mish-mash currently, and we have a lot of work still to do. Some predictions are way more effective than others, as you can see browsing through it. And others (like source and author quality and attribute prediction) are learning and improving over time. But we have a lot of iteration and experimentation ahead of us!

In practice, the initial feedback has been that the predictions are surprisingly good. But sometimes they are way off the mark.

The background is that social media is making us dumber and less productive.

So I want to make something that lets you focus on your work by finding the best content matching what you're working on or interested in, and presenting it in a distraction-free way.