Hacker News Reader

I didn't use pushshift, sorry. The data was collected from bigquery, stored locally into CSV files, and then I just wrote a node.js script to compute similarities.

View on HN

Did you simply collect "user has posted to X, Y, and Z subreddits", or did you look at frequency too?

The reason I asked the question is because back in 2016 I had a similar (now out of date) approach to finding related subreddits at scale using Jaccard similarity: https://minimaxir.com/2016/06/reddit-related-subreddits/

There, I only built a user edge if a given user commented on 5 distinct threads in a subreddit, since a lot of subreddit interaction was due to brigading.

I didn't look into frequency. Is there a version of jaccard similarity that accounts for frequencies?

There's a weighted variant: https://en.wikipedia.org/wiki/Jaccard_index#Weighted_Jaccard...

Reply on news.ycombinator.com