anvaka

Yes! I picked only the highest scores to form an edge in relationships graph. Typically a sigma (std deviation) or two away from the mean. So if there is a direct link between your project and others - the similarities are abnormally high.

Note that I'm not rendering direct links outside of the country yet, there might be more there. Will probably add a "focused" view to see those better

Thank you! Google BigQuery has it readily available, so probably took 10 minutes to fetch it =)

https://cloud.google.com/blog/topics/public-datasets/github-...

Check this out https://github.com/anvaka/map-of-github#how-was-it-made - I gave a short overview there - does this help?

Yes! Thank you

Note that some country are missing their names because I didn't have enough knowledge to assign a name to them.

Here is my naming process if anyone wants to help with the rest: https://github.com/anvaka/map-of-github#country-names

:D btw, if you find any country label to be missing or could be improved - right click it, change it, and then send a PR (it's two clicks away)

I knew GitHub is not a tiny website, but I didn't imagine how big it actually is. Each of those dots are giant parts of someone's life.

There are a lot of interests that I didn't know exist. For example https://github.com/cat-milk/Anime-Girls-Holding-Programming-... - someone collects anime girls holding programming books.

https://github.com/tylertreat/Comcast - and here is someone who is amazing at coming up with funny project names =)

I've got data from 2020 to end of March 2023

Thank you. I agree :)

Going to play with myself :D https://anvaka.github.io/go-mo-ku/?w=15&h=15&s=XO&l=5&m=33_3...

Each dot is a subreddit. "countries" are clusters of usually related subreddits. Relationship is determined based on Jaccard Similarity (users who post to A also post to B)

This was my hobby over past few years. I collected ~176 million unique pairs `(user, subreddit)` of comments, computed jaccard similarity, clustered a graph, performed a layout, created a webgl renderer and streaming svg parser, and today releasing it to the wild. Hope you enjoy it!

* The source code for the rendering is available here: https://github.com/anvaka/map-of-reddit

* To change the map - go here https://github.com/anvaka/map-of-reddit-data

I heard this might be caused by some adblocking extensions - they consider reddit to be an ad/tracking system, so they block all javascript requests to it. Do you happen to have one of those extensions?

It seems though relationship can be considered to have a sign: positive, when people mostly align in their upvote intents, and negative when people do the brigading, etc.

I'd still see ability to determine absolute value of relationship as a valuable property of a recommender

Right, I don't think it's 100% accurate either. It gives just some hints what might be related, but like you said, it's not necessary the best possible measure of similarity.

Jaccard similarity does not count only the number of people who posted to A and B, it checks how many people posted to A, how many people posted to B, and how many of those people posted TOGETHER to A and B. That togetherness gives us hints what is related, and after it is computed, we can divide by the total number of poster to both A and B (independently), which brings the value to something that we can use to compare against other subreddits. If that value is close to 1, it means that almost all users who posted to A have also posted to B. If it is close to 0, then the overlap is much smaller.

Yay! Thank you!

Thank you for your suggestion!

This implementation is tailored to smaller graphs with sometimes long text boxes.

I'll make a note to extract it to a reusable component.

Thank you!

I'm so glad to know this! Thank you!

I didn't know about either. Utterly impressed

Thank you! I'm very happy to hear this!

Yay! Thank you for sharing the links and thank you for your kind words!

I see you are doing something with quantum tensors, which is very impressive! I'm still struggling with concept of regular tensors, and would love to have a good intuition/visualization for them - do you have any pointers?

Thank you for the suggestion!

I think when I created this tool there was no recommendations on reddit.

When it was introduced later on reddit I was contemplating about using reddit's own recommendations, but at that time it was missing a few smaller subreddits, so I just put it of onto the shelf of projects to try.

This is absolute treasure trove. Thank you so much!

Oh thank you! I'm very glad you liked it :)!

Thank you :)! Under the hood it is all very naive counting of users who posted to A also posted to B.

Hi there. Here is a similar question and answer: https://news.ycombinator.com/item?id=22178373

This is super awesome, thank you for sharing!

If you guys are interested in seeing how your recommendation work for the entire reddit, I'd be happy to build you a spaceship similar to this one https://github.com/anvaka/word2vec-graph .

I couldn't find an easy way to download the entire recommendation graph, but it would be awesome if we could make it work. My email is the same as this account at gmail, and twitter is all open: https://twitter.com/anvaka

I used my own metric that is based on jaccard similarity. Which in turn is based on "users who posted to X also posted to Y" metric.

That said, there are a few subreddits that are too popular and similarity results were too saturated (/r/videos, /r/funny, etc.) so I did a manual override by looking into most commonly mentioned other subreddits, and sometimes into `about` blurb of subreddit).

Please don't consider these recommendation as source of truth! It's just a fun way to discover other subreddits :).

I'm also very open to change this metric to something else - please let me know if you have any recommendations!

[1]: https://github.com/anvaka/sayit#the-data - describes the data, indexing scripts are here: https://github.com/anvaka/sayit/tree/master/scripts

[2]: Manual overrides can be found here https://github.com/anvaka/sayit-data#sayit---recommendation-...