Hacker News Reader

Comment by jv22222 | Hacker News Reader

Co-founder here happy to answer any questions!

You guys have built quite a nice and useful product. I was thinking of a similar project which required crawling all of reddit. But presently there are some hiccups due to the large amount of data involved.

I wanted to ask how you guys manage the reddit data? Do you use praw/pushshift to get all the comments/post titles or is it limited to selected subreddits? Including all comments and all subreddits is quite a lot of data I guess. Any suggestions to collect that data and then work with it effectively? Thanks.

We use praw, it works great if you fill out Reddit's forms to register your user-agent as per their terms of service. We get data about all posts, but don't get the comments themselves, just the post metadata (titles, likes, number of comments, etc). Good luck!

Is there somewhere I can suggest sources? I really love your product, but I couldn't see where you got your data from.

Also a small but annoying thing, when I delete topics the confirmation box doesn't respond to the enter key, only the mouse.

Anyway, those are relatively minor issues and I definitely look forward to my daily brief tomorrow.

Thanks so much for your feedback. Oh yeah I hate it when you can't do stuff by keyboard. I'll have a look at fixing.

Re sources. Right now it is select Twitter accounts, all of Reddit, Hacker News, Indie Hackers.

Some we are interested in is github and dev.to - just need to wrap our heads around the details.

What ideas do you have for sources?

Reply on news.ycombinator.com