Hacker News Reader

Comment by fiatjaf | Hacker News Reader

Seems very good for a weekend -- for an year, even.

How do you do it?

Thank you. I'm using Firebase as a database and for hosting, which makes development really fast.

For getting the data, I'm using a worker written in Python, which queries the Twitter and Reddit APIs and does some regex filtering before writing it to the database. In the future, it could be interesting to experiment with more advanced NLP techniques for filtering/grouping/classifying.

Hey, I'm thinking about doing something similar with baseball JSON apis of all things. Would you mind sharing your firebase/python implementation details with me? Thanks!

Very interesting, thanks for sharing. Im impressed by what you've done with just regex for the filtering. Which makes it possible to curate a dataset of wishes to do some unsupervised clustering and fitering.

Filtering could benefit from knowing what other syntax ( phrases ) would map to wishes, broadening the coverage.

Would love to work on this --let me know if you're open to that.

Either way, this is a useful project

Re: finding other syntax, have you seen https://answerthepublic.com ?

That site lets you type in some key words and then it finds what questions people are searching for on Google based on those keywords (with prefixes like "who", "what", "where", and search suggestions).

Then it makes a nice big graph of the questions.

That's an amazing resource, thanks for sharing. Using autosuggestions might actually be really useful.

Unsupervised learning could be really interesting for this. Let's have a chat about how we could work together. Would you mind dropping me an email either via the website or to julian{at}wishminer.com?

Hey guys, we're working on a platform that literally collects wishes from any touchpoint and it has quite some traction, especially in the gaming industry. Could you send me an email to hn{@}stomt.com? Would love to catch up on this. We also do a lot of research in the fields of NLP and clustering.

I am curious to know how you did this filtering with regex. Can you share an example or any tutorial for reference?

Search for tweets containing "I wish there was". Remove the "I wish there was" part. (Profit?)

Pretty much what undai says. It then comes down to building a dictionary of search queries and a 'blacklist'. The regex itself is relatively simple.

Reply on news.ycombinator.com