Comment by bigdict | Hacker News Reader

Top New Best Ask Show Jobs

Comment by bigdict | Hacker News Reader

Parent Full thread

Instead of sending thousands of requests through that API you can download the full official dump on Google BigQuery Datasets.

Author of the post here (Nir, CEO of Arnica). Thanks for the feedback! That's a great tip, I wish I knew before. I'm not a developer (our developers are busy building the actual product) so I learn new things every day. I'll give the BigQuery dataset a try. EDIT: I took a look and it seems like that the last modification was on 2016.

> I'm not a developer ... so I learn new things every day

Hopefully, that's true for developers, too!

Refer to this comment: https://news.ycombinator.com/item?id=29017712.

No there's a different one, you need "full" under hacker_news. It's updated at least daily.

I don't believe that BigQuery dataset is actively updated?

For posterity, here's a link to the dataset that is up-to-date. https://console.cloud.google.com/bigquery?p=bigquery-public-...

For details on how to get it out, read this comment tree: https://news.ycombinator.com/item?id=25075318

I'll look it up when I'm at the keyboard again and I'll message you.

It is, it's just hard to find and it's not the first thing you get when you search.

Reply on news.ycombinator.com