Hacker News Reader

Hey,

This is absolutely something I want to do, but at the guild level. The database itself is over 13TB which is much to large to create regular exports of. I will probably provide a SQLite export of each guild, regenerated each week/month. Anyone is free to download whatever they want in real time from the API.

Thanks for your question!

View on HN

Big +1 for dumps

You might try reaching out to Anna's Archive and see if this would be a dataset they'd be interested in helping host/distribute. I think they'd agree that such data is important and should be archived.

Yes, we'd happily host and torrent this.

I was imagining something like a month-by-month historical export so only recent data would need to be regularly exported.

SQLite dumps would be awesome to work with though.

Reply on news.ycombinator.com