Hacker News Reader

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

there is nearly 10TB of youtube metadata available on archive.org https://archive.org/details/youtube-metadata

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

redd-archiver uses postgres full text search. for static search you could use lunr.js

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

the torrent has data for the top 40,000 subs on reddit. thanks to watchful1 splitting the data by subreddit, you can download only the subreddit you want from the torrent

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

the data from 2025-12 has been released already, it is usually released every month, it just needs to be split and reprocessed for 2025 by watchful1. i will probably eventually add support for importing data from the monthly arctic shift dumps so that archives can be updated monthly.

https://github.com/ArthurHeitmann/arctic_shift/releases

Arctic Shift https://academictorrents.com/browse.php?search=RaiderBDev

Watchful1 https://academictorrents.com/browse.php?search=Watchful1

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

I included a metadata dump of every subreddit found in the torrent. it includes a status field which will show of a subreddit is private along with a much more details

data catalog readme: https://github.com/19-84/redd-archiver/blob/main/tools/READM...

reddit data: https://github.com/19-84/redd-archiver/blob/main/tools/subre...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

ive created tooling for an instance registry and team based leaderboard. the API has function to support this as well, so that we can collectively host archives in a decentralized and distributed manner

registry readme: https://github.com/19-84/redd-archiver/blob/main/docs/REGIST...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

I've updated the workflow and added a placeholder page that will serve before archives are created. thanks again! https://github.com/19-84/redd-archiver/commit/0dfd505ca81cb2...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

thank you for your comment, some example dot files were not copied in my original repo, they have now been added.

https://github.com/19-84/redd-archiver/commit/0bb103952195ae...

the docs have been updated with mkdir steps

https://github.com/19-84/redd-archiver/commit/c3754ea3a0238f...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

thank you for your comment, I will support any platform that has complete dataset available. I will take submissions for any complete datasets through github issues. https://github.com/19-84/redd-archiver/blob/main/.github/ISS...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

the API and MCP server is very powerful ;)

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

I have also published sub statistics and profiling for each platform. these can be used to help identify which subs to prioritize for archiving.

reddit: https://github.com/19-84/redd-archiver/blob/main/tools/subre...

voat: https://github.com/19-84/redd-archiver/blob/main/tools/subve...

ruqqus: https://github.com/19-84/redd-archiver/blob/main/tools/guild...

19-84on Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever

the API should allow for a lot of different integrations