Ask HN: What database does HN use? | Hacker News Reader

Ask HN: What database does HN use?

10 pointsCieplak4 comments

I've looked at the arc source for HN at

https://github.com/nex3/arc/blob/master/news.arc

and don't see any references to a persistence layer.

HN stores everything as files in certain directories. A directory is sort of like a table, and each file is sort of like a row in that table. I think there is a directory for profiles, stories, and votes. I forget how cookie-user mappings are stored, it might be a single file or another directory.

The data is stored in files, but there is a lot of caching done to make everything fast.

It's NoDB. "What database did you use? We didn't use one. We just stored everything in files. The Unix file system is pretty good at not losing your data..." http://paulgraham.com/vwfaq.html That was about Viaweb, but I think the same philosophy continues in HN.

I found this in news.arc:

    ;For some reason vote files occasionally get written out in a 
    ;broken way.  The nature of the errors (random missing or extra 
    ;chars) suggests the bug is lower-level than anything in Arc. 
    ;Which unfortunately means all lists written to disk are probably
    ;vulnerable to it, since that's all save-table does.

I wonder if/how they have fixed this.

We did fix it but I forget how. I know we switched to a log format for vote files. But I'm not sure if that was the fix for this problem or something we did later to improve efficiency.

Reply on news.ycombinator.com