Hacker News Reader

It crashes every couple of hours and then takes a few seconds to minutes to come back up.

I've just been playing around with the source code to see if I could figure out why it would lose the port, but the only thing I can think of is if the whole mzscheme process dies, otherwise it would just hang while trying to connect and it doesn't do that.

View on HN

It's running out of memory. The half million or so of HN comments and stories no longer all fit in memory at once. This is not in itself a problem. We only load stuff as needed, and only crawlers ever look at the older stuff. But it does mean that eventually (currently after a day or so) we run out of space and the server crashes and restarts. I'm working on a less crude version of garbage collection.

I guess you don't need to worry about all that complicated fragmentation with the current GC. :)

It's a great scalability test then! Memory is cheap, so that should not be a really big issue then, 32G of ram in a server is nothing special these days. Unless of course you're already maxed out, then it is a different issue.

How big is the working set of HN ?

We're maxed out on the current server.

Reply on news.ycombinator.com