Hacker News Reader

Comment by azdle | Hacker News Reader

This looks really interesting.

How are you writing the data to disk? I don't see exactly what you're doing in the article. Are you just writing the data to files directly or is there some sort of DB in there? From what I've seen, the storage is the most difficult part.

View on HN

Data appears to be persisted to disk via Basho's bitcask. You mentioned in another comment that ETS is being used, which it is, but that's not on disk (DETS is the disk-backed version); ETS is used here to communicate between processes via public, named tables.

Thanks for correcting me. I'm still just dipping my toe into erlang and forgot that there were two different versions.

(And thanks to lambdadmitry for the additional insight.)

You are completely right, thank you.

The data that we had (disclaimer: I started the project and left a couple of years ago) tolerated occasional loss of a small part of it, so there were nothing fancy there: "dirty" NIF-based circular buffers (courtesy of ecirca library) are dumped to binary and wrote on disk using basho's bitcask from time to time. Dumb and pretty efficient solution given that constraints.

OTP includes a distributed database. Maybe that's what's used?

Just shared a link with HN community. You may try to ask author on an article's page.

Actually, I just looked at the source and it looks like they're using ETS the built-in database in Erlang.

Reply on news.ycombinator.com