Hacker News Reader

Comment by mrspandex | Hacker News Reader

Logging seems like the perfect candidate for a relational database. It's a write-heavy system with need for powerful ad-hoc query capabilities. Am I missing something? Why put this in a key-value store?

View on HN

You are right, a relational database could be used. However, SimpleDB is a very simple system that has a much higher availability than any relational solution that we could think of. In general, it is just there. Next to that, it does not have many features, but it supports just the stuff you would like to do for logs. On top of that, we write many more logs than we read, and we don't care so much for realtime consistency in logs. And it is cheap.

What about a highly-structured ACID datastore designed primarily for short transactions reading & mutating small sets of random rows makes you think that? Log data is semi-structured and very heterogenous. I would choose the filesystem far before an RDBMS.

Maybe RDBMS isn't the /most/ efficient way to store the data, but the advantage of being able to write simple and powerful queries against that data is huge. I seem to lose that with file or key/value based storage. Am I wrong? (not trying to criticize, I am genuinely curious)

Also, I don't need to worry about files across servers and rolling log files that file logs usually entail.

I'm not suggesting that a K/V store is better, just that an RDBMS isn't a good solution to this problem either. How do you effectively utilize the query capabilities of an RDBMS with semi-structured data? You're better off sticking the data in a flat file and using text processing tools to work with it.

Reply on news.ycombinator.com