Hacker News Reader

Very nice & all, but... MongoDB?

MongoDB is very popular, but all the (limited) criticisms of it seem to related to insert performance once the dataset it too big to fit in RAM.

Normally the easy-of-development arguments make up for that, but log files is one of those areas that has a tendency to expand quickly beyond any expectations.

There is a reason why most companies are using HDFS and/or Cassandra for structured log file storage.

View on HN

fluentd's greatest advantage is, it's written in Ruby. So, it's really easy to write the plugin for any datastores.

This is HDFS (Hoop, the HDFS REST gateway) plugin.

* https://github.com/tagomoris/fluent-plugin-hoop

And also, Cassandra plugin are now in under development.

* https://github.com/tomitakazutaka/fluent-plugin-cassandra

You can see the user contributed plugin list here.

* https://github.com/tomitakazutaka/fluent-plugin-cassandra

If you were going from logs -> hdfs, flume would be a much better chose imho

Reply on news.ycombinator.com