Graylog2: Java, Ruby, MongoDB-powered log management, monitoring, and alerting | Hacker News Reader

Graylog2: Java, Ruby, MongoDB-powered log management, monitoring, and alerting

thechangelog.com

30 pointsmattyb8 comments

It was fun to try out this software. It lacks some features in the interface though... With some more automatic processing and UI improvements, it could definitely be a low-end rival to splunk, but right now it can only do a standard search on custom attributes or whole text (not with a proper full-text index though) or show stuff "around that message". That means basics only.

But with the development happening all the time, it's definitely a project worth keeping in bookmarks. If it can get some of the features available in octopussy without looking just as fugly, I'm in :)

Be sure to check out LogStash as well. Uses a similar approach with MongoDB, plus ElasticSearch for...search: http://code.google.com/p/logstash/

This looks great! Is anybody using it in production though? I can't find any case studies on their site.

yes. there are some startups and grown companies using it.

We are using it and loving it. More important than just shipping syslog to graylog2, we're using it to monitor 30+ servers in a cluster. Each server has a "health agent" that sends all the important metrics that happened in the last minute, packed into a GELF message. This could easily scale to 1000+ servers. On the monitoring side, one server dips into MongoDB to pull out the records and maintain records and graphs of everything that's going on (including sending alerts when a machine fails to check in as frequently as it should.)

Graylog2 is pretty solid, and the people working on it are extremely responsive. If you suggest features that make sense, they'll probably be implemented within a few weeks (if not a few days!)

Any chance you would open source that "health agent"? Or is it a part of Graylog2 already? I haven't checked it out yet, but I am pretty excited about using Graylog2 in our environment to replace some other monitoring and seeing things like CPU idle times and HD space remaining would be very helpful indeed.

Bonus. I just watched a Railscast highlighting the Notification system in Rail3 and I can see where it would compliment Graylog2 very well. :)

Let me just caution those with highly scalable expectations. A large MongoDB instance is going to weight in at around 1-10TB. Here's an updated list of production MongoDB instances for reference: http://www.mongodb.org/display/DOCS/Production+Deployments

At both Splunk and Loggly I've seen customers sending in multi-tens-of-millions events a day from a handful of boxes without even breaking a sweat. It wouldn't take long to blow through several TB of storage in MongoDB if you were storing all that log data with a retention of a few months.

Log volumes vary from use-case to use-case though, and I absolutely love the GrayLog2 guys and the way they listen to their users. It's definitely a great tool for a job that is usually a real pain in the ass!

We're using it. I was honestly convinced by their ridiculous tag line:

"Manage your logs in the dark and have lasers going and make it look like you're from space". Awesome!

Also, when you login it gives you status message like "mounting party hats", and "enraging gorilla". It's nice to have some comedy in what can potentially be such a dull arena.

Reply on news.ycombinator.com