Effortless monitoring with collectd, Graphite and Docker | Hacker News Reader

Effortless monitoring with collectd, Graphite and Docker

blog.docker.io

72 pointskalessin23 comments

UDP is really great, but it also makes it extremely easy to spoof the source IP and flood your monitoring server with fake metrics. It's probably not a huge concern, but something to be aware of. See a blog post I wrote in the context of statsd/carbon[1]

<shameless plug> Also, if you're looking for a nice open-source front-end to graphite, take a look at Giraffe[2]. It also has a collectd plugin[3] </shameless plug>

[1]http://blog.gingerlime.com/2012/statsd-and-carbon-security/ [2]http://giraffe.kenhub.com/ [3]https://github.com/bflad/giraffe-collectd

Yeah, that's why collectd supports authentication and encryption (https://www.collectd.org/documentation/manpages/collectd.con...). I can't really speak about Graphtie since I'm less used to it; but in this case you can decide to not expose the Graphite port and only expose the collectd port.

Ah. I wasn't aware of this. Graphite/Carbon/Statsd are much more "basic" in that respect.

Good point. However, collectd's protocol is much more complex that carbon's; it can sign or encrypt the data to prevent spoofing. If spoofing is a concern, then instead of having collectd on your systems being monitored send directly to carbon, have them send to a collectd instance on your graphite server, and have that instance send to carbon on localhost.

https://collectd.org/wiki/index.php/Networking_introduction

The biggest problem is setting up and upgrading Graphite, which is a pain and semi-broken right now.

I've been successful using collectd + Librato, works great, specially since you can setup alerts effortlessly.

Yes, setting up graphite can be confusing and painful. Looks like the author created a Docker app that should also make it easier.

Another alternative and a shamless plug - I've created an open-source fabric script that automates the graphite install processes (on debian-based hosts). See https://github.com/gingerlime/graphite-fabric

Yep, I think it has too many dependencies and the architecture is over complicated. I've built a standalone stats collection/storage daemon that just works with any charts packaging - google charts, or whatever. Works great and doesn't require 10000 dependencies.

https://github.com/lsh123/stats-rrdb

I made a gist to show how to install it on the newest Ubuntu release: https://gist.github.com/kaspergrubbe/5792356

There is too many moving parts for my taste, but the web-interface is the worst.

I know, I've used your script in the past actually :)

That's why they use docker: it automates the process of building and packaging Graphite, so you don't have to worry about it.

    $ docker run lopter/collectd-graphite

You're done.

Yeah, I get that from the post. It's not a definitive solution though, docker is of limited use and doesn't excuse Graphite setup being broken in the first place.

I don't understand this idea that you should build a metrics system from components rather than use one of the end-to-end projects out there like Zabbix. Can someone clue me in?

I don't, either. And Graphite doesn't allow you to tag metrics for easy aggregation, which is a huge win if you're monitoring more than a few instances of a metric. (Nor does Zabbix, BTW.)

I'm a reasonably happy Datadog customer.

Have you used any other cloud monitoring/metric/etc providers? If so, how do they compare to DataDog?

To my knowledge, none of them supports multi-dimensional metrics. So I had to cross them off my list.

Datadog is very cool. What are some of the other solutions out there that are similar? I like how it mostly just integrates with anything I need, that makes things easy for a non-enterprise-level project.

Barrier to entry. Zabbix is "enterprise" out its ass.

Graphite, statsd, collectd are small simple systems that you can have mostly figured out in a few minutes. Meanwhile Zabbix's website makes it seem like the sole reason for that project to exist is to sell consulting and training gigs.

Datadog seems like it might be slightly better, but I haven't spent much time looking at it.

That's not true. Maybe that's the impression it gives off, but it's a very good package that blows Nagios out of the water. It's UI/UX needs work and some things are a bit quirky (to say the least), but it scales well, works quickly out of the box (basic level), has decent and improving API, integrated graphs, etc.

It's not perfect and suffers from the same problems as other monitoring systems, but given the options it's better then most systems with similar scope.

P.S. I am not affiliated with them in any way shape or form. Just used it for a little while.

Compared with graphite/etc, it absolutely is true. A project is judged against its competition. The things you are talking about (UI, scaling, graphs, etc) are not my concerns with it. I don't doubt that it is "very good", for at least several values of "good", but that is missing the point.

Right off the bat, here are differences between barrier to entry (ignoring the "enterprise" stench and consulting/contracting emphasis for the moment):

  Installation:
    Graphite:
      Homepage -> links to  pip / github / launchpad
    Zabbix:    
      Homepage -> Downloads -> Download -> RHEL binaries?!?!  -> Oh nevermind, sourceforge link.....

  Sending metrics to the system:
    Graphite:
      Homepage -> documentation -> getting your data into graphite
    Zabbix:
      I'll be honest, I just gave up.

It just gets worse from there. I don't want to become my companies "Zabbix guy". I want something that I can just get working as rapidly as possible with as little heartache as possible. Particularly heartache that seems suspiciously intentional considering the number of ways you can find yourself on a "hire us to train you!" page on their site...

Zabbix reminds me of all the things I hate about working for a PHB who makes technical decisions at large corporations. Maybe that is an unfair take-away, but that take-away is entirely their own fault.

Honestly, it seems to me you gave up pretty easily there.

I spent about 5 minutes looking, I have better things to do on my downtime, but it seems you have forgotten what my original claim was (or what your original question was).

High barrier to entry. Maybe you think the barrier to entry for Zabbix is low enough, and maybe you think it is a better project once you get over that bar, but what is undeniable is that Graphite very clearly has a lower barrier to entry.

If you want to know why so many people use Graphite instead of Zabbix, that is the reason. If you want to tell all of those people that they are wrong for going that route, be my guest, but that isn't what the discussion is about.

I've answered your question, I feel, entirely accurately.

To extend on this idea a little bit, once you have the data in graphite you can use it as input to an alerting system. As an example, here's a Nagios plugin I wrote to do that- https://github.com/sciurus/grallect

I'd love to see a "fixed" version of statsd/graphite/etc that is painless to install, understand and use.

Reply on news.ycombinator.com