Prometheus: A Next-Generation Monitoring System [video]

We started using Prometheus for a Go-based app in production a few weeks ago. So far, so good.

They recommend starting with PromDash [1], their Rails-based visualization tool, but I didn't find the console-based templates [2] very difficult to dive into (Although more examples would be handy).

Worth checking out, IMHO.

[1] http://prometheus.io/docs/visualization/promdash/

[2] http://prometheus.io/docs/visualization/consoles/

I've been slowly deploying this for server and network monitoring. The combination of Go's fat binaries and the exposition format have made it ridiculously easy to get started with (and almost immediately produced insights - PromDash became popular the moment people saw it working).

We are using Prometheus as the central part of our logging and alert infrastructure. We have collectd inside individual docker containers feeding data to Prometheus via collectd_exporter. I hate yak shaving and there's been surprisingly little of it with Prometheus so far.

Pull model kills it for me - I know I can use the 'push gateway' and setup a grep of 'exporters' but then you've just killed the 'easy to deploy' advantage.

I still think something like Riemann/Influxdb/Graphana is the best bet for a roll your own type solution. Riemann is a really flexible router/aggregator/alerter with realtime metric display and it can throw everything over to influx/grafana for dashboarding/time series analysis.

This looks really cool, but I'm always instinctively turned off by the pull model.

- Assuming metrics are stored in memory, they'll be lost if the app or server recycles. Of course, a push model should probably batch so there's probably some kind of overlap here.

- I've heard pull is good for detecting if a server is down. The flip side is no automatic discovery and dynamic scaling where a server going down might not necessarily be a bad thing.

- Security. Not super thrilled with exposing something that says "Hey come get my data".