Hacker News Reader

Comment by linsomniac | Hacker News Reader

Experiences with Zabbix? I tried it back around a decade ago and wanted to like it, but didn't find it very reliable. And now the details are escaping me. I ended up sticking with Nagios and Opsview. Around 5 years ago I switched to a templated Icinga2 config and have been pretty happy with that, but it's pretty low level.

View on HN

Surprised to see Prometheus hasn't been mentioned yet, and even Nagios is being mentioned as a better alternative. My company (higher-ed, ~100k combined students/fac/staff) is desperately trying to get away from Nagios. Once you get Nagios to the scale where you have to implement mod_gearman, you've gone too far.

I'd recommend taking a look at Prometheus[1]. It has its own _very_ performant TSDB, there's exporters for just about everything, it's the defacto way that things like Kubernetes expose metrics, and it has first class support in Grafana for visualization.

We POC'd Zabbix, Icinga, ScienceLogic, Instana, Sensu, and Prometheus. Prometheus was our favorite. Take a look at the comparison between it and other popular monitoring products to see if it fits your needs though [2].

[1] https://github.com/prometheus/prometheus [2] https://prometheus.io/docs/introduction/comparison/

The problem I have with Prometheus is, I have most of my nodes in very closed networks I don't have control (Healthcare) and I can't set up proxies so Prometheus can reach them, I can only go outside. So, by now, my best option seems to be InfluxDB, which doesn't look bad to me.

I've been using InfluxDB for ~3 years now for storing metrics (almost exclusively via Telegraf, a few custom ones), and it has been great! It replaced a collectd setup and dramatically decreased load across my fleet.

When I first started using it, it was pretty early and had some issues. In fact, I nearly trashed it. I also didn't like the pull vs. push model from Prometheus. They ended up resolving the InfluxDB issues I was having right as I was about to give up on it, and it's been solid since. I use it with Grafana to generate graphs of system use. I set it up before TICK was a thing.

I was about to like InfluxDB but ever since people say it eats memory and your data, I stopped caring.

https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/FAQ

("How does VictoriaMetrics compare to InfluxDB?")

That hasn't been my experience. I've been running it for ~3 years in our dev, stg, and prod environments. Prod is using 1.5GB of RAM on a 5GB instance. I've never had a data loss issue.

Push gateway?

https://prometheus.io/docs/instrumenting/pushing/

I'd recommend giving a try to VictoriaMetrics. It requires less hardware resources - RAM, CPU, disk - comparing to InfluxDB [1] and it supports PromQL - much nicer query language for typical time series queries comparing to InfluxQL or Flux [2]. It may be used as a drop-in replacement instead of InfluxDB on the ingestion path [3].

[1] https://medium.com/@valyala/insert-benchmarks-with-inch-infl...

[2] https://medium.com/@valyala/promql-tutorial-for-beginners-9a...

[3] https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Sing...

Zabbix is a bit opaque to tune and the support forums aren't super helpful unlike say Nagios.

That said it does some really cool stuff like tree walking across all the HP switches on our network, auto monitoring all ports it finds and then reporting on their stats and on any UP/DOWN states for every port.

Good for detecting unauthorized usage or a device which is rebooting itself.

Its IPMI support is also pretty good, we had it monitoring Supermicro IPMI interfaces with zero issue.

It handles vSphere and auto scans the entire cluster, adding all guests and monitoring them without needing to install an agent on every VM.

All in all a very good solution with some very cool features, but a steep learning curve and not much help on their forums although the docs are pretty good.

I’ve run Zabbix with thousands of monitored hosts. It’s not perfect, and it requires some bending to just how Zabbix wants things done, but it’s nice. We have it monitoring all manner of stuff, hardware, power, cooling, services, batteries, weather, network, disks, etc

Absolutely terrible. Like you said, totally unreliable. Scaling it is insanely difficult. Documentation is weak. Their APIs seem like an afterthought and performance was pretty bad.

Nagios is not great, but it’s reliable and when it breaks you can figure it out.

One thing I like about zabbix is the excellent grafana plugin, which provides a very good ability to view and ack host-by-host alerts from within grafana.

That said, I'm not very familiar with the alternatives.

I've had a similar experience to you with Zabbix in the past. We have it bundled with some HPC stuff I support curently, it's ok but I prefer Sentry/TICK/Prometheus shaped things that we also run.

If you're happy with Icinga2, stick with that. I've used that too at a previous gig and found it better that Zabbix, but my personal take on it. YMMV

My experience with Zabbix has been positive. I deployed it 3 years ago and it's been solid since then. It monitors a few dozen CentOS VMs and a bunch of JBoss/JMS instances.

One feature I particularly like is the zabbix_send command, which I use to push the status of shell-scripted Borg backup jobs into Zabbix.

Zabbix looks like shit and feels like it was made in 1995 but it is great once set up.

If better visuals are needed, I would hook it up to Grafana. I have previously used Grafana with Graphite as backend but it was too unreliable. If it actually works with Zabbix then it could be the perfect match.

Yeah I agree. Zabbix sucked when I tried it many years ago. Definitely not going near it again.

Reply on news.ycombinator.com