Hacker News Reader

Comment by XorNot | Hacker News Reader

Prometheus can monitor via proxies. Or monitor via federated scraping.

Though it is an odd requirement to need to only have Prometheus outside a firewall.

Edit: guessing from your post - the way I'd do it is run a Prometheus instance at all 4 sites, and have them all federate each other. That way each site is the HA redundancy for the others.

View on HN

That's sensible, thanks.

What if the inside of the network is further segregated, and the Prometheus instance does not have access to all endpoints. Can I use the push gateway as a "proxy", or is a direct route required.

Sorry for the stupid questions. Now it's getting entirely academic of course :)

Edit: just re-read your post and saw the proxy note. Time to read the documentation again. This setup should make for a good blog post.

It's advised not to use the pushgateway in that fashion, it's for service-level batch jobs - not trying to subvert your organization's network security policies. See https://prometheus.io/docs/practices/pushing/

Can you monitor alerts via a federated scraping? So for example you have one master alerts dashboard?

Alerts are push anyway at the moment - so sending multiple Prometheus alerts to a single alertmanager is easy.

But yes, the actual "alert firing" metrics are exposed by the federated endpoints.

You'd lose annotations and other metadata doing that, plus it's another hop on the critical path of alerting. Probably not the wisest of ideas. You want your alerts coming from as close to what you're monitoring as possible.

Reply on news.ycombinator.com