danlimerick

I apologize for the disruption we caused you when rolling out Grafana 9. We are working on improving our releases to Grafana Cloud and also on making sure that errors due to breaking changes in a major release won't affect customers in the future. As a Grafana Cloud customer, you shouldn't need to read docs about breaking changes when we upgrade your instance.

Internally (in the Grafana team), we call it unified alerting because it is a unified UI for non-Prometheus data sources together with Prometheus alerting (and Loki/Cortex/Grafana Cloud alerting) and allows you to see all your alerts together.

- For Prometheus, can view and search for alerts.

- For Loki and Cortex, can create and edit alert rules. See Alertmanager data source: https://grafana.com/docs/grafana/latest/datasources/alertman...

- For non-Prometheus data sources, uses the Grafana alerting engine but also has an embedded Alertmanager. Can use the same routing, grouping, deduping and silencing features that Alertmanager provides but for data sources like InfluxDB or MySQL.

As Kyle says, we are sidestepping the template variables problem by decoupling alerts from dashboards. You can instead link from an alert to a dashboard or panel (or run book) and get a similar experience to the old alerting.

When you get a notification, you have a link to the appropriate dashboard and it is also easy to create an alert from a query in a dashboard. You can see the linked alert rule in the panel edit mode.

Maybe the thresholds feature in Grafana would work for you?

Otherwise depending on the database, they have functions for drawing lines or you can do a hack like this one for Prometheus:

https://play.grafana.org/d/h1lglb7ik/threshold-hack?editPane...

We are also working on an expression language to backfill other data sources. You can see the frontend version of this - transformations in 7.0. The backend version which will support functions like timeshift is in alpha for now. Hopefully it will be officially released in the near future.

There is the timeshift function in the query editor but that is per panel and not per query. So you would have to have a panel per year.

It should be possible with SQL but it is trickier. TSDB's like Prometheus, Graphite and Influxdb (Flux) have the timeshift function built into their query languages. (https://community.grafana.com/t/advanced-graphing-part2-visu...)

Checkout the contribute folder on GitHub. Here is the developer guide:

https://github.com/grafana/grafana/blob/master/contribute/de...

Have you seen this talk/blog from last year's Monitorama about anomaly detection with Prometheus:

https://about.gitlab.com/blog/2019/07/23/anomaly-detection-u...

This has been possible for ages. The built-in text panel has an HTML option (though you have to disable this setting to allow script tags: https://grafana.com/docs/grafana/latest/installation/configu...)

There are some external plugins that offer more advanced support:

- https://grafana.com/grafana/plugins/aidanmountford-html-pane... - https://grafana.com/grafana/plugins/ryantxu-ajax-panel

We know that there are some gaps in the alerting feature (we dogfood it ourselves). The Grafana team will be focusing a lot more on alerting in 2020. For Grafana 7.0 in May, we are aiming to build better alerting that retains the simplicity of the current alerting but that will fill some of the those gaps. The new engine will decouple alerting from the graph panel and hopefully sidestep the problem with template variables. Once we have got further in the design stage then we will share more with the community about the proposed solutions.

There is some documentation about the Loki search syntax in the Grafana docs:

http://docs.grafana.org/features/explore/#logs-integration-l...