I wrote a little glue to make this straightforward for anyone else who uses Prometheus/Alertmanager: https://github.com/jrockway/alertmanager-status This ensures that the website check checks the health of the whole alerting pipeline; Prometheus has an always firing alert, Alertmanager is set to send that alert to alertmanager-status, and alertmanager-status starts failing its external health check if it isn't seeing that alert firing at the configured interval. If one of [Prometheus, Alertmanager, alertmanager-status] fails, then your website health check fails.
FWIW our core application is hosted in AWS but we maintain our own Grafana infrastructure independently. So it's not hosted on our own tech, per se, though we're still responsible for keeping it online.
This looks great & would also love to see a self-hosted option. Honestly the more stuff like this that gets rolled into the OSS Grafana it actually makes me both more likely to try it and then more likely to eventually end up on the managed Grafana Cloud, as I will inevitably get sick of trying to maintain our own separate infra & the business case for centralising in Cloud makes more and more sense.