This is basically how Datadog works. Unfortunately they do service monitoring rather than exception tracking, but certainly lessons can be learned from them.

Hands down the best designed operations product I've used.