Don’t mean to sound snarky, but there are tools that do this and have been for years. If you’ve been grepping through logs for the last 3 years, you’re doing it wrong for the cloud era.

Often times the answer is writing better alert triggers that take historical activity into account to cut down on false positives. Other times it’s simply to reduce the number of alerts. In every case you need an alerting strategy that takes balances stakeholder needs, and you need to realign on that strategy quarterly. It’s ultimately an operational problem, not a technical one.

Alas, back in the real world, logging is always the last thing teams have time to think about...