Monitoring != Alerting on critical failures. It's worthwhile to have a real-time data feed for useful metrics(e.g. avg load, latency, requests, etc) not just to alert on critical failures, but to understand the overall behavior of your system, and to predict potential failures based on the cumulative trend over some window. Not sure how a single failure on a single test communicated via email can be similarly useful.

I guess its a matter of standard on what "minimal monitoring" entails.