> As an example, imagine you're monitoring /. Nagios runs NRPE, NRPE looks at df /, and if it's 85 percent full (by default), sends a warning page. At 90 percent it sends a critical page. A smarter system collects the df / results, delivers it to a central timeseries database. The new data point is used to create a new projection, and the new projection is used to determine the time to an actual failure. The system above might have an idea of how long it takes to respond, repair, and resolve and issue a page when the disk will fill up if not responded to within 4 hours. That's the ideal solution, IMO.

We've actually implemented exactly this at my current workplace. We have a nagios check that queries graphite and calculates a "days until full" value, and alert based on that. We have similar checks for monitoring other infrastructure. These checks take a lot of work to get the right calculation and threshold values, but once they work, it's pretty great.