In my experience, the drives report "healthy" until they fail, then they report "failed"
I've personally never tracked the detailed metrics to see if anything is predictive of impending failure, but I've never seen the overall status be anything but "healthy" unless the drive had already failed.
> I've personally never tracked the detailed metrics to see if anything is predictive of impending failure
Backblaze has!
From experience, we have found the following five SMART metrics indicate impending disk drive failure:
SMART 5: Reallocated_Sector_Count.
SMART 187: Reported_Uncorrectable_Errors.
SMART 188: Command_Timeout.
SMART 197: Current_Pending_Sector_Count.
SMART 198: Offline_Uncorrectable.
That's good to know, I might start tracking that. I manage several clusters of servers and hard drive failures just seem pretty random.However, one time a drive got a burst of reallocated sectors, it stabilized, then didn't have any problems for a long time. Eventually it wouldn't power on years later.
I've never seen an HDD fail overnight without any indication at all.