That mantra is simply bullshit.
I have written massively distributed systems which have an insanely high reliability requirement and it is really not the answer. I've been doing this for 25 years.
A more appropriate statement is:
Fail early and gracefully, recover always, expect failure.
Fail early - assertions up front. Prevention is better than cure.
Fail gracefully - don't allow assertions to take the entire process out. Don't allow the language to crash the process out.
Recover always - design your system for recovery and understand recovery conditions.
Expect failure - know where and when something is going to fail and handle it.
Fail fast, fail often results in nasty shit like processes hung in restart cycles etc.