I think it is important in the context of what people are trying to use NNs for. Most of the time it is some type of classification, in which case there is always a chance it will be completely wrong in the sense that anything other than the expected output is not correct in any way.

Which leads into unknowingly building NNs that are actually building classification networks and not realizing approximations might not fit into your model.