(IIUC, that is. It may be something like a "should" not a "must".)
Quoth the standard (emphasis mine):
> For an operation with quiet NaN inputs, other than maximum and minimum operations, if a floating-point result is to be delivered the result shall be a quiet NaN which should be one of the input NaNs.
My assertion, that R's NaN is not "non-standard", seems upheld by the article. It's a quiet NaN with a payload, which is well-defined by the IEEE 754 standard.
As other posters pointed out, it's relying on a "should" behavior from the spec, which is risky but common. It sounds like disabling the "RunFast" mode cleared up their issues, which seems quite far from it being an "obscenely bad" design decision.
It's not terribly unusual to require IEEE 754 compliance in numerical code, like the usual options for avoiding --ffast-math -style stuff.