The comments on this point to another question of which this is an exact duplicate. Interested parties should definitely read the top answer there, as it was written by a member of the IEEE-754 committee.
Strangely, I was just adding some more material to this answer as I’ve never really been completely satisfied with it; then I flip over to hacker news and there it is. Anyway, if anyone has other IEEE-754 questions, I’ll try to answer.
The other reason mentioned is also unconvincing. Surely preserving the law x==x is more important than the law x==y <=> x-y==0.
Then we have:
> Regarding your comment "that doesn't mean that the correct answer is false", this is wrong. The predicate (y < x) asks whether y is less than x. If y is NaN, then it is not less than any floating-point value x, so the answer is necessarily false.
Clearly this reasoning is incorrect for the simple reason that the same reasoning can lead to the opposite conclusion. If y is NaN then we can't say that y < x is definitely true but neither can we say that y < x is definitely false.
Here is my argument for why NaN == NaN should be true. The == operator for floats should rarely be used. If you want to know whether two floats can be considered the same, then you should always use some other condition like |x-y| < epsilon, and NOT x == y. The reason is that floating point arithmetic is inexact, so even a slight roundoff error will make x == y false. So if its job is not to test whether two floats are the same numerically then what is the job for the == operator? Its job is to test whether x and y are precisely the same float. The x == y operator should not pretend to work on the abstraction that x and y are kinda-sorta real numbers, because such an operator is useless anyway because the criterion for that needs to be application specific. It should compare floats while admitting that we are comparing floats which can take on the value +infinity -infinity +0 -0 and NaN, and we are not comparing real numbers which do not have those values. That's why NaN == NaN should be true.
Not in languages that didn’t have a name for NaN it wouldn’t.
> you should always use some other condition like |x-y| < epsilon, and NOT x == y.
This is a popular myth, but it is false. There are circumstances in which exact equality comparisons in floating-point are perfectly appropriate.
> Its job is to test whether x and y are precisely the same float.
No; if it were we would have +0 != -0. You’re looking for totalOrder(x,y) or a related predicate.
...then use NaN = 0/0.
> This is a popular myth, but it is false. There are circumstances in which exact equality comparisons in floating-point are perfectly appropriate.
Can you give an example where == is appropriate, but where you want NaN == NaN to be false?
> No; if it were we would have +0 != -0. You’re looking for totalOrder(x,y) or a related predicate.
Exactly, +0 == -0 should be false too. totalOrder(x,y) is what == should have been (or at least the equality part of it).
p.s. An arguably more serious issue with IEEE floats is rounding modes. The round up/round down modes are tantalizingly close to providing the ability to do ironclad approximate real arithmetic in the sense that you could give upper/lower bounds on the exact value. The problem is that the rounding modes only involve giving an upper/lower bound on the output given an exact input, rather than an upper/lower bound on the output given an upper/lower bound on the input. So it's not compositional which means you can't guarantee anything.
Many languages throw an error when you divide by zero. As far as I can tell, if x != x didn't work you would have to resort to bizarre contortions like `x == float("NaN")` for such tests in Python (the problems: there are no NaN or Infinity constants and division by 0 throws).
Also, since you expect == to do bit equality on floats, you will be distinguishing between the huge number of representation of NaN, so NaN = 0/0 isn't valid even if there is a NaN constant (and which one would it be?).
Response to edit:
There can be several answers to this question. If you're gonna be stuck with the IEEE floats, obviously the right thing to do is an isNaN predicate, but arguably there shouldn't be more than 1 NaN in the first place.
On the other hand, X/0 (integer division-by-zero, where X is not a floating-point value either) usually does cause an exception. In fact, in bare-metal ASM without a trap-handler interrupt set up, it halts the processor entirely.
That's true for x86, but not for most other architectures.
You may not be able to do that it traps are enabled for the invalid exception. (The NaN may be coming in from a data file | over the wire | from another process prepared under an entirely separate floating-point environment that didn’t have traps enabled).
> Can you give an example where == is appropriate, but where you want NaN == NaN to be false?
I’ll give you one where == is appropriate but where you want -0 == +0:
if (x == 0) return someFormulaValidAtZero(x);
else return somethingEasierToComputeWithAnImplicitFactorOfx(x)/x;
Here’s my standard question for the “NaN should equal NaN" crowd, to which I’ve never seen a satisfactory response: If NaN == NaN, how should NaN compare against zero? I think we all agree that NaN != 0. If NaN < 0 or NaN > 0, you’re giving up x > y ==> -x < -y, which to my mind is a bigger loss than trichotomy. If NaN ?? 0, then trichotomy goes out the window, and presumably NaN ?? (any number), at which point NaN is unordered with respect to every number, but somehow equal to itself, which is also pretty darn weird.
You give something up any way you tackle this. You may reasonably argue that one of these options is less bad than another, but you can’t pretend that there’s a canonical right approach (though nearly every algebraist I’ve ever met believes that you should be able to until they really dig into it; analysts are much more realistic).
w.r.t. rounding modes, I partially agree. Had I been on the committee in 1985, I would have argued against formalizing round up and round down (round to zero is quite useful for some signal-processing applications). It’s a mistake to look at them and say “they aren’t interval arithmetic so they’re useless”, however. They were never intended to be interval arithmetic; rather they were intended to be building-blocks for interval arithmetic systems. (My problems with them are that (a) they’re global rather than operation-by-operation—this was corrected in the 2008 IEEE-754 document and (b) round-up and round-down are no more useful building blocks than round-to-zero (which we already have) and round-away-from-zero, which would free up space for another useful rounding mode (either round-ties-away “schoolbook rounding”) or round-to-odd, which is extremely useful for library implementors).
........................................if NaN is incoming from a data file, then tadaaaa there's your way to make a NaN to compare to. I can't believe we're still playing that game.
> I’ll give you one where == is appropriate but where you want -0 == +0:
And how would that example work for x very close to 0? Even in that example, you'd want |x|<eps. But anyway, I'm not even sure that having 2 zeros is a great idea either.
> If NaN == NaN, how should NaN compare against zero?
It shouldn't. < is an operation that works on the abstraction that floats are kinda-sorta real numbers, and NaN isn't so NaN < x should be an error.
> And how would that example work for x very close to 0?
Just fine; 0./0. is what this example needs to avoid, tiny/tiny is perfectly OK. That’s the whole point. You don’t want |x| < eps, you really do want x == 0. This happens all the time; if you’re not aware of such uses and you feel strongly about these issues, you may want to spend some more time studying the way these tools are used in low-level numerics.
> NaN < x should be an error.
That may be an option for some high-level languages. It’s very much not an option for C/FORTRAN-type languages, and not an option at the level of IEEE-754 specification. It can either be true or false or a trap (and the latter would be wholly inconsistent with the rest of IEEE-754). You don’t get to make up some exception or error or “not-a-boolean” or Maybe.
It sounds like you want a more abstract floating-point spec for higher-level languages; that’s fine. I think it’s a great idea, actually. IEEE-754 necessary needs to support very low-level implementations, however.
(That said, if NaN < x is an error, that will really cause issues when someone tries to use it as a key in an ordered container).
Tell me, which is clearer: x != x, or x == NaN. Obviously, the latter is much clearer. We don't live in the 1960's any more, we have libraries now. No need for getting NaN to be obscure.
> Just fine; 0./0. is what this example needs to avoid, tiny/tiny is perfectly OK. That’s the whole point. You don’t want |x| < eps, you really do want x == 0. This happens all the time; if you’re not aware of such uses and you feel strongly about these issues, you may want to spend some more time studying the way these tools are used in low-level numerics.
Please, do give a concrete example, if they are so abundant. Otherwise you will keep saying "yea, but for some examples, that's not an issue". Anyway, this is wholly besides the point. You still refused to answer the question I actually asked: NaN not 0.
> (That said, if NaN < x is an error, that will really cause issues when someone tries to use it as a key in an ordered container).
As it should. The current behavior may cause silent errors and corruption, instead of blowing up and notifying you. Which would you prefer?
Neither. isnan(x) is by far the clearest. Why are you worried about this?
> Please, do give a concrete example, if they are so abundant.
Looking at the OS X math library sources, since I happened to have them in front of me, the implementation of the following functions use an exact equality test with zero (this list is not complete; I simply grepped for “== 0.”): acos, asin, atan, atan2, cbrt, ceil, all of the complex.h functions, atanh, exp, sinh, erf, exp2, floor, fma, fmod, hypot, lgamma, log10, log2, logb, logf, nearbyint, nextafter, pow, round, asinh, tanh, sin, cos, tan, tgamma.
Looking at LAPACK, the following functions use one or more exact comparisons with zero (listing double-precision real functions only, and again this is an incomplete list): dbdsdc dbdsqr ddisna dgbcon dgbequ dgbequb dgbsvx dgebal dgecon dgeequ dgeequb dgees dgeesx dgeev dgeevx dgegv dgejsv dgels dgelsd dgelss dgelsx dgelsy dgesvj dgesvx dggbal dgges dggesx dggev dggevx dgsvj0 dgsvj1 dgtcon dgtsv dgttrf dhgeqz dhsein dlaed4 dlaed6 dlaein dlaev2 dlag2 dlagtf dlagtm dlagts dlagv2 dlahqr dlaic1 dlals0 dlalsd dlanv2 dlapy2 dlapy3 dlaqr0 dlaqr1 dlaqr2 dlaqr3 dlaqr4 dlaqr5 dlaqtr dlar1v dlarf dlarfg dlarfp dlarft dlarfx dlargv dlarrc dlartg dlarzt dlas2 dlascl dlasd4 dlasq1 dlasq2 dlasq3 dlasq4 dlasq6 dlasv2 dlasyf dlatzm dpbcon dpocon dppcon dpstf2 dpstrf dptcon dsfrk dspcon dsptrf dsptri dstedc dsteqr dsycon dsytf2 dsytri dtbcon dtbtrs dtfsm dtgevc dtgexc dtgsen dtgsna dtpcon dtptri dtptrs dtrcon dtrevc dtrexc dtrsen dtrsna dtrtri dtrtrs.
Any one of these, or millions of other functions, will provide your concrete example. Can we finally give up the pretense that such usage isn’t justified and widespread, or are you going to insist that you know more than the implementors of these functions?
Hold on. I am not the one who came with this argument that NaN == NaN should be false because that gives us a way to test for NaN, you are.
> Can we finally give up the pretense that such usage isn’t justified and widespread
Not yet. I've looked at a couple of your examples, and so far all of them either test whether an absolute value is zero, or whether a value that was previously assigned the exact value zero is zero. So they do not actually support your case.
And yet again, this is beside the point; your original claim was about NaN, not 0.
They are detecting precisely x == (+/-)0, whereas you earlier said "you should always use some other condition like |x-y| < epsilon, and NOT x == y”. They are all perfect examples of my case; the fact that some of them happen to be written as abs(x) == 0 doesn’t effect that (in the case of LAPACK, this is defensive programming because many LAPACK routines predate widespread adoption of IEEE-754; the compiler simply optimizes away the absolute value on a modern system).
My original claim was that neither NaN == NaN nor NaN != NaN is completely satisfactory, but the designers of the 8087 concluded that the latter provided an easier and more portable way to test for nan on systems that had no name for a NaN constant. I’m not the one who “came with this argument”, either; it’s directly from the writings of William Kahan, who was a consultant on the design of the 8087 and was the primary architect of original 1985 IEEE-754 standard.
Good job on that strawman by picking only half the sentence and out of context.
> I’m not the one who “came with this argument”
I don't care who originally came up with it, you are the one who brought this into the discussion, and then you pull a 180 "Why are you worried about this?".
Anyway, the mistakes made in IEEE floating point are nothing that programming language designers can't fix. Hopefully future languages will make the sensible decision to let == compare floats exactly, and stick the IEEE == deep inside some library somewhere as ieeeFloatingPointComparisonThatIsVirtuallyNeverUsed(x,y).
Edit: just checked and haskell behaves the same way.
There are many NaN values. Should they all compare equal to each other?
If there was any thought process to this change, would you mind sharing it?
Optimizing means trading other things away. One thing we're willing to trade away–if it's a clear win for quality–is fidelity to the original submission. HN's focus should always be on the content, and the details of the original submission are of little interest compared to the subject at hand.
Was this particular change a clear win for quality? Well, the new link was better, as was the related subthread, which even included the author of the content, who even served on the relevant IEEE committee, and who was even game to politely bandy about design decisions with strangers superciliously claiming (in the internet's own special way) that he and his colleagues had done their work badly. It's pretty hard to get higher quality than that.
HN isn't a purist's kind of place. Our goal to make the front page and the threads as interesting and substantive as possible. We change a lot of things to serve that principle. If we knew how to serve it better, we'd happily break more crockery.
Someone (perhaps a purist!) will object that terms like "quality", "interesting", and "substantive" have no precise definitions and are in the eye of the beholder. True, but (a) they're not arbitrary either, (b) the alternatives suck, and (c) someone has to make the calls. We don't get them all right, but we do try hard to correct mistakes, and I'll defend the principle any day.
> HN isn't a purist's kind of place.
I feel like that statement belongs in the guidelines... :-)