Why is NaN not equal to NaN?
stackoverflow.com
stackoverflow.com
Strangely, I was just adding some more material to this answer as I’ve never really been completely satisfied with it; then I flip over to hacker news and there it is. Anyway, if anyone has other IEEE-754 questions, I’ll try to answer.
The other reason mentioned is also unconvincing. Surely preserving the law x==x is more important than the law x==y <=> x-y==0.
Then we have:
> Regarding your comment "that doesn't mean that the correct answer is false", this is wrong. The predicate (y < x) asks whether y is less than x. If y is NaN, then it is not less than any floating-point value x, so the answer is necessarily false.
Clearly this reasoning is incorrect for the simple reason that the same reasoning can lead to the opposite conclusion. If y is NaN then we can't say that y < x is definitely true but neither can we say that y < x is definitely false.
Here is my argument for why NaN == NaN should be true. The == operator for floats should rarely be used. If you want to know whether two floats can be considered the same, then you should always use some other condition like |x-y| < epsilon, and NOT x == y. The reason is that floating point arithmetic is inexact, so even a slight roundoff error will make x == y false. So if its job is not to test whether two floats are the same numerically then what is the job for the == operator? Its job is to test whether x and y are precisely the same float. The x == y operator should not pretend to work on the abstraction that x and y are kinda-sorta real numbers, because such an operator is useless anyway because the criterion for that needs to be application specific. It should compare floats while admitting that we are comparing floats which can take on the value +infinity -infinity +0 -0 and NaN, and we are not comparing real numbers which do not have those values. That's why NaN == NaN should be true.
Not in languages that didn’t have a name for NaN it wouldn’t.
> you should always use some other condition like |x-y| < epsilon, and NOT x == y.
This is a popular myth, but it is false. There are circumstances in which exact equality comparisons in floating-point are perfectly appropriate.
> Its job is to test whether x and y are precisely the same float.
No; if it were we would have +0 != -0. You’re looking for totalOrder(x,y) or a related predicate.
...then use NaN = 0/0.
> This is a popular myth, but it is false. There are circumstances in which exact equality comparisons in floating-point are perfectly appropriate.
Can you give an example where == is appropriate, but where you want NaN == NaN to be false?
> No; if it were we would have +0 != -0. You’re looking for totalOrder(x,y) or a related predicate.
Exactly, +0 == -0 should be false too. totalOrder(x,y) is what == should have been (or at least the equality part of it).
p.s. An arguably more serious issue with IEEE floats is rounding modes. The round up/round down modes are tantalizingly close to providing the ability to do ironclad approximate real arithmetic in the sense that you could give upper/lower bounds on the exact value. The problem is that the rounding modes only involve giving an upper/lower bound on the output given an exact input, rather than an upper/lower bound on the output given an upper/lower bound on the input. So it's not compositional which means you can't guarantee anything.
Many languages throw an error when you divide by zero. As far as I can tell, if x != x didn't work you would have to resort to bizarre contortions like `x == float("NaN")` for such tests in Python (the problems: there are no NaN or Infinity constants and division by 0 throws).
Also, since you expect == to do bit equality on floats, you will be distinguishing between the huge number of representation of NaN, so NaN = 0/0 isn't valid even if there is a NaN constant (and which one would it be?).
Response to edit:
There can be several answers to this question. If you're gonna be stuck with the IEEE floats, obviously the right thing to do is an isNaN predicate, but arguably there shouldn't be more than 1 NaN in the first place.
On the other hand, X/0 (integer division-by-zero, where X is not a floating-point value either) usually does cause an exception. In fact, in bare-metal ASM without a trap-handler interrupt set up, it halts the processor entirely.
That's true for x86, but not for most other architectures.
You may not be able to do that it traps are enabled for the invalid exception. (The NaN may be coming in from a data file | over the wire | from another process prepared under an entirely separate floating-point environment that didn’t have traps enabled).
> Can you give an example where == is appropriate, but where you want NaN == NaN to be false?
I’ll give you one where == is appropriate but where you want -0 == +0:
if (x == 0) return someFormulaValidAtZero(x);
else return somethingEasierToComputeWithAnImplicitFactorOfx(x)/x;
Here’s my standard question for the “NaN should equal NaN" crowd, to which I’ve never seen a satisfactory response: If NaN == NaN, how should NaN compare against zero? I think we all agree that NaN != 0. If NaN < 0 or NaN > 0, you’re giving up x > y ==> -x < -y, which to my mind is a bigger loss than trichotomy. If NaN ?? 0, then trichotomy goes out the window, and presumably NaN ?? (any number), at which point NaN is unordered with respect to every number, but somehow equal to itself, which is also pretty darn weird.
You give something up any way you tackle this. You may reasonably argue that one of these options is less bad than another, but you can’t pretend that there’s a canonical right approach (though nearly every algebraist I’ve ever met believes that you should be able to until they really dig into it; analysts are much more realistic).
w.r.t. rounding modes, I partially agree. Had I been on the committee in 1985, I would have argued against formalizing round up and round down (round to zero is quite useful for some signal-processing applications). It’s a mistake to look at them and say “they aren’t interval arithmetic so they’re useless”, however. They were never intended to be interval arithmetic; rather they were intended to be building-blocks for interval arithmetic systems. (My problems with them are that (a) they’re global rather than operation-by-operation—this was corrected in the 2008 IEEE-754 document and (b) round-up and round-down are no more useful building blocks than round-to-zero (which we already have) and round-away-from-zero, which would free up space for another useful rounding mode (either round-ties-away “schoolbook rounding”) or round-to-odd, which is extremely useful for library implementors).
........................................if NaN is incoming from a data file, then tadaaaa there's your way to make a NaN to compare to. I can't believe we're still playing that game.
> I’ll give you one where == is appropriate but where you want -0 == +0:
And how would that example work for x very close to 0? Even in that example, you'd want |x|<eps. But anyway, I'm not even sure that having 2 zeros is a great idea either.
> If NaN == NaN, how should NaN compare against zero?
It shouldn't. < is an operation that works on the abstraction that floats are kinda-sorta real numbers, and NaN isn't so NaN < x should be an error.
> And how would that example work for x very close to 0?
Just fine; 0./0. is what this example needs to avoid, tiny/tiny is perfectly OK. That’s the whole point. You don’t want |x| < eps, you really do want x == 0. This happens all the time; if you’re not aware of such uses and you feel strongly about these issues, you may want to spend some more time studying the way these tools are used in low-level numerics.
> NaN < x should be an error.
That may be an option for some high-level languages. It’s very much not an option for C/FORTRAN-type languages, and not an option at the level of IEEE-754 specification. It can either be true or false or a trap (and the latter would be wholly inconsistent with the rest of IEEE-754). You don’t get to make up some exception or error or “not-a-boolean” or Maybe.
It sounds like you want a more abstract floating-point spec for higher-level languages; that’s fine. I think it’s a great idea, actually. IEEE-754 necessary needs to support very low-level implementations, however.
(That said, if NaN < x is an error, that will really cause issues when someone tries to use it as a key in an ordered container).
Tell me, which is clearer: x != x, or x == NaN. Obviously, the latter is much clearer. We don't live in the 1960's any more, we have libraries now. No need for getting NaN to be obscure.
> Just fine; 0./0. is what this example needs to avoid, tiny/tiny is perfectly OK. That’s the whole point. You don’t want |x| < eps, you really do want x == 0. This happens all the time; if you’re not aware of such uses and you feel strongly about these issues, you may want to spend some more time studying the way these tools are used in low-level numerics.
Please, do give a concrete example, if they are so abundant. Otherwise you will keep saying "yea, but for some examples, that's not an issue". Anyway, this is wholly besides the point. You still refused to answer the question I actually asked: NaN not 0.
> (That said, if NaN < x is an error, that will really cause issues when someone tries to use it as a key in an ordered container).
As it should. The current behavior may cause silent errors and corruption, instead of blowing up and notifying you. Which would you prefer?
Neither. isnan(x) is by far the clearest. Why are you worried about this?
> Please, do give a concrete example, if they are so abundant.
Looking at the OS X math library sources, since I happened to have them in front of me, the implementation of the following functions use an exact equality test with zero (this list is not complete; I simply grepped for “== 0.”): acos, asin, atan, atan2, cbrt, ceil, all of the complex.h functions, atanh, exp, sinh, erf, exp2, floor, fma, fmod, hypot, lgamma, log10, log2, logb, logf, nearbyint, nextafter, pow, round, asinh, tanh, sin, cos, tan, tgamma.
Looking at LAPACK, the following functions use one or more exact comparisons with zero (listing double-precision real functions only, and again this is an incomplete list): dbdsdc dbdsqr ddisna dgbcon dgbequ dgbequb dgbsvx dgebal dgecon dgeequ dgeequb dgees dgeesx dgeev dgeevx dgegv dgejsv dgels dgelsd dgelss dgelsx dgelsy dgesvj dgesvx dggbal dgges dggesx dggev dggevx dgsvj0 dgsvj1 dgtcon dgtsv dgttrf dhgeqz dhsein dlaed4 dlaed6 dlaein dlaev2 dlag2 dlagtf dlagtm dlagts dlagv2 dlahqr dlaic1 dlals0 dlalsd dlanv2 dlapy2 dlapy3 dlaqr0 dlaqr1 dlaqr2 dlaqr3 dlaqr4 dlaqr5 dlaqtr dlar1v dlarf dlarfg dlarfp dlarft dlarfx dlargv dlarrc dlartg dlarzt dlas2 dlascl dlasd4 dlasq1 dlasq2 dlasq3 dlasq4 dlasq6 dlasv2 dlasyf dlatzm dpbcon dpocon dppcon dpstf2 dpstrf dptcon dsfrk dspcon dsptrf dsptri dstedc dsteqr dsycon dsytf2 dsytri dtbcon dtbtrs dtfsm dtgevc dtgexc dtgsen dtgsna dtpcon dtptri dtptrs dtrcon dtrevc dtrexc dtrsen dtrsna dtrtri dtrtrs.
Any one of these, or millions of other functions, will provide your concrete example. Can we finally give up the pretense that such usage isn’t justified and widespread, or are you going to insist that you know more than the implementors of these functions?
Hold on. I am not the one who came with this argument that NaN == NaN should be false because that gives us a way to test for NaN, you are.
> Can we finally give up the pretense that such usage isn’t justified and widespread
Not yet. I've looked at a couple of your examples, and so far all of them either test whether an absolute value is zero, or whether a value that was previously assigned the exact value zero is zero. So they do not actually support your case.
And yet again, this is beside the point; your original claim was about NaN, not 0.
They are detecting precisely x == (+/-)0, whereas you earlier said "you should always use some other condition like |x-y| < epsilon, and NOT x == y”. They are all perfect examples of my case; the fact that some of them happen to be written as abs(x) == 0 doesn’t effect that (in the case of LAPACK, this is defensive programming because many LAPACK routines predate widespread adoption of IEEE-754; the compiler simply optimizes away the absolute value on a modern system).
My original claim was that neither NaN == NaN nor NaN != NaN is completely satisfactory, but the designers of the 8087 concluded that the latter provided an easier and more portable way to test for nan on systems that had no name for a NaN constant. I’m not the one who “came with this argument”, either; it’s directly from the writings of William Kahan, who was a consultant on the design of the 8087 and was the primary architect of original 1985 IEEE-754 standard.
Good job on that strawman by picking only half the sentence and out of context.
> I’m not the one who “came with this argument”
I don't care who originally came up with it, you are the one who brought this into the discussion, and then you pull a 180 "Why are you worried about this?".
Anyway, the mistakes made in IEEE floating point are nothing that programming language designers can't fix. Hopefully future languages will make the sensible decision to let == compare floats exactly, and stick the IEEE == deep inside some library somewhere as ieeeFloatingPointComparisonThatIsVirtuallyNeverUsed(x,y).
Edit: just checked and haskell behaves the same way.
There are many NaN values. Should they all compare equal to each other?
If there was any thought process to this change, would you mind sharing it?
Optimizing means trading other things away. One thing we're willing to trade away–if it's a clear win for quality–is fidelity to the original submission. HN's focus should always be on the content, and the details of the original submission are of little interest compared to the subject at hand.
Was this particular change a clear win for quality? Well, the new link was better, as was the related subthread, which even included the author of the content, who even served on the relevant IEEE committee, and who was even game to politely bandy about design decisions with strangers superciliously claiming (in the internet's own special way) that he and his colleagues had done their work badly. It's pretty hard to get higher quality than that.
HN isn't a purist's kind of place. Our goal to make the front page and the threads as interesting and substantive as possible. We change a lot of things to serve that principle. If we knew how to serve it better, we'd happily break more crockery.
Someone (perhaps a purist!) will object that terms like "quality", "interesting", and "substantive" have no precise definitions and are in the eye of the beholder. True, but (a) they're not arbitrary either, (b) the alternatives suck, and (c) someone has to make the calls. We don't get them all right, but we do try hard to correct mistakes, and I'll defend the principle any day.
> HN isn't a purist's kind of place.
I feel like that statement belongs in the guidelines... :-)
For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
I thought that the philosophy of JS was to make meaning out of every execution and value, even the meaningless ones? Kind of the opposite of the fail-fast-and-hard philosophy.
String index of NaN? JS will find a way.
-> Can't be used as a branching conditional.
Think about it: if (NaN) { a; } else { b; }
The pedantically-correct answer would be to execute neither, but that wouldn't be very helpful (however fun debugging that might have been).
The pragmatically-correct answer, imho, would be to throw an error. Would I rather have a; or b; executed, when I can't trust the conditional value? Neither, by the time the NaN has hit branching code, it's time it escalated into an error.
edit: I may have reinvented "Maybe" https://news.ycombinator.com/item?id=7747284
For some reason I hadn't thought about it till you mentioned it, but I do like the idea of throwing an exception when a nonsensical comparison of NaNs happen. Of course that would be inimical to the way JS normally operates.
I don't think what you're getting at is exactly Maybe. You're able to easily programmatically determine whether a Maybe has a value, whereas the equality of NaNs is basically unknowable.
0 - Talking in JS again, although it's similar for Java and other languages.
I was talking about this property of Maybe. I was thinking that assigning a NaN value to a boolean should probably be "fine" (no exception), but using that when branching would mean that this carried-forward-error (which the NaN represents, effectively) is just about to go beyond "infecting" just data, to affecting code flow. So, throw.
That's not true, NaN means "not a number". Take for example f(x) = 1/x. What is f(0)? Infinity? Well, you can't divide by zero. In fact, I tricked you. When you talk about a function in mathematics, you need more information that just the formula, you also need to know over what domain I am allowed to evaluate such a function. What I should have said was, define f: [1, 2] -> R s.t. f(x) = 1/x. Now you are not allowed to ask the question, "what is f(0)?" because I didn't include zero in the domain. You might then ask, "so how do you define f at zero?" That question mentions 'f', but the 'f' it is referring to is now different. Such an 'f' you are allowed to ask this question for would be the following:
f:[0, 2] -> R
f(x) = 1/x when x > 0
f(0) = 1
Now, when you ask me what f(0) is, I can tell you. It is 1. "That's cheating!" you might add. No it isn't. It is a different function with different properties. Just because the formula is the same does not mean you can evaluate it wherever you want.
Note: you are not allowed to divide by zero. The result would be meaningless if you could. And 'NaN' is only something the computer spits out when it encounters an operation that doesn't return a number. Usually, a computer will give you 'Inf' when you try to divide by zero numerically. Mathematically, you cannot do it. So don't try.
NaN means 'not a number', and '=' is an infix operator to compare numbers. You can't compare for equality on things that are not numbers.
You can compare infinities, however. "I thought you said dividing by zero was meaningless!" It is. I'm not going to divide by zero, but I can still talk about infinity. Take for example the following function:
f: (0, 1] -> R
f(x) = 1/x
f is defined on every number strictly greater then zero, but less than or equal to one. Zero is not in the domain, so I can't evaluate f there. However, I can still study what happens near zero. Take a really small number bigger than zero. The reciprocal of that is huge. Now take a smaller number, the reciprocal becomes even bigger. In fact, we can keep doing this: construct any positive sequence that converges to zero (but never attains it). Now evaluate f at each element in that sequence, you will notice that the result becomes larger and larger. We call the limit of this 'infinity', and we have notation for this. Notice I did NOT evaluate f at zero, I merely studied what happens as I approach zero. Infinity is not a real number, and I cannot have f ever evaluate to infinity, but I can talk about limiting properties. Computers do this too, but they have a much lower standard for doing so. Essentially, anything bigger than the maximum value of a double (about 10^300, I think?) is called 'Inf' on a computer.
Edit: I forgot to address the second paragraph of the parent comment.
>For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
This is not a sensible thing to do.
Let me elaborate. I'm going to ignore any cancellation issue and say it is definitely true that 2/x != 3/x for x != 0. Let's also say that each of these functions has domain (0, 1]. These two functions are not equal are x = 0 because you cannot evaluate them at x = 0. It is forbidden. However, behaviour near to zero is very similar. See my discussion above. In fact, you can make a stronger statement. You can say that as x approaches zero (from the positive side), in the limit, 2/x and 3/x are equal. That is, they have the same limit as you approach zero. This does not make them equal at zero, because I cannot evaluate them at zero. To make this point absolutely clear, I will add zero into the domain and define two different functions:
f:[0,1] -> R
f(x) = 2/x for x > 0
f(0) = 1
and
g:[0,1] -> R
g(x) = 3/x for x > 0
g(0) = 2
These functions have the same limit as you approach zero, but they are not equal at zero. I can evaluate them at zero this time, because I defined what it meant to do so. And indeed f(0) = 1 != 2 = g(0).
That is not true. You can mathematically divide by zero, just not over the Reals (or any other nonzero commutative ring). There is no reason that you cannot define a number system where division by zero is allowed, however in doing so you loose some nice properties that we enjoy about the Real numbers. A common example from math is the real projective line (which adds a point at infinity, and defines that a/0=infinity when a=/=0).
Another commonly used number system that defines division by zero is the IEEE floating-point standard. In fact, IEEE defines division for all values of 0 (it has a positive and a negative 0). Specifically:
a/+0=+infinity | a>0
a/+0=-infinity | a<0
a/-0 =-infinity | a>0
a/-0 = +infinity | a<0
0/0 = NaN | for any combination of 0 values
I should have been a little clearer there. Thanks.
I absolutely agree with you. But the domain and range of the functions I defined were all subsets of the reals. And computers don't deal with number systems other than the reals very well. (In fact, computers don't even deal with the reals themselves very well either!)
There are some packages that allow one to operate over those spaces, but they are not commonly used in the commercial space. And, of course, it's fairly easy to extend the reals to the complexes.
Thanks for pointing that out.
> undefined == undefined
In Haskell? Exception: Prelude [...]
It seems like the desired result is to make it determinable when a NaN value is produced in a language that doesn't have an is_nan function (such as an assembly language lacking such a predicate) so that a simple boolean statement does the trick, for example:
def is_nan(val): return not (val == val)
This absolutely bizarre statement of non-identity gives us a surefire test. It is from the perspective of higher level languages that the non-identity of NaN with itself seems bizarre.
def is_nan(val): return (val == NaN)
It's true that it can't just give NaN or you'd be stuck.
But then, it's indeed quite surprizing. It get worse on pure tanguages where the expressions (0 / 0) and (0 / 0) are equal before evaluation, but different afterwards. Also, it trashes hash tables.
Yet, NANs being different is the sanest mathematical definition. Maybe people should have opted for the ugliest choice because of real world concerns (it wouldn't be the first time), but this time, they didn't.
Any invalid calculation produces NaN as opposed to raising an exception/signal because that would be non-portable (especially in the early 80s). The goal was that a correct algorithm on one platform or in one language would translate and be correct somewhere else. Further, some algorithms depend on being able to probe a function to find its bounds, requiring them to be able to locate the out of bounds condition without halting execution; doing that in a safe, portable way is actually quite hard unless you do it in software as part of the spec, hence NaN.
If NaN==NaN, then (NaN/NaN)==1, etc etc and you still end up in a world of nonsense as far as naive algebra is concerned. This is just some of my fellow programmers whining and crying about having to think about floating point, the same sort of complaining I hear about Unicode (and full of just as much self-sure awful advice that produces wildly incorrect results for everything except the complainer's specific use case).
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.ht...
http://www.eecs.berkeley.edu/~wkahan/ieee754status/754story....
But is this an effective way of looking at the question? In reality what you're trying to do is not compare a NaN to another NaN, you're actually attempting to check if both objects are NaN. This hurts my brain.
#include <math.h>
#include <stdio.h>
int main(int argc, const char* argv[] )
{
double nan = 0.0 / 0.0;
double x = 1.0;
double y = nan + x;
printf( "nan=%f\n", nan );
printf( " x=%f\n", x );
printf( " y=%f\n", y );
return 0;
}
/* Should print (gcc):
nan=nan
x=1.000000
y=nan
*/Much better, in terms of early failure, would be signalling NaN or something like...
...
maybenan_double maybe_nan = 0.0 / 0.0;
double x = 1.0;
if(is_nan(maybe_nan)) {
fprintf(stderr, "maybe_nan is NaN\n");
return 1;
}
double not_nan = from_maybenan(maybe_nan);
double y = not_nan + x;
...
For textures, they are great as they don't interpolate w/ nearby neighbors and properly discard fragments & vertices without the aforementioned cost of an extra boolean attribute.
Most here don't remember the totally sad state of floating point before this came to be.
The nice thing about NaN is that you can just do a calculation as normal, and check for NaN at the very end. This means you don't have to do an expensive test/branch after every arithmetic or other numeric operation. The hardware is much, much easier to design if you don't have to make it branch, and the code is much faster without the compiler inserting branches. People who work with floating point numbers every day care very deeply about performance.
If you don't care as much about performance, why not write your code in a language that does throw exceptions? Python, for example.
Those of us that do use numerics love NaN, love signed zeros, and can live with NaN ≠ NaN even though it's kind of dumb.
Edited to add: That said, I'd rather that 100 hours of computation be cut short than take longer and be equally useless...
What if you're doing some kind of matrix math and you just need the results of the diagonal and a single NaN creeps in somewhere but doesn't destroy your matrix. Would you accept your program needing 50% more LOC and taking 4x as long for this result?