Two Case Studies of NaN
sebsite.pw
sebsite.pw
I wish there were a recent remake of it.
It is certainly orders of magnitude simpler than specifications for things like HTML, CSS, C++ or Vulkan.
The source of the problem is elsewhere. Unlike in the first 2 or 3 decades after the invention of automatic computers, nowadays there are much more than a half of the programmers who never encounter in their jobs the need to solve problems where complicated numeric computations are important and where the knowledge of mathematics branches like arithmetic and algebra is important.
Moreover, the programmers that happen to work on compilers or standard libraries for programming languages are even more likely to belong to the class of programmers who are not interested in computational applications.
This has led to the current situation, when almost all programming languages, especially the most popular of them, have bad defaults for the IEEE 754 options and they provide very awkward means to access all the facilities specified by the standard.
A decent programming language should have 2 distinct types for floating-point numbers, one for those that cannot be NaNs (to be used only in a program that enables and handles invalid operation exceptions) and one for floating-point numbers that may be NaNs (to be used in programs that disable the invalid operation exception, like most programming languages wrongly do by default).
Moreover, a decent programming language should have 14 relational operators, to enable the correct use of partially-ordered sets, e.g. also for user-defined data types, not only for floating-point numbers.
With a standard programming language, which has only 6 relational operators, if one insists on keeping disabled the invalid operation exception, then every floating-point comparison must be preceded by testing the operands for being a NaN, which is tedious.
You've already explained why such a type separation is a bad idea - those types being only usable with specific setup; so, completely utterly breaking composability.
Never mind this making float ops impure & stateful (also forbidding autovectorizing anything with more than one potentially-NaN-producing op if you don't want to break semantics).
Perhaps if hardware supported embedding exception behavior in individual instructions this'd not be insane, but I haven't heard of any architecture having such, making it a complete non-option for sane languages designed to be used.
(there is the option of making compilers insert the necessary fp state transitions, though then you'll have the desire to embed it into calling conventions & function types to avoid transitions when a certain state is expected to stay for a prolonged period of time, which, while certainly possible and would be quite neat to have (also for rounding modes, FTZ/DAZ, etc), is very much in the territory of something almost noone will bother doing, and as such things would just quietly be slower)
While partially-ordered types are neat, what can you really sanely do with a comparison over such? Seems like a rather pointless thing to have.
They depend on a set of configuration flags, which are stored in some internal register, e.g. the MXCSR register of Intel/AMD CPUs.
The same program can provide wildly different results if you run it twice, with different settings in the MXCSR register.
Any programmer who writes floating-point computations should be very aware that any FP operation depends on the state of a global register.
Hopefully, all functions that are invoked are well behaved and if they ever change the global settings they restore them to the value that the register had upon entry, otherwise the effects would be unpredictable.
What you say about the floating-point operations being "impure" is incorrect. A pure function may have not only explicit input parameters, but also implicit input parameters, a.k.a. global variables that are read-only.
For example any such function can be marked with __attribute__((pure)) in compilers like gcc and clang.
The functions that have only explicit input parameters are a subset of the pure functions. For example, such functions are marked with __attribute__((const)) in the gcc and clang compilers.
In a rigorous manner, FP operations would not belong in an __attribute__((const)) function, but the compilers do not care about the global variables that are CPU registers, because they assume that those are preserved by all functions (and upon context changes they are saved and restored by the operating system).
Enabling some or all kinds of FP exceptions has absolutely no influence on auto-vectorization. It also has no influence on speed, in well-designed CPUs.
The only difference is that the 1st error will crash the program, while with all errors masked the program will terminate successfully regardless how many errors happened. However in this case the user should analyze the results, to determine whether they are correct.
If with the exceptions masked, the programmer fails to test for NaNs wherever necessary, then the errors will be lost and bad results will be wrongly considered as correct.
Many real life things can be modeled correctly only as partially-ordered sets. When this is ignored and they are handled as totally-ordered sets, it is very easy to generate erroneous results. This can be avoided by various tests before comparing any values, but this is exactly what would be done automatically by the compiler in a programming language where data types that are partially ordered can be defined.
Having certain exceptions enabled only for an operation or for a function, instead of for an entire executable program, would be a very bad idea.
The reason is that this would offer no guarantees that can simplify the program. If you know that the invalid exception is enabled, then there cannot exist NaNs and you can omit testing for them everywhere. Similarly, if the overflow exception is enabled, you know that there cannot be infinities, so you can omit any tests for infinities.
If exceptions were enabled only for a part of the program, then the rest of the program can still generate special values and you need to test for them everywhere. So you get the worst of both choices, a complex program that can be crashed by an error.
But sane programming languages entirely hide that (to the same extent that they hide integer flags and registers in general), at the very least by default; and in such sane languages, programmers do not need to be aware of any of this mess.
e.g. C's requirement of "#pragma STDC FENV_ACCESS ON" for non-default FP status, and lack thereof in 99.99% of code; C spec even explicitly notes:
> With these conventions, a programmer can safely assume default floating-point control modes (or be unaware of them). The responsibilities associated with accessing the floating-point environment fall on the programmer or program that does so explicitly.
(never mind basically every other language, which don't even allow anything other than round-nearest-ties-even + no-exceptions)
> Enabling some or all kinds of FP exceptions has absolutely no influence on auto-vectorization. It also has no influence on speed, in well-designed CPUs.
If you want to have deterministic semantics (i.e. want to be a sane language), it absolutely does; it means that, in `a+b; x[i];`, the `x[i]` load must not happen if `a+b` results in a trap. So you cannot ever execute the next loop iteration before being sure the previous doesn't trap. Entirely forbidding autovectorization.
If you want to still have autovectorization, you'd need to have some washy semantics of like "the trap of an fp op can be delayed to an arbitrarily-later point in time, up to [insert some messy ambiguous incomplete attempt at establishing some limit to this]", which, besides making basic code non-deterministic (yikes! we don't need more of that in the world), will make proving compiler optimization correctness extra-messy.
Non-trapping exceptions (i.e. accruing) would be simpler to preserve through vectorization (allowing reordering up to reading the status, which is explicit in code), but you're back to allowing observable NaNs and making programmers insert manual checks. (and even then, you'd run into issues if you wanted to vectorize the tail of a loop on architectures without maskable FP SIMD ops)
(and, of course, unrelated to any of this, giving the programmer control over FP status means compilers are massively-restricted on optimizing code - can't DCE float ops, can't insert new FP ops even if it would be beneficial, etc)
https://gcc.gnu.org/onlinedocs/gcc/Common-Attributes.html states:
> However, functions declared with the pure attribute can safely [...] in a way that does not affect [...] or the observable state of the program
So an exception-aware FP op cannot be __attribute__((pure)) in a program that may read the FP status later, or has trapping on FP exceptions enabled.
This is obvious from the fact that gcc & clang will entirely delete a call to a pure function if the result isn't used, thereby making potential FP traps not happen, or accrued exceptions not mutated.
(also traps on exceptions would make FP ops clearly not pure by the definition in functional programming too)
> If exceptions were enabled only for a part of the program, then the rest of the program can still generate special values and you need to test for them everywhere. So you get the worst of both choices, a complex program that can be crashed by an error.
Would be perfectly fine if combined with separate NaNful and NaNless types; design the language s.t. exception handling state must always be (or is automatically set to) the necessary one, and you never get NaNs in the NaNless types.
Is this a well-known set that I've never heard of? What are the other 8? Subset, superset, structural equality? Logical equivalence?
The last is "less or greater", as used in BASIC and SQL, instead of "not equal".
In a partially ordered set, their 6 negations are distinct relations, so 6 + 6 makes 12 relational operators.
The last 2 relational operators of the 14 are "less or equal or greater", which could be written "<=>" or ">=<", together with its negation (which is true when one of the operands is a NaN).
Zero is not absence of a value. It has a value, it happens to be 0. Same for undefined/infinity. They exist.
Let the hardware engineers figure out an efficient way to deal with it. If it costs more, well if we add up all of the costs of bugs due to zero/null conversions, maybe it was worth it.
Equality comparison must be reflexive for things on which it is defined.
It is wrong to say that NaN is not equal to Nan (because a NaN is neither equal nor not equal with a NaN).
When the invalid operation exception is disabled, because the programmer is too lazy to write a SIGFPE handler or for whatever other mechanism is provided by the operating system to report hardware exceptions, then the set of floating-point numbers is extended with NaNs and the resulting extended set is no longer a totally-ordered set, but only a set on which a partial order is defined.
While floating-point number with NaNs are the best known because they are standardized, there are a lot of real sets that are best modeled as partially-ordered sets, so programming languages should have supported this and programmers should have been taught that partial order relations are something normal and natural, not something weird.
In any partially-ordered set there are elements like the NaNs, on which the relations of order and of equality may be undefined.
Thus NaN is neither equal nor not equal with NaN, but this relationship is undefined.
While for totally-ordered sets 6 relational operators are sufficient, for partially-ordered sets 14 relational operators are necessary.
When the 6 ALGOL relational operators (inherited by all later programming languages) are extended to 14, 6 new operators are provided by the negation of the 6 ALGOL operators, because in a partially-ordered set none of the 6 negations is equivalent with one of the 6 standard operators (like it happens in totally-ordered sets, where e.g. "not less" is the same as "greater or equal"). (In this statement, I considered among the 6 standard operators "less or greater", like in BASIC or SQL "<>", instead of "not equal", like in C "!=".)
The remaining 2 extra relational operators are "less or equal or greater", which could be written "<=>" or ">=<", which may be true only when no operand is a NaN, and its negation, which is equivalent with "one of the operands is a NaN".