Float and integer arithmetic follow two different paradigms | Hacker News Reader

Float and integer arithmetic follow two different paradigms

blog.pkh.me

67 pointsibobev19 comments

The "integer arithmetic paradigm" would mean either checking after (almost) every operation or living with potentially incorrect results.

The first is terribly inefficient, the second is just wrong (even if insanely common).

This is sad because there is no reason at all why integers couldn’t follow what OP calls the "float paradigm". It doesn’t even have to be slow. Most modern processors (if we ignore x86) support some form of sticky arithmetic flags. Unfortunately, programming languages don’t support them, so they aren’t used.

There is also something to be said about the special treatment division by zero gets.

Pretty sure zig has safe integer operations: integers aren't allowed to overflow with the standard operators, there are overflowing and saturating operators if that's what you want.

Rust and D and probably others have this too. The interesting question is:

Can you chain these operations in a way that will propagate the error to the end result while simultaneously have a compiler produce code without branches after every operation. Safe integer operations alone are not enough for that.

I think the operators panic. In zig, If you want safe operations I think there are functions in the stdlib

The branching does not cost as much as you think, I'm pretty sure the compiler marks the erroring path as cold, which the cpu can use to elide most of the branch cost through specex

A branch is a branch and no branch will always be better. With sticky flags there is no difference in performance to regular integer arithmetic.

> Most modern processors (if we ignore x86) support some form of sticky arithmetic flags. Unfortunately, programming languages don’t support them, so they aren’t used.

I'd say the reason that programming languages don't have built-in support for checking the CPU's sticky status flags is precisely because of a lack of x86 support. Plenty of languages do have support for checking overflow on each individual operation, e.g. Rust's `overflowing_foo` methods, which return a tuple whose second member is a boolean indicating overflow: https://doc.rust-lang.org/std/primitive.i32.html#method.over...

Sticky flags tend to be really annoying for a compiler to support for various reasons, but principally it makes every operation have a hidden dependency to shared global state that is almost never read. Note that IEEE 754 standardized support for sticky flags 40 years ago, but support for these sticky bits is still poor to nonexistent in most programming languages, and sticky bit stuff for floating-point operations is less problematic than integer operations because FP math is already far more black box in practice.

I mostly agree, but not with the flags part. From my past comment (https://news.ycombinator.com/item?id=45174074):

>I sometimes wish that signed integers were symmetrical. i8 would represent the range of [-127 to 127] with 0xFF representing NaN. Any operation which can not be computed (division by zero, overflows, operation with another NaN, etc.) would result in NaN. For further symmetry we could do the same for signed integers as well.

> i8 would represent the range of [-127 to 127] with 0xFF representing NaN

There's an interesting "special case" of that in the Alesis HR16 and SR16 drum machines. They store 16-bit sample data in an 8-bit ROM. Okay, you say, nothing unusual about that, it's int16 pairs, right?

Nope.

It's int8 data, that gets scaled. Drum sounds tend to decay, so you start off with the 8 bits of sample data representing the eight highest bits of the 16 bit DAC value. At some point the maximum level has decayed by a half so all subsequent values will fit in 7 bits. So there's a "tick" to -128 in the data that makes the sound ASIC "change up a gear" and shift the 8-bit data right one place with the data scaled to suit. Once it decays enough again, another gearchange, another scaling, another "notch" on the Christmas tree-shaped sample.

At the very end of the sample to signify that the note is to stop, you just have a tail of -128s until the shifter has "run off the end" - from memory (it was around 2010 that I last looked at this) there are eight or nine "ticks" in total which are necessary for the sample to stop before it goes right on into the start of the following sample.

So, yeah, nothing new under the sun. Signed 8-bit audio with -128 considered a special value.

> The "integer arithmetic paradigm" would mean either checking after (almost) every operation or living with potentially incorrect results.

Would an int type with three special values +inf, -inf and NaN be useful? In addition to handling overflow, it would be symmetrical; the relation

  INT_MIN == -INT_MAX

would hold.

For the float stuff, beware of `-ffinite-math-only`. If enabled, `isfinite()` and may compile out as assumed true (with similar assumptions around `isnan()` and `isinf()`).

And `-ffinite-math-only` is enabled by `-ffast-math` which in turn is enabled by `-Ofast`.

Not only that: `-ffast-math` might change the result of flops when the arguments are regular floats too. (It abandons IEEE754 compliance)

I was skeptical about this article's claim about LLMs so I tried it out myself. Looks like there are several layers of pedantry involved, for example, strictly speaking in base C it's undefined behavior to divide by 0.0, even for floats, so the author's initial implementation is incorrect and a optimizer could elide the call to isfinite in certain scenarios where the denominator is 0.0.

Presumably the airtight implementation assuming Annex F is as follows:

    #pragma STDC FENV_ACCESS ON

    int my_div(float x, float y, float* r) {
      if(!r) {
        return -1;
      }

      fenv_t environment;
      if(feholdexcept(&environment) != 0) {
        return -1;
      }

      float result = x / y;
      int failed = fetestexcept(
        FE_INVALID | FE_DIVBYZERO | FE_OVERFLOW | FE_UNDERFLOW);

      if(fesetenv(&environment) != 0 || failed != 0) {
        return -1;
      }

      *r = result;
      return 0;
    }

When I asked about the author's implementation I got the response that the author's implementation is deficient in several areas:

   - Underflow produces a finite result and passes this check.
   - Overflow can produce a finite maximum value under some rounding modes.
   - Quiet NaNs and valid infinite results fail this check, even without an arithmetic exception.

     It also assumes floating-point traps are disabled. Annex F defines the arithmetic behavior; your function still needs a chosen definition of “failure.”

So the answer really is... safely dividing two floats really depends on what trade-offs you're willing to make, and the original reply of just checking for a 0 denominator is honestly the most sensible, portable, and safest.

FWIW one can configure floating point exceptions at runtime at no overhead, see `man 3 fenv`

fenv: https://manpages.debian.org/testing/manpages-dev/fenv.3.en.h... :

> These eleven functions were defined in C99, and describe the handling of floating-point rounding and exceptions (overflow, zero-divide, etc.).

Except gcc doesn't actually support it to any sane extent (it ignores and warns on "#pragma STDC FENV_ACCESS ON", which C requires for fenv.h to actually function; and as such gcc makes a bunch invalid optimizations); clang supports it, but only as of somewhat-recently.

It's no overhead in the same sense that zero-cost exception-handling is zero-cost: turning them on properly (e.g., #pragma STDC FENV_ACCESS ON or equivalent command-line flags) disables a fair amount of optimizations which incurs an overhead in and of itself.

Oh interesting I thought it was more or less a wrapper around MXCSR (or equivalent for other architectures)

That is what you have to do enable turning FP exceptions into traps on a hardware level. But the FP instructions your compiler actually emits as hardware instructions bears only a superficial resemblance to the code you originally wrote. Compilers generally assume FP operations are pure operations without side-effects, so it happily speculates them (including ones that might generate the trap you're looking for) on paths where they didn't, or remove them, or evaluate them at compile time instead of run time, etc. Turning this behavior off so that the traps you get are actually the traps you expected to get involves disabling a lot of optimizations.

Reply on news.ycombinator.com