Comment by acqq | Hacker News Reader

Comment by acqq | Hacker News Reader

Parent Full thread

The biggest problem is that Java designers ignored the numerical aspects of language almost completely. The IEEE 754 is an international standard for FPU calculations which is implemented on practically all relevant CPU's for at least some 20 years.

Java designers practically ignored the standard since they obviously didn't understand it or decided that serious computations will never be important for their target group. A lot of people commenting here don't understand it either. OK, a lot of people don't need it. But those that need such calculations simply can't implement decent numerical calculations in Java.

So the next time somebody claims that "Java is comparably fast to C" remember that it can be true only for some of the algorithms.

If you ignore IEEE 754 for Rust you're creating one more language that will not be "as good as C." And here I don't mean "C as defined by language lawyers" but "C as implemented in the compilers to at least minimally support the use of the underlying hardware for those who need that."

And those who need that are the people who use computers for computations. Don't ignore them. Or at least don't be surprised that they stick to old Fortran and C compilers.

Sure, I agree we shouldn't neglect the needs of scientific computing. But we need concrete suggestions. It's not our intent to "neglect IEEE 754" (Graydon Hoare was on that committee!)

I haven't seen any concrete reason that eliminating implicit floating point promotion affects scientific code. Just use doubles if you want 64-bit precision. We match the underlying LLVM IR much more closely than C does.

Concrete:

C didn't have "in a language standard" IEEE support for a while, but it had in practice. C99 had it standardized, and it's not "just libraries." If you make a serious language, at least that's what you shouldn't ignore.

http://en.wikipedia.org/wiki/C99 (IEEE 754 floating point support)

Still the practice is even more important than the standardization. In the compilers I used I actually had to do "bit-tricks" to do some parts of manipulations of doubles and floats. Last times I've checked, even initializing values to the desired values was something where you actually "danced" around the standard. It works and it's needed, even if you won't find it in the standards. For an example of what people actually have to use and C "allows" see:

http://en.wikipedia.org/wiki/Fast_inverse_square_root

If you make a language that is "nice theoretically" but doesn't allow you to use the hardware, it will remain just "nice theoretically."

Edit: regarding "just use doubles:" it's about the engineering trade-off: for a lot of uses you want to read from and store to floats but do all the calculations in doubles. Only the calculations which are specifically designed to use floats as partial results should be done with floats. The language should be easy to accommodate this assumption. Computation has simply different logic than "classes" and "theoretical types."

Yes, for that stuff I totally agree that we should do it right. It isn't very well defined at the moment on Rust, as we mostly delegate to what LLVM does; when we do spec it, we should make sure to spec it correctly. (Note that delegating to what LLVM does helps us here, as they have to implement C99 correctly and we can piggyback on that.)

Would syntax for choosing between 64-bit intermediate vs. 32-bit intermediate be appropriate?

> syntax for choosing between 64-bit intermediate vs. 32-bit intermediate

That's exactly what Kahan proposed to be added to Java in the title work ("How Java’s Floating-Point Hurts Everyone Everywhere," which most of the people arguing here didn't digest fully, but I have understanding for that, people who didn't have to do computations don't have enough familiarity with the whole subject), see the page 78:

"To fit in with Java’s linguistic proclivities, Borneo allows a programmer to declare a minimum width to which everything narrower is to be widened before Java’s rules are invoked: anonymous float follow Java’s rules ( Borneo’s default; it must match Java’s ) anonymous double widen every narrower operand to double as does K-R C anonymous long double widen every narrower operand to long double ( use on Intel ) anonymous indigenous widen every narrower operand to indigenous. Of course, Java should be repaired promptly to adopt anonymous double as its default, which would then become Borneo’s default too. The scope of an anonymous declaration is a block."

"just use doubles"

Okay, and all of the data that I send off to my GPU for either computation or rendering - I should just accept the huge performance hit I get when using double on them?

Or I am up against the cache limits - I should just make everything double and accept that my code now thrashes as the CPU's cache misses?

Sorry, that probably comes off as antagonistic. But this is not a 'just do X' situation. Those of us that do heavy computation tend to think about this a lot, if it was as trivial/cost-free as changing float to double we would have done that already.

What I mean is that if you want to do 64-bit math on 32-bit values, convert them to 64-bit first. (This matches what the compiler frontend is inserting into the IR and assembly anyway.) Not "use 64 bit everywhere".

To provide meaningful results, 64-bit math should be implicit, 32-bit explicit.

The reason is: just like you believe that "it's same" or "it's not important" (it's not the same and it is very often actually important) there are a lot of programmers who don't want to spend any energy in "making the computer doing what's right" they simply assume that the computer "does what's right." In your language case, if it's hard to have 64-bit partial results, the programmers won't do this where they should. And the proper computations are the serious thing. Like calculating if the bridge will fall or not.

OK, so we agree that this is serious, and we need to make sure that programmers understand whether 32-bit or 64-bit arithmetic is being performed, lest the bridge collapses.

In Rust a programmer familiar with the rules would write it this way, making it clear what's happening:

    bridge.d = (bridge.a as f64 * bridge.b as f64 + bridge.c as f64) as f32;

Taking advantage of the implicit conversions, a programmer familiar with C++'s rules might write:

    bridge.d = bridge.a * bridge.b + bridge.c;

Now suppose that, years later, someone comes along and wants to refactor the code by adding a temporary. In Rust:

    let tmp = bridge.a as f64 * bridge.b as f64;
    bridge.d = (tmp + bridge.c as f64) as f32;

In C++:

    auto tmp = bridge.a * bridge.b;
    bridge.d = tmp + bridge.c;

And now they've changed the meaning of the code in a serious way, because adding a temporary silently changes the meaning of the code.

It seems you missed something:

    auto tmp = bridge.a * bridge.b;

The type of the right hand expression is 64-bit FP by definition we discuss, so "auto" means "double" in your example. Only if you do simple assignment the type remains 32-bit, every "+" "-" "*" etc promotes it to 64. It can actually be implemented in your language too and the rest of the compiler will keep functioning.

    #include <stdio.h>

    int main() {
        float a = 1.0f;
        float b = 2.0f;
        auto c = a + b;
        printf("%lu\n", sizeof(c));
        return 0;
    }

    $ clang++ test.cpp
    $ ./a.out
    4

This is a perfect example of how C++'s floating point conversion rules are so complex nobody knows them!

Have you tried float a = 1, b = 7; auto c = a / b? It should give you 8 (I don't have clang here). Note that in the example you constructed you have practically a constant which doesn't lose precision by staying as 32 bits?

Have you also checked compiler options that affect the computations? The proper numerics is sometimes (often!) not default, but C and C++ compilers are not "one size fits all."

Tried that, also in -std=c++11 mode, still 4…

Edit: By default clang++ has FLT_EVAL_METHOD set to 0 (i.e. Rust's behavior). When I tried with -mno-sse FLT_EVAL_METHOD was set to 2 (long double promotion), but the sizeof was still 4.

Now you know the pain of the people who do numerics. C99 should use 8 bytes for partial results at least the real ones (not the constants computable by the compiler). You should see that in the debugger too. C++ is unfortunately fully another topic. I also admit that I don't know if clang implemented numerics properly, I've never used it. This looks suspicious and at least conceptually wrong: it loses the precision "just because."

After something like:

        fld dword ptr a
        fdiv dword ptr b

there is a full 64-bit result in the FPU waiting to be stored even if the recent compiler implementers think "32-bits should be enough for everybody." That's that "bridges-can-fall-once dangerous" thing I've mentioned.

I've also learned something: "auto" seems to be more dangerous than I expected! Wow.

g++ seems to duplicate your clang results, for the record (as more or less expected).

Okay, and all of the data that I send off to my GPU for either computation or rendering - I should just accept the huge performance hit I get when using double on them?

When necessary, manually cast your floats to doubles, perform the calculation, then assign the result back to a float. It achieves the same thing, though it is, of course, verbose. For example:

  float result = (float) ((double) floatVar1 + (double) floatVar2);

You obviously don't have to write numerical code. I also guess, if you had to, you'd also be fast to start avoiding language that forces you to do the above dance everywhere.

You obviously don't have to write numerical code. I also guess, if you had to, you'd also be fast to start avoiding language that forces you to do the above dance everywhere.

I didn't claim it would be pretty, I was just demonstrating that it's possible.

Reply on news.ycombinator.com