Hacker News Reader

Comment by anonymous | Hacker News Reader

In what context would I want less accurate results from my arithmetic expressions? I mean, I have some numbers, I write an expression with them, and then I expect to get the most accurate possible result. Doesn't it make a lot more sense to let users specifically specify that a given expression is to be evaluated only using 32-bit float values, than to make everybody always cast every value in every expression to 64-bit float? I can tell you that having to sprinkle (double) all over your Java code (and then an additional (float) in front) to be sure you're getting the most available precision is pretty tedious.

View on HN

Two reasons:

1. I mostly write graphics and browser layout code when I use floating point. I never want 64-bit precision. I want the fastest thing available, and I'm willing to accept some error. It's surprisingly tricky to get C to actually do 32-bit floating point arithmetic, and I suspect we may actually gain a little performance over existing browser engines with Rust here (probably negligible, but hey).

2. Wouldn't you expect these two expressions to be the same (assuming b, c, d are all f32)?

    let a: f32 = b + c / d;

and

    let tmp = c / d;
    let a: f32 = b + tmp;

In Kahan's preferred semantics, they are not the same. This interferes with refactoring, complicates the language, and violates intuition, IMHO. Introducing temporaries without changing the semantics is something programmers should be allowed to do.

1. Faster? Really? Those values only live in registers for a short while, is saving one or two cycles all that important?

2. That is an interesting point. Gut instinct says tmp should end up being f64 and then both expressions would be the same. Equally valid is the interpretation that the user selected explicitly for conversion to f32 of the intermediate value. We all know computers don't perform platonically perfect math with real numbers, there isn't much you can do about it, but I think the choice which results in the most precision should win at the end of the day.

Perhaps in a language which doesn't require specifying types like Rust, the best thing to do is to simply dispense with 32-bit floating point values entirely. I'd be equally happy with compiler flags like -fmore-precision. Or maybe you could have other arithmetic operators like +64 and +80 which cast all their arguments to 64/80-bit floating point numbers and produce a result with that precision. So I'd write

    let a: f32 = b (+80) c (/80) d as f32

Given that Rust is a pretty young language and still far from a 1.0 release, do consider if you can provide any kind of syntax to actually make full use of computers' numerical capabilities.

Regarding point (1), it starts to matter when autovectorization kicks in. If this expression is in a loop and the compiler can autovectorize (which rustc does, if you have a new enough version), you really want to be able to pack more values into a SIMD register if you can. (This is one point in which this paper is out of date…)

As for point (2), well, I think that making the size of "tmp" f64 would interact badly with other features like operator overloading and generics, since you're playing fast and loose with types.

Gory details (warning, type theory ahead): "+" is implemented by a trait, Add(RHS,Result) where RHS and Result are concrete types and there is a functional dependency so that Self + RHS → Result. This fundep is necessary because otherwise "let tmp: b + c" wouldn't work, as the compiler doesn't know what the type of "tmp" is (is it f32 or f64? It won't guess.) So you can't simultaneously implement f32 : Add(f32,f32) and f32 : Add(f32,f64). We'd have to introduce a lot more experimental type machinery to make this work (a "floating point functional dependency", I guess?) and I'm not sure there wouldn't be fallout (for example, I can foresee issues with higher-kinded type parameters).

It's very simple: as soon as the floating value is read, it should go to the 64-bit virtual register unless you specify that register to be 32-bit too.

If you want to use 32-bit partial results, you should say that explicitly. Autovectorization can be applied even if the rules are like suggested.

I don't know, I wouldn't expect that tbh but I am used to C. I also don't see why I should need it, I am not going to compare non-zero floating point numbers anyway but I would take better accuracy in long computations any day. I think there is value in following the standard (what people doing floating point computations are used to) and you need very good reason to deviate from it. You state that performance gains will be negligible if any in your domain but you could potentially cause a lot of headache in other domains. I don't think elegance/simplicity/negligible performance gains (if you are right about it at all) is enough of a reason to deviate from standard behavior.

> I don't think elegance/simplicity (if you are right about it at all) is enough of a reason to deviate from standard behavior.

The elegance and simplicity in this case are what are needed to make generics work. (See my other response for the gory details.)

Still the fact is that computation has different goals than "generics" or whatever you want from the rest of the language "cleanness." On x86 and most other CPU's, reading 32-bit FP variable and doing computation with 64-bits doesn't cost you anything: it's just how you fill the bits of the register during the load. As long as you don't have FP registers spills, computation has the same speed, and the final store to the destination is of course faster if you store only 4 bytes instead of 8.

You can omit some aspects from your language, but don't be surprised that people who use C continue to use C. If you really want bigger acceptance, even if it's "harder" to do it "properly," you can't pretend that people don't need it.

Cleanliness is not about theoretical purity in this case. It's about type soundness, something that C doesn't have but which is a major design goal of Rust.

And it does cost you something: autovectorization, as I explained earlier.

The problem with these kinds of discussions is that we aren't talking about expressive power: it's obvious that you can do both 64-bit or 32-bit math in either C or Rust. The issue is just what the defaults should be. In these discussions, small communities (like scientific computing in this case) tend to have oversimplified notions of how things should "obviously" be without understanding the subtleties from other perspectives (like PL design). You don't "just" make the "+" operator return different types depending on context in a language with typeclasses. That can break soundness.

Do you really think that C introduced that accidentally? Don't you think that it were also easier for all the compilers written in the last 40-50 years to simply have 1-1 correspondence between stored variables and partial results?

The reason they didn't do this was that the implementers knew that most of calculations turn out wrong if the partial results are too short.

> Don't you think that it were also easier for all the compilers written in the last 40-50 years to simply have 1-1 correspondence between stored variables and partial results?

Not really; the size of intermediate representations wasn't even specified prior to C99. It was easier to just leave 80-bit intermediate representations in e.g. the x87 FPU unit.

You're wrong regarding "80 bits." The intermediate was double at least twenty years: the practice is older than the standards, as I already claimed. You can check the code, I claim that the first thing C std libraries do on x86 with 80-bit registers is set the precision to 64-bits. The FPU unit has 80 bits registers but the computation can be fixed and it was.

It's not "easier" to leave it at 80-bits: you get bad numerical outputs if you don't implement the spilling to 80-bits too and if you don't have proper language support.

Numerics is hard but it should be done right.

Note the practice. The standards were intentionally "almost useless" because they reflected the influences of those involved.

No, I was correct about the standards: http://www.altdevblogaday.com/2012/03/22/intermediate-floati...

C98: "5.10 The values of the floating operands and the results of floating expressions may be represented in greater precision and range than that required by the type; the types are not changed thereby."

You're right that most compilers in practice changed the precision to double. I think we're in violent agreement (and I learned a lot from this debate!) :)

On page 43, Kahan gives a four rule of thumbs. According to rule 3 (declare temporary/local variables all with the widest ﬁnite precision that is not too slow nor too narrow), I think Kahan would actually agree with you.

Maybe you confused Kahans rant with C semantics?

More precisely, I think the scenario is different with type inference. What is the type of "c/d", if c and d are f32? Would it be unreasonable, if the expression had type f64 or even more precise?

Reply on news.ycombinator.com