Rendering floating numbers: advances in problems you didn’t know you had (2011) | Hacker News Reader

Rendering floating numbers: advances in problems you didn’t know you had (2011)

serpentine.com

76 pointsgoranmoomin22 comments

This needs a 2011 on it, which will explain why it is missing the very recent Ryū algorithm, which I believe is the fastest algorithm at this point:

https://dl.acm.org/doi/10.1145/3192366.3192369 (PLDI 2018 paper)

https://github.com/ulfjack/ryu (C source code)

Apparently Ryu is slower on shorter numbers than the Rust standard library's implementation which uses Grisu/Dragon. It might be only due to the implementation though. https://www.reddit.com/r/rust/comments/93d045/fast_floattost...

Also it seems that Ryu prints some numbers in weird ways even though "nicer" representations exist: https://github.com/rust-lang/rust/issues/52811#issuecomment-...

That “weirdness” is just scientific notation, which the Ryu implementation always emits. It is “superficial” in the sense that it’s separate from the core algorithm. When I adapted it for C++17 charconv in MSVC, I implemented fixed, general, and “plain shortest” notation, which will print 0.3 there. (General notation follows C printf %g’s rules for switching between fixed and scientific; “plain shortest” notation selects the one that’s fewer characters, tiebreaking to prefer fixed).

In ClickHouse* we patched Ryu to provide nicer representations and to have better performance for floats that appear to be integers.

* https://github.com/ClickHouse/ClickHouse/pull/8542

The change to Ryu is here: https://github.com/ClickHouse-Extras/ryu/pull/1

Stephan T Lavavej also had a surprisingly fun and approachable talk on that problem, talking about how charconv got implemented and improved in Microsoft's C++ standard library: https://youtu.be/4P_kbF0EbZM

What do you make of this algorithm which claims to be faster than the Ryu algorithm [0]? I think this is the implementation in C++ [1]. The author also worked on an algorithm for half-precision conversions. [2]

[0] http://www.fox-toolkit.org/ New, faster, and often more accurate floating point to string conversion. No, this is not based on Ryu paper. New method reliably generates 16 digits, and does much better rounding.

[1] https://github.com/gogglesguy/fox/blob/77396c6/lib/fxprintf....

[2] [PDF] http://www.fox-toolkit.org/ftp/fasthalffloatconversion.pdf

If I understand correctly, it's a printf-style function for which the %g format code outputs the shortest decimal string that round trips. This implies that it would output no more than seventeen significant figures, regardless of the requested precision.

The first thing I notice is this comment in a later changelog:

> I believe we have a very fast numerical conversion that appears to be reliably generating about 15 to 16 digits out output [last digit may be off by 1 or 2 in typical case].

If the last digit of a fifteen- to sixteen-digit output is off by one or two, are we sure it round trips?

Second, the test coverage for this feature in tests/format.cpp looks pretty sparse, and it's not obvious to me how the output is verified.

Interesting find. I may play around with this later.

Edit: I tried the test cases from here:

https://www.exploringbinary.com/the-shortest-decimal-string-...

Here's the result:

format="%.30g" output="0.1"

format="%.30g" output="50388143.06823721984"

format="%.30g" output="54167628.18"

format="%.30g" output="9161196241250.051072"

The first and third cases output the correct string. The second case outputs nineteen digits, instead of seventeen, and does not round trip (i.e., it rounded incorrectly). The last case outputs nineteen digits instead of fifteen, but does round trip.

Nice work, thanks for looking into it.

Looking at the code briefly, it doesn't seem like it's actually an accurate conversion. The core of the conversion algorithm appears to be based on doing a series of floating-point multiplies, which of course will tend to trash the last few bits. Using volatile to try to avoid reassociation here. But there's no comments indicating why the rounding is going to be correct in the range provided, and my gut is that it's not going to be correct. (Especially since the double-to-int conversion is truncating, and I'd naïvely expect half the rounding error to round down, which means the last digit is off by one).

Actually, there's been another advance since Ryu: Schubfach by Raffaello Giulietti and its optimized derivative Dragonbox by Junekey Jeon. Here's a cool page with implementations and a benchmark: https://github.com/abolz/Drachennest/

This page has a benchmark graph, but without labels on the axes, thus it doesn't even say which algorithm was faster.

My guess is that smaller y values indicate proportionally faster executions.

I have to deal with this all the time. Unfortunately I wrote my programs in FreePascal, because Pascal was supposed to be the best, fastest, and safest language, but it is very bad at solving these kinds of problems.

I wrote my own rendering function. It was not so hard. I just wrote functions to calculate with arbitrary precision decimal numbers and then put the float in there. Reallly slow, but perfectly accurate unlike those in FreePascal.

Now FreePascal has implemented Grisu3, so it is fine to use their functions. Although it is printing weird numbers like 1.0000000000000001E-1 for 0.1. With my arbitrary precision calculation, I get 0.1 is exactly 0.1000000000000000055511151231257827021181583404541015625 and can round it to the shortest representation 0.1.

But the reverse problem -- parsing floating numbers -- is also really hard. And I cannot just bruteforce it by printing arbitrary precision binary floats. FreePascal's parsing function still rounds wrongly ( https://bugs.freepascal.org/view.php?id=29531 ), so it is risky to use

For fast parsing of 99% cases, the Eisel-Lemire algorithm can be used. Unfortunately, there is no FreePascal implementation. I hope someone will port it FreePascal, so I do not have to do it.

And then it still leaves the hard 1% of cases. How do you parse them? Is there a simple way to do it with some arbitrary precision calculations?

This is the kind of stuff that’s got me hooked on CS: small problems that no one thinks about much but have dramatic impact on developers’ lives and the things they build. There’s a lot of societal problems that we’ve inadvertently created, (see Technopoly by Postman or The Social Dilemma on Netflix) but it’s nice to be reminded about what makes this field fun and worthwhile.

This is in the running for the best HN post I've seen this year. I've read all about floating point precision, never about rendering.

The existence of this problem is so obvious in hindsight and I was so completely unaware of it previously.

I think Python was the first to bring this into my consciousness. See this enhancement request, which changed the float-to-string conversion from seventeen significant figures to shortest:

https://bugs.python.org/issue1580

After they did that, I learned that Scheme specifies similar behavior. In fact, I think it was added to R4RS after the Steele and White paper. As the article says, this behavior spread across languages, so that it's weird to see something different now.

If you want to nerd out on this, here are a few other posts that explore the topic:

https://www.exploringbinary.com/the-shortest-decimal-string-...

https://www.exploringbinary.com/java-doesnt-print-the-shorte...

Here is the reference for the most recent work (2019) of note in this domain. Ulf Adams extended his 2018 Ryū algorithm to implement Ryū printf which is 4x faster than the best other implementation tested on Linux, Mac and Windows. https://dl.acm.org/doi/pdf/10.1145/3360595

Unless you actually need to output human-readable numbers for some reason, the sensible approach when serializing floating point numbers is to just use hexfloats and skip the problematic "rendering" part altogether.

I completely agree

That is great, until you have to use JSON

It could be argued that if so many people have to use json to serialize floats then our problem is not a technical one.

Mojibake? Now that's something I haven't seen in a while. (In the post script)

Reply on news.ycombinator.com