Herbgrind analyzes binaries to find inaccurate floating point expressions | Hacker News Reader

Herbgrind analyzes binaries to find inaccurate floating point expressions

herbgrind.ucsd.edu

70 pointsbshanks13 comments

Wow, didn't expect this tool to be on Hacker News six years after publication! Herbgrind author here, ask me any questions you like! Also, if you're interested in this stuff, the Herbie project (which I also worked on) for numerical program synthesis is also really helpful for writing numerical code, and has had a lot of development over the years.

What do you think programming languages/libraries/tools could do to make floats less scary for the common people? I feel there is lot of superstition around floats, some of it more well deserved than others, but it leads people to approach fp with great distrust.

Of course herbie is already good step here, but it is still somewhat niche.

> to make floats less scary for the common people?

The obvious thing would be making sure that it's easy to get a unambiguous, natural representation of what a float value actually is. (Similar to what C printf %a specifiers produce, rather than something like Ryu float-to-decimal, which is useful for what a float value means for human-readable purposes.)

   0X1.999999999999AP-4
  +0X1.999999999999AP-3
  =0X1.3333333333334P-2
  ≠0X1.3333333333333P-2

is ugly (and could be improved[0]), but far more understandable (and less superstition-inspiring) than:

  0.1 + 0.2 = 0.30000000000000004 ≠ 0.3

0: At the very least, please make it easy to get "0xA.BCDEFp+1", with capital "ABCDEF" and lowercase "xp", so the digits look like digits and the punctuation looks like punctuation.

I agree, although strictly speaking I think conventional decimal representation should be unambiguous, i.e. there is always one float that is closest to the decimal value

Strictly speaking, unambiguous decimal representation - such as Ryu and its less-efficient predecessors - is unconventional, but I agree that it's correct and should be the default. (That's the "0.1 + 0.2 = 0.30000000000000004 ≠ 0.3" in my comment.) My point is that it should also be easy to get a natural representation that corresponds closely to the actual bits of the float, in rouchly the same way that 0xAAAB corresponds closely to the actual bits of (int16_t)-21845, and so gives you a chance to notice that something weird is going on when, for example, you encounter a multiplication by -21845. (Something like hex(*(uint64_t*)&x) can work, but makes printf %A look beautifully readable by comparison. Also most languages - especially interpreted ones - manage to be inferior to C in this respect.)

Huh, good question. I think having a herbie-style error visualization graph built into an IDE would go a long way. In practice I think dynamic sampling is pretty good at capturing error behavior, and immediate visual feedback on the floating point code you're writing would make it a lot easier to trust the working code you have, and understand the problems with the error-prone computations. Something like https://github.com/herbie-fp/odyssey, but analyzing the fp expressions in the code files you're editing.

Is it feasible to run this on something large like Solvespace[1] (CAD) which is ~5MB executable? Or would we just get an insanely long list of issues?

[1] https://solvespace.com/index.pl

There are hundreds of numerical algorithms in there, and we have some bugs that might be related to this kind of implementation error.

It should be possible. Herbgrind is designed under the philosophy that it's not bad numerics themselves that are significant, it's how they affect the outputs of your program. So the reports are organized around program outputs (either through print calls, or you can annotate values you care about with macros), and errors are reported in terms of how they are affecting your outputs. That should allow you to skip any numerical errors that are insignificant in the final output, and prioritize the ones that are affecting the outputs you care about most greatly.

But Herbgrind is a heavyweight dynamic binary analysis using the Valgrind framework, so the slowdown it imposes on the runtime might be prohibitive for debugging some programs which take a while to compute their results.

I've tried out HerbGrind but it often gets stuck and reports an error E_STONED.

I believe that's a PEBCAK error

Thinking of all the people (some even actual friends) worked so long on numerical stability on scalar floating point operations, nearing a time when model-checking and proof tool are actually usable by the really-motivated developer ; thinking of them witnessing the arrival of GPUs and Tensor Cores, mixed-precision-everywhere and the tools now needing to scale so far it seems like starting from scratch...

Is it just me, or does the website not say which OSes it supports?

"Herbgrind currently primarily supports 64-bit Linux, but 32-bit might also work, and theres some work in progress to also support OSX."

https://herbgrind.ucsd.edu/getting-herbgrind.html

Reply on news.ycombinator.com