Hacker News Reader

Comment by vortico | Hacker News Reader

Does anyone have code to reduce the search space by a few orders of magnitude to test functions with a float64 argument or two or three float32 arguments in a few seconds? Perhaps by skipping "boring" floats that aren't near endpoints that are unlikely to produce errors that more interesting floats do?

View on HN

You can't naively do it with the bits of a float, but for most purposes N-wise testing will get you a lot of coverage relative to random or manual cases for little expense. I've had success with it using a library to generate cases testing math kernels with many parameters.

https://en.m.wikipedia.org/wiki/All-pairs_testing

Yes, I've done this using libFuzzer. Supply custom mutation and crossover functions that generates interesting floating point numbers. Then let the coverage guided fuzzer exercise your code. https://rigtorp.se/fuzzing-floating-point-code/

That's essentially what fuzzy testing libraries do. You tell the library what types of arguments you want, write your test, and it puts in some known corner cases and a few random values that change each time you run the test (but are saved as new corner cases when the test fails in the better libraries).

Well, if the ceil function failed on 1.0f, maybe even the ordinary numbers are worth testing.

But... would randomness help? Take a representative sample of like 1/64th of the double space and you might run into most of the common problems.

all floats are equally boring.

No, -0.f is more interesting than 4.1045857301e23f. It's more interesting because it's more likely to fail a test where others don't.

That's a bold assumption: let's make sure it's correct by testing against all floats, instead of only the ones you consider interesting, in the process making any float as boring as any other float again.

Reply on news.ycombinator.com