Hacker News Reader

Comment by 098799 | Hacker News Reader

Nothing weird about it. It should be obvious that subtracting two floats that are very close to each other results in a loss of numerical precision:

1.000000003456e0 - 1.000000002345e0 = 0.000000001111e0 = 1.111numericalnoise e-9

It's exactly the same issue here. `math.exp(1e-15)` is `1.000000000000001`. If you subtract 1, you get 1 significant digit and numerical noise.

View on HN

The weird thing here is that the only change is making the denominator ln(exp(x)) instead of x. Catastrophic cancellation is still happening in the numerator (it’s still exp(x)-1), and the denominator winds up being some really tiny number.

It’s just that, due to the quirks of the floating point calculations involved, the numerator and denominator wind up being nearly the same noisy approximation to x, whereas in the original calculation that wasn’t true.

That's not what the post says if I understand correctly - the post explains why in certain situations the "noise" disappears, and in other cases it doesn't.

See comparison between f and g functions.

I see! yes, the magic is you can cancel the noise by repeating it twice:

``` In [1]: math.exp(1e-15)-1 Out[1]: 1.1102230246251565e-15

In [2]: math.log(math.exp(1e-15)) Out[2]: 1.110223024625156e-15 ```

risky business though, I imagine it's implementation dependent

It’s not (or shouldn’t be), it’s simply a result of math, as the article explains in length.

Many libm implementations don't have an accurate `log` or `exp` routine, so there does exist a risk. (Of course, it's also true that many of them also special-case `log(x) ~= x - 1` and `exp(x) ~= x + 1` for small enough `x`.)

The math hinges on that there is the same error for exp(x) at both places. So as long as exp(x) is deterministic then this should be alright.

I don't know of any libm that have log or exp sufficiently inaccurate for this to break. Do you?

Indeed, any well-known enough libm wouldn't do that. But I can imagine some less-known libms with wild error bounds.

Agreed, this is risky business. The intermediate values still need to fit into floats and are still losing precision.

From the article:

    g(1e-9)  returns 1.0000000005,
    g(1e-12) returns 1.0000000000005,
    g(1e-15) returns 1.0000000000000004

but... g(1e-16) throws ZeroDivisionError: float division by zero.

Reply on news.ycombinator.com