What, if anything, is epsilon? (2014) [pdf] | Hacker News Reader

What, if anything, is epsilon? (2014) [pdf]

tom7.org

53 pointsnotmysql_17 comments

> JavaScript programmers have the highest tolerance for error of all languages tested, with over 1,000 using epsilon of 0.5 or higher.

JavaScript actually defines "Number.EPSILON" as being "the difference between 1 and the smallest floating point number greater than 1." https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Do yourself the favor of reading the plots. They are the high point of this “paper.”

I started to get suspicious after the "horizontal axis title" )

Isn't it just a small number to prevent divide by zero errors?

Not exactly. In math, epsilon is basically "it's not zero, except when it is."

When you're dividing, then it's close to zero, but still qualitatively not zero. But when you're multiplying or adding, then we say it's close enough that we'll treat it as exactly zero. So it's not a specific number, it's something where we can handwave its qualitative size depending on what's convenient.

Limits are a formalization of this concept. It's not a specific value, it's very much a variable that "approaches" zero. This establishes rules about how much bullshit you can actually do it. It's not zero, you do some arithmetic, then it becomes zero and you do the rest of the arithmetic.

But when you're writing a program, epsilon is indeed often just a small number. The trick is choosing the right "small."

> In math, epsilon is basically "it's not zero, except when it is."

In the math I know, it’s not necessarily small, but it is larger than zero. https://mathworld.wolfram.com/Epsilon-DeltaDefinition.html:

“An epsilon-delta definition is a mathematical definition in which a statement on a real function of one variable f having, for example, the form "for all neighborhoods U of y₀ there is a neighborhood V of x₀ such that, whenever x in V, then f(x) in U" is rephrased as "for all ε > 0 there is δ > 0 such that, whenever 0 < |x - x₀| < δ, then |f(x) - y₀| < ε.”

(Also used in proofs. https://mathworld.wolfram.com/Epsilon-DeltaProof.html)

https://en.m.wikipedia.org/wiki/Limit_of_a_function#%7F'%22%...:

“For every real ε > 0, there exists a real δ > 0 such that for all real x, 0 < |x − p| < δ implies |f(x) − L| < ε.”

> it's something where we can handwave its qualitative size depending on what's convenient.

Aha! You’re doing physics, not math. Mathematicians only handwave when they can prove it doesn’t affect results ;-)

The paper, on the epsilon-delta: > This has nothing to do with the subject of the paper

There is subtlety in selecting that "small number". 0.000000001 may look like a "small number", but if you're doing calculations in the 1e-20 range, suddenly it isn't small anymore. Small is relative to the task at hand. Simple uses of floats in known situations can get away with a hard-coded epsilon constant, and it may not even much matter what the number is, but as your usage becomes more sophisticated and more important, proper selection of epsilon values becomes non-trivial.

If you want to not divide by zero, you can just check for zero. You want to use some epsilon if you want to compare floating point numbers, 0.1 + 0.2 == 0.3 for example is false and not what you wanted most of the time. To do this properly you calculate the difference (0.1 + 0.2) - 0.3 and check if the absolute value of the difference is smaller than some tolerance, your epsilon.

To oversimplify, epsilon is “the smallest number that still makes a difference in the given context”.

In actual code, you will see it used with wide safety margins, though.

I thought this was a real paper at first.

Are you sure it's not? I'm uncertain.

Searching for the Association for Computational Heresy leads me to https://sigbovik.org/ which appears to be a real conference in the same way that extreme ironing is a real sport.

As a CMU alumnus, I resemble that remark. Of course it’s a real conference. It has a review committee and proceedings and LaTeX and everything!

It's a real paper - go look at the other things tom7 has done!

Nice graphics

Lolz. Delightful!

Reply on news.ycombinator.com