I thought Float Epsilon was the smallest possible number which could be represented by a float? This statement doesn’t seem to make sense...
The utility of this metric is that it tells you the best possible relative error, as you can never guarantee a result will be more correct to the true result (in the domain of mathematical real numbers as opposed to machine floating-point numbers) than half of machine epsilon.
This concept is often extended into discussions of "units in the last place" (ULPs), which is the magnitude of the value of last bit of the mantissa. So ulp(1.0) = ulp(1.5) = machine epsilon, ulp(2.0) = ulp(3.9) = 2 * machine epsilon, etc. A good floating-point library will document its accuracy in terms of ulps, so that we might say that the maximum error of the exp function is 1 ULP and that of pow might be 6 ULPs.
- If you add 1.0f to FLT_EPSILON then you get an exact result - If you add 1.0f to FLT_EPSILON * 0.5 then you get a number that cannot be represented and it will either round to 1.0 or 1.0+FLT_EPSILON - Similarly, if you add 0.5f to FLT_EPSILON * 0.25 then you get a number that cannot be represented and it will either round to 0.5 or to 0.5 + FLT_EPSILON * 0.5
With the default rounding mode that last calculation rounds to 0.5. There are a _lot_ of floats below FLT_EPSILON.