0.30000000000000004 | Hacker News Reader

0.30000000000000004

0.30000000000000004.com

533 pointscocoflunchy241 comments

Just adding 0.1 to 0.2 in the language of your choice can be misleading because printing the number might just cut off the digits at some point.

Here's an example using perl5:

perl -e 'print 0.1+0.2' yields '0.3', which looks OK, however

perl -e 'print 0.1+0.2-0.3' reveals '5.55111512312578e-17'

So, it's better to print the result of 0.1+0.2-0.3

For what its worth, perl6 passes this test with flying colours.

> printing the number might just cut off the digits at some point.

I'm not sure, but I think it's not really about cutting off the digits (although that's the effect it has). At least with python, I think what happens is:

* repr gives you the decimal number with the exact value of the floating-point number, i.e. 0.30000000000000004

* print gives you the least-precise decimal number whose closest floating-point representation is equal to the floating-point number

That is, there are (infinitely) many decimal numbers such that the closest floating-point representation has the value 0.30000000000000004. One of these decimal numbers is 0.3, and of all such decimal numbers, this has the fewest digits. So that's the one that gets printed.

Someone pointed out that this isn't correct, and then deleted their comment. (?)

I now think the behiviour I ascribed to print, is actually what repr does. I'm not sure about print, but I guess it is just rounding to a certain number of places.

    >>> from decimal import Decimal

    >>> Decimal(0.1+0.2)
    Decimal('0.3000000000000000444089209850062616169452667236328125')

    >>> Decimal(0.3)
    Decimal('0.299999999999999988897769753748434595763683319091796875')

    >>> Decimal(0.300000000000000000000000000001)
    Decimal('0.299999999999999988897769753748434595763683319091796875')

    >>> print 0.300000000000000000000000000001
    0.3

    >>> repr(0.300000000000000000000000000001)
    '0.3'

    >>> D(0.30000000000000004)
    Decimal('0.3000000000000000444089209850062616169452667236328125')

    >>> repr(0.299999999999999988897769753748434595763683319091796875)
    '0.3'

    >>> repr(0.30000000000000004)
    '0.30000000000000004'

    >>> print 0.30000000000000004
    0.3

you are using Decimal "wrong". If you want it to handle exact values, you need to pass a string with the decimal interpretation

    Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from decimal import Decimal
    >>> Decimal('0.1') + Decimal('0.2') - Decimal('0.3')
    Decimal('0.0')
    >>> Decimal(0.1) + Decimal(0.2) - Decimal(0.3)
    Decimal('2.775557561565156540423631668E-17')
    >>> Decimal(0.1) == Decimal('0.1')
    False

In this case, that was deliberate. I was using it to show the exact decimal value of the floating point approximations to decimal numbers, not to do decimal arithmetic.

There's some discussion here: https://docs.python.org/3.1/tutorial/floatingpoint.html. Basically: eval(repr(x)) == x is guaranteed, and some attempt is made to give the shortest such string representation for Python 3.1+ (and 2.7?), but it's not guaranteed on all platforms.

It also matters whether you're using Python 2 or 3. This is one of the things they changed in 3:

    >>> print(0.1 + 0.2)
    0.30000000000000004

I believe this is relevant: https://bugs.python.org/issue1580

> For what its worth, perl6 passes this test with flying colours.

Do we really expect calculations on floating point numbers to be the same as calcualtions with real numbers?

The only problem I see is that often the floating point format is not specified properly. When one writes "0.3", what does it actually mean? Does the format the compiler uses internally conform to IEEE 754? Is it 32bit, 64bit, 80bit or something else?

The only problem I see is that often the floating point format is not specified properly. When one writes "0.3", what does it actually mean?

It's perfectly well-specified. If you are using double-precision floating point, "0.3" means 0x3fd3333333333333 (as bits), or 0.299999999999999988897769753748434595763683319091796875 (as an exact number: the closest representable number to 0.3).

When your compiler does its internal calculations in double precision float you are right.

But why should it do so? Do all languages specify the details of how floating point input is handled by the compiler/interpreter?

Most languages do (though implementation bugs are common—plenty of languages where 64-bit precision is required actually end up being implemented in such a way that 80-bit precision is used on x86).

That said, a lot of low-level languages don't: C doesn't guarantee this, for example. C doesn't even mandate IEEE 754, and it merely requires that "float" is a subset of the values of "double" which is itself a subset of the values of "long double".

I think so, yes.

Perl 6 doesn't use IEEE 754 by default at all. It internally stores decimal literals as rational numbers, with numerator and denominator in separate fields, and it only resolves back to a decimal number when you convert it to a string.

In other words, there are about 5648798746465 possible interpretations of the "(\+|-|)[0-9]+.[0-9]+" format, and the problem is with mismatched assumptions ("of course I'm using the One Correct Interpretation, like everyone else!").

> Just adding 0.1 to 0.2 in the language of your choice can be misleading because printing the number might just cut off the digits at some point.

It's amazing that someone went to the trouble to register a custom domain for this and put up a nicely formatted page, yet doesn't explain the above: that it's not (just) a matter of representation, but printing precision.

What does it mean to "pass" this test? Just that the software is massaging its output to match human expectations rather than giving the most accurate (if not the prettiest) answer?

> For what its worth, perl6 passes this test with flying colours.

This is because Perl 6 decimal literals are stored in a built-in rational type called Rat, which internally stores numbers as ratios between integers. So that 0.1, 0.2, 0.3 are internally stored as the tuples (1,10), (2,10), and (3,10), and the decimal value you see is just the string representation and not the actual internal value.

If you want your decimal literal to be floating-point (called Num in Perl 6), you want to add e0 to the end.

To wit:

perl6 -e 'say 0.1+0.2-0.3' reveals '0'

perl6 -e 'say 0.1e0+0.2e0-0.3e0' reveals '5.55111512312578e-17'

That's interesting, I had no idea. There's probably a way more detailed Rosetta Code style project that could cover all these nuances, but for a single page demo, there's only so much detail I can get into before it would be more confusing than helpful.

If you'd like to flesh out the perl section a bit though, maybe include the differences between 5 and 6, I'd be happy to merge a pull request.

You've had a perl6 pull request outstanding for a while. The differences are far too big to list simply though. Ultimately Perl6 uses Rationals by default, not Floats.

Those 0.1, 0.2, and 0.3 numbers do not have any floating point representation at all. Computers just can not use them.

Of course they do, just not a binary floating point representation. They can be represented with decimal floating point just fine and there are plenty of programs and libraries that can do this.

It's very commonly used in finance, for example.

I thought finance worked using integers internally for this exact reason.

"Finance" uses lots of different things. Sometimes it uses symbolic math; sometimes it uses binary floating point, sometimes it uses decimal floating point, sometimes it uses arbitrary precision decimals or rationals, sometimes it uses integers in the smallest unit of concern for the application.

Finance is a big domain.

There are several ways to do it.

One it so use fixed point decimal with a 1/100 scaling instead of floating point. So yes, integers and then you shift the decimal point when you output.

But you have to watch out for interest calculations, you might earn a fraction of a penny on a daily basis but over a year it adds up and you can't throw away that accuracy.

So you might do all math as floating point, but with pennies instead of dollars.

Or you can do Binary Coded Decimal, which stores each group of decimal number after the decimal point exactly, in many ways it similar to fixed point decimal, since it is a fixed point.

Do they do USD interest calculations in something smaller than 1/10000 (decimill I think)?

The issue with BCD would be data store support and CPU support right? Do CPUs support doing math directly with BCD values and does your DB support doing math directly with BCD values?

In any case, using floating point for any currency calculations is a huge no-no, even though a lot of things do it anyway.

I work with Magento daily and all their currency math uses PHP floats and the rounding errors drive me bonkers.

> Do CPUs support doing math directly with BCD values

Perhaps surprisingly to some, x86 has instructions for BCD manipulation at the CPU level. Legacy.

In IBM POWER 6 this returned as an optimisation in hardware.

https://books.google.ch/books?id=SPXHAgAAQBAJ&pg=PA44&lpg=PA...

Maybe for accounts bookkeeping, but I doubt it is feasible, or even desirable, for financial statistical work.

I figured they used Binary Coded Decimal [0] when they need to support decimals.

[0] : https://en.wikipedia.org/wiki/Binary-coded_decimal

Well, I know from doing payment gateway integration that many of them use integers for currency amounts. They are the smallest currency unit and for display you then convert based on the currency specific precision value.

Accounting systems I've worked with do most of their logic in a RDBM and store monetary types using a 'decimal' type.

But these numbers are perfectly computable! Sure they cannot be represented in the commonly used binary implementation of floating point numbers, but that doesn't mean computers cannot operate with these numbers at all.

Well, they can't be specified as IEEE 754 floats. A computer can still use those numbers in the form of decimal fixed-point or numerator-denominator formats.

Since 2008 IEEE-754 also supports decimal floating-point: https://en.wikipedia.org/wiki/IEEE_754#Basic_and_interchange...

>There are three binary floating-point basic formats (encoded with 32, 64 or 128 bits) and two decimal floating-point basic formats (encoded with 64 or 128 bits)

Just say (0.1+0.2)*10-3. That works everywhere no matter the amount of rounding.

You've just invented fixed point numbers.

I'm not sure I follow? The point here was to show that a language can't represent the decimal fractions exactly.

But they can. Many languages have decimal and rational types.

"Computers can only natively store integers, so they need some way of representing decimal numbers"

Technically, [digital] computers only natively "store" high and low voltages. The interpretation of a sequence of those signals as "integers" is entirely up to you.

In all seriousness, representing integers is not a given either (e.g. two's complement vs one's complement, BCD, bignum, sparse integers, etc.). If you are going down in abstraction levels to describe floating point representation, you should not just call integers "native" blindly.

Furthermore, floating point is not the only way to represent computation involving real values. It is indeed possible to represent computation on real values symbolically in many cases and lazily compute arbitrary precision results for output-printing purposes.

Technically, much storage does not need voltage being high or low either. Pits burned with lasers, and magnetic changes are also common ways to store stuff that don't involve voltage in the stored state.

Here you can see python has a fractions.Fraction type which can be used for rational number arithmetic.

  >>> from fractions import Fraction
  >>> Fraction("0.1") + Fraction("0.2")
  Fraction(3, 10)

  >>> repr(float(Fraction("0.1") + Fraction("0.2")))
  '0.3'

Here we see the floating point behaviour.

  >>> 0.1 + 0.2
  0.30000000000000004

Here we see that python has a nice behaviour for when printing floats.

  >>> print(0.1 + 0.2)
  0.3

Here is a common error where people feed the input of Fraction (and Decimal) with a float. Even though you use the Fraction here it still carries the floating point error through.

  >>> Fraction(0.1) + Fraction(0.2)
  Fraction(10808639105689191, 36028797018963968)
  >>> float(Fraction(0.1) + Fraction(0.2))
  0.30000000000000004

Here is the Decimal type which does not have this float problem. Some people have argued (including myself) that Decimal should be used by python instead of float. Since this is much friendlier to people doing things like adding these numbers together.

  >>> from decimal import Decimal
  >>> Decimal("0.1") + Decimal("0.2")
  Decimal('0.3')
  >>> float(Decimal("0.1") + Decimal("0.2"))
  0.3

This is a good case for macros in Python, actually. Fraction('0.1') is usually what you want, so Fraction(0.1) should mean just that and if you do want the current behaviour, Fraction(float(0.1)) would do it. Same for Fraction(17/3) for example.

Here is the Decimal type which does not have this float problem.

Are you sure?

  >>> Decimal(0.1)
  Decimal('0.1000000000000000055511151231257827021181583404541015625')

use Decimal('0.1'), otherwise you will first generate a floating point number (with accuracy loss) and the convert it to a decimal.

It does not if you pass the string representation of the float you're working on. Like BigDecimal in Java.

I think the words we're really looking for are "discrete" vs "continuous". Any discrete values will be bounded by some sort of precision limit.

That depends on the number you want to represent. Some values can be represented exactly, others with varying degrees of precision because of the difference between what is actually stored and what you wanted to store. This will collapse a number of values into the same precision 'bucket'.

What is continuous or precise depends on the base you use. I can't think of any example in base 2, but I believe in base 3 you could represent numbers that have infinite digits in base 10.

If I'm not mistaken, 0.3333... would be 0.1 and 0.6666... would be 0.2. It probably gets nasty for a lot of simple base 10 numbers, so it isn't a good solution.

Right you are. One half is 0.5 in decimal or 0.1 in binary but 0.11111... in base 3

> Technically, [digital] computers only natively "store" high and low voltages. The interpretation of a sequence of those signals as "integers" is entirely up to you.

And then there's MLC NAND flash which stores bit sequences as different voltage levels within the same cell.

That's valid, but I was going for an ELI5 style explanation of a concept that surprised and confused me at the time. If you want to provide a better one, I'm happy to merge any pull requests.

when you aim for reddit style stuff like "ELI5", don't be surprised if HN tears you up. reddit likes things to be cool and cliquey. we like things to be correct.

also we don't have 5 year olds.

Those languages which print 0.30000000000000004 are doing something which is silly as a default: they are printing 17 decimal digits, out of a type that only supports 15.

   $ txr -p '(+ .1 .2)'
   0.3

Why is that? Here, the underlying type is the C double type. The printed representation is obtained according to a default precision. That default is taken from the C constant DBL_DIG. For an IEE754 double, that constant is 15.

It is misleading to print more decimal digits out of a double than DBL_DIG; that is the constant which tells you how many decimal digits of precision double can reliably store. Thus I chose that constant as the default printing precision for floats: to give the programmer/user the maximum realistic precision. That is to say, print the decimal digits which are plausibly there, and not any fictional ones.

17 decimal digits requires a 57 bit mantissa -- ceil(log 10 / log 2) * 17). The 64 bit double type has only 52. So if you print 17, you're "making shit up".

This is a common but fundamentally wrong view of floating-point numbers. You are not "making shit up" by printing more that 15 digits of a floating-point number. Floats, doubles, etc. have precise values. As I said in another comment, the double represented by the literal `0.1` is precisely 0.1000000000000000055511151231257827021181583404541015625; similarly, `0.2` is precisely 0.200000000000000011102230246251565404236316680908203125 and `0.3` is precisely 0.299999999999999988897769753748434595763683319091796875. From these exact values, you can see why `0.1 + 0.2 != 0.3` – the left hand side is greater than 0.3 while the right hand side is smaller than 0.3. Printing only 15 digits is not doing programmers any favors: even though `0.1 + 0.2` will print as "0.3", that just makes the lie even worse – they will be even more confused when `0.1 + 0.2 != 0.3` even though both values print identically.

Your claim is fallacious, because a floating-point value denotes a range of the real number line, whereas you're insisting that, no, it stands for its literal value: the absolutely precise rational number that lies at the center of that range.

That is no more true than my current body temperature being precisely 36 4/10 degrees Celsius because the thermometer reads 36.1.

All numbers in that range alias to the same double, yet differ wildly in their decimal digits beyond the fifteenth. That "long tail" of digits is a meaningless residue arising from the arbitrary difference between the chosen center-of-range point and the actual number it approximates.

> Printing only 15 digits is not doing programmers any favors

Note that the ISO C function printf uses 6 digits of precision under the %g conversion specifier by default. So a team of experts can find it justifiable to severly truncate precision on printing. I find that justifiable also, like this: printing is not only for constants, and trivial expressions like 0.1 + 0.2, but for the results of complex calculations, which accumulate significant rounding errors. Six digits of precision is prudent: for many complex calculations, it will avoid misleading the user with too much precision. Fifteen decimal digits will be wrong after just a few operations; there is hardly any "headroom" to absorb error.

> just makes the lie even worse

If merely neglecting to reveal some aspect of the stark truth is tantamount to lying, then you're also lying when you pull a number with 54 significant decimal figures out of a 64 bit double. Revealing some truth without an explanation can also be construed as "lying", as in "[N]one of woman born shall harm Macbeth".

I personally find it convenient that when I bang the token 0.3 into my REPL, it comes back with a tidy 0.3. I know that there are several values of double which will print as 0.3, and don't compare with exact equality except in special circumstances (like when deliberately using integral values not far from zero).

> Your claim is fallacious, because a floating-point value denotes a range of the real number line, whereas you're insisting that, no, it stands for its literal value: the absolutely precise rational number that lies at the center of that range

See Misconception #2 of http://lipforge.ens-lyon.fr/www/crlibm/documents/cern.pdf.

Floating point numbers are _not_ intervals. If you read carefully any formal definition of floating point numbers (The IEEE standards, TAoCP Chapter 4, or Higham's _Accuracy and Stability of Numerical Algorithms_, to name just three possible references), you will see that floating point numbers by definition form an exact rational subset of the extended real line.

> All numbers in that range alias to the same double, yet differ wildly in their decimal digits beyond the fifteenth. That "long tail" of digits is a meaningless residue arising from the arbitrary difference between the chosen center-of-range point and the actual number it approximates.

See Misconceptions #1 and #2 on Kahan's list (https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf).

You are not being clear about the set of real numbers a user may input and their ultimate representation as a floating point number. While it's true that many real numbers round to the same floating point number _f_, that's not the same as then saying that _f_ carries that interval around with it. The latter is false, since the intervals do _not_ propagate in floating point arithmetic. It's also false that the floating point number _f_ is the midpoint of the set of numbers that round to _f_; the precise set depends on the rounding mode and the granularity of the set of floating point numbers around _f_.

I didn't say they were intervals (I'm well aware of interval representations for numbers, which single point rationals are not).

> that floating point numbers by definition form an exact rational subset of the extended real line.

Sure, that's what they form; it's not what they (usually) denote.

> It's also false that the floating point number _f_ is the midpoint of the set of numbers that round to _f_;

It is close enough to being the case for my purpose in the grandparent article. Whether the cluster of numbers is lopsided one way or the other doesn't really detract from my point.

Are they specific values or are they intervals? You can't have it both ways.

Your view has been repeatedly refuted by Kahan and others. See page 25 of [1], page 1 of [2], page 22 of [3] for just a few examples. Conceptually, the reason the interval view is wrong is that if floating point values represented intervals, then their arithmetic would be incorrect. For example, the sum of an interval of width w1 and an interval of width w2 should be an interval of width w1 + w2. That, of course, is not how floats work at all. The correct view is that floats represent exact rational values and operations on them, instead of being the true mathematical ones – under which floats are not closed – compute the closest float to the true result of each operation.

[1] https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf

[2] https://math.mit.edu/~stevenj/18.335/fp-myths.pdf

[3] http://lyoncalcul.univ-lyon1.fr/IMG/pdf/FloatingPoint.pdf

Or, for that matter, 1.0 + 2.0 != 3.0

1.0, 2.0, and 3.0 are all represented perfectly and precisely in IEE754 floating point, as are all integers up through 2^24 (for 32-bit floats) or 2^53 (for 64-bit doubles).

So 1.0 + 2.0 == 3.0.

How is that? I assume the mantissa is chosen to represent the full number, but how is the exponent set so that the granularity is of exactly 1.0?

And of course 1x10^7 + 2x10^7 != 3x10^7, right?

2^0 is 1, so if the exponent is 0 then the mantissa times one is your value, so every integer has an exact representation.

If the exponent is 1, then the mantissa is multiplied by 2 to get the actual value. Put another way, the mantissa is the value divided by 2 and rounded, i.e. shifted right one bit.

> 1x10^7 + 2x10^7 != 3x10^7, right?

This particular case is interesting.

If we're talking about 64-bit doubles, then you have 53 bits of precision for integers. All those values are well within that range, so this arithmetic and comparison are done with precise integer values, and the expression will compare equal.

If we're talking about 32-bit floats, the range of precise integer values is -16,777,216 through 16,777,216, or -2^24 through 2^24.

1x10^7 is 10,000,000, within that range, but the other two values are outside the range.

So you might expect that != would be the answer here.

But if you test it, that isn't the case: the expression compares equal!

The reason: although 20,000,000 and 30,000,000 are outside the range where every integer has a precise representation, they are within the range where every even integer is precise: -33,554,432 to 33,554,432. Values within this range but outside the range of precise integers are rounded to a multiple of 2.

Similarly in the range -67,108,864 to 67,108,864, all integers which are a multiple of 4 are represented precisely.

Basically, as you go outside the range of precise integers, values get rounded to a multiple of 2, 4, 8, etc. as required.

When the mantissa overflows the available precision, it is shifted to the right enough so that it fits without losing the most significant bits. Instead, the least significant bits are discarded, and the exponent is incremented by the number of discarded bits.

Of course you could choose other values where the rounding doesn't work out in your favor, and then you'd get the unequal comparison you expect. A simple example:

33333333.0f + 1.0f != 33333334.0f

Here, the value 33,333,333 is rounded down to 33,333,332. Add 1 to that and you get 33,333,333, which is again rounded down to 33,333,332.

33,333,334 is represented precisely, so the comparison fails.

https://en.wikipedia.org/wiki/Single-precision_floating-poin...

Because integers are an integer multiple of a clean power of 2 (2^0 == 1)

Stratoscope was already done a superb job explaining. But just to show why my question was not that dumb...

Let's say you want to represent "two-million" in 32bit float:

Your answer is: 0x1e8480 * 2^0.

However, I was expecting something like: 0x1.e848 * 2^X. But then "125-thousand" is 0x1.e848 * 2^Y, so how do you standardize what is the correct exponent for integers, X or Y? And how do you know there is not a pair (M,Z) != (0x1.e848, X) such that M * 2^Z is also two-million?

the fraction of a floating point is always "starts with on invisible 1.xxxx" so that forces unique representations for your exponent for any given number. Is it not clear why this is the case?

let's take a simple case where you have one bit of fraction.

   exponent 0, 2^0 = 1, fraction (1)0 = 1.0b = 1

   exponent 0, 2^0 = 1, fraction (1)1 = 1.1b = 1.5

   exponent 1, 2^1 = 2, fraction (1)0 = 10b = 2

   exponent 1, 2^1 = 2, fraction (1)1 = 11b = 3

if we have more fraction bits it looks like this:

   exponent 0, 2^0 = 1, fraction (1)00 = 1.00b = 1

   exponent 0, 2^0 = 1, fraction (1)01 = 1.01b = 1.25

   exponent 0, 2^0 = 1, fraction (1)10 = 1.10b = 1.5

   exponent 0, 2^0 = 1, fraction (1)11 = 1.11b = 1.75

   exponent 1, 2^1 = 2, fraction (1)00 = 10.0b = 2

   exponent 1, 2^1 = 2, fraction (1)01 = 10.1b = 2.5

   exponent 1, 2^1 = 2, fraction (1)10 = 11.0b = 3

   exponent 1, 2^1 = 2, fraction (1)11 = 11.1b = 3.5

So for any given exponent multiplication with a fraction with an invisible bit creates products that are hemmed between 1.0x2^n and 2.0*2^n so there's no collisions between any pair (e,f) in your representation.

Deleted comment

You quoted out of context. The full quote was:

> the double represented by the literal `0.1` is precisely

Which is completely true. 2/3 represented in a format that has 5 decimal digits is precisely 0.66667 as well.

I don't think you understand what he is saying here. The value is stored in memory as a precise number.

> It is misleading to print more decimal digits out of a double than DBL_DIG

If you want round-trip conversion for all `double` numbers, i.e. from `double` to text and back, you need 17 digits of precision (%.17g).

Likewise, if you want round-trip conversion for all `float` number, you need 9 digits (%.9g).

The reason for this is that even though these types can really only reliably give 15 and 6 digits of precision, some of the numbers they can represent are closer to each other than that.

In C++, you can get the former limit using std::numeric_limits<double>::digits10, and the latter using std::numeric_limits<double>::max_digits10.

http://en.cppreference.com/w/cpp/types/numeric_limits/max_di...

> In C++, you can get the former limit using std::numeric_limits<double>::digits10, and the latter using std::numeric_limits<double>::max_digits10.

Also, via GCC specific constants:

  $ echo | gcc -E -dM - | grep -E '(FLT|DBL).*DIG'
  #define __DBL_DIG__ 15
  #define __FLT_MANT_DIG__ 24
  #define __LDBL_MANT_DIG__ 64
  #define __FLT_DIG__ 6
  #define __DBL_MANT_DIG__ 53
  #define __DBL_DECIMAL_DIG__ 17
  #define __LDBL_DIG__ 18
  #define __FLT_DECIMAL_DIG__ 9

__DBL_DIG__ versus __DBL_DECIMAL_DIG__. Useful to know.

__DBL_DECIMAL_DIG__ corresponds to DBL_DECIMAL_DIG which was added to C in C11.

In C90 or C99 we can do some hack like:

  #if DBL_DECIMAL_DIG
  use this
  #elif __DBL_DECIMAL_DIG__
  use this from gcc.
  #else
  fall back on DBL_DIG + 2
  #endif

> If you want round-trip conversion for all `double` numbers, i.e. from `double` to text and back

I don't. Rather, I want round-trip conversion of all 15-digit-precision decimal numbers to the machine and back. If the machine calculates some numbers which are different from some of these but alias to them textually, I don't care.

Someone else might want the decimal text representation to provide bit-exact storage semantics for arbitrary doubles.

For that reason it might be a better design choice to obtain the default printing precision from a special variable, which itself defaults to 15, rather than a hard-coded default.

The downside to printing only 15 is that you then have numbers that print the same but compare as not equal.

If you want that, that comes at the cost of printing 0.99999999999999989 as "1".

Well, yes; 0.99999999999999989 isn't a 15 digit number, so it lies outside of the requirement.

> that only supports 15.

it supports somewhere between 15 and 16, actually. There are 52 bits of precision in a double's mantissa[1]. This gives you log_10(2^52) = 52*log(2)/log(10) ~ 15.654 digits of precision, that is, the 16th decimal place is accurate about 65% of the time.

You can also exactly represent in decimal the binary value stored, because as long as you have 52 decimal places, you have enough 2's in the denominator for all of the bits that you can represent (each 10 in the denominator pairs up with each 2 from each bit).

[1] Plus an implicit bit, but not in the fractional part of the mantissa.

Better all-round solution:

  $ ./txr
  This is the TXR Lisp interactive listener of TXR 123.
  Use the :quit command or type Ctrl-D on empty line to exit.
  1> (tostring (+ 0.1 0.2))
  "0.3"
  2> (let ((*flo-print-precision* flo-max-dig)) (tostring (+ 0.1 0.2)))
  "0.30000000000000004"
  3> *flo-print-precision*
  15
  4> flo-dig
  15
  5> flo-max-dig
  17

- 15 by default is reasonable for everyday programming; we don't need results like 0.30000....4 popping up in our faces.

- provide the constant which indicates how many decimal digits are needed to preserve the binary value exactly: this is needed for faithful storage and communication --- we wouldn't want a Sexp-based RPC call to behave differently from a local computation.

- provide the printing precision default as a special variable which can be overridden over a dynamic scope.

   15 by default is reasonable for everyday programming; 
   we don't need results like 0.30000....4 popping up in
   our faces.

That just changes the problem. You can't get rid of rounding error by representing decimal numbers in binary. 0.3 - 0.2 - 0.1 still won't show up as zero even if you only display 15 decimal digits.

I'm not about to change the floating-point type to a non-binary support.

I think that 15 digits is the "right" value for ordinary display purposes. If you present a literal value of up to 15 digits to the machine, it gets converted to some approximation which, when printed back with 15 digits of precision, gives you the same digits.

By using 15 we avoid this:

  1> (let ((*flo-print-precision* flo-max-dig)) (tostring 0.3))
  "0.29999999999999999"
  2> (tostring 0.3)
  "0.3"

I have no "rounding error" because I haven't performed a calculation. I understand that when I enter 0.3 into the machine, the floating-point value isn't exactly 0.3. I have an error, because the representation can only provide a close approximation of 0.3.

Yet I know that this approximation is so close to 0.3 that I would like it printed that way. Now of course it will be printed that way if I round to, say, four places. But that will throw away precision for other numbers. If I round to DBL_DIG digits (15), I get the maximum precision, without having my input numbers altered into something else.

It is useful and elegant for the default printing precision to be as high as possible, yet such that numbers that I enter into a REPL are echoed back at me pretty as I entered them, modulo choice of notation.

No, of course, DBL_DIG doesn't guarantee that I can do any sequence of calculations, like 0.3 - 0.2 - 0.1 and still get back a result which is exact to DBL_DIG digits! There is no lower bound on how inaccurate a floating-point calculation can be, if it is carried through enough iterations!

Nitpick: the 64 bit double type has a mantissa of 52 bits, but there's a 1 implied in front, so the precision is 53 bits.

Implied fixed material doesn't contribute precision. Otherwise we could pretend that there are twenty-four implied 1's and the precision is 76 bits.

Because the implied 1 cannot be zero, it isn't a bit; it doesn't carry information.

For instance if we define a four bit binary number type which has an implied 1, it can only represent values from 16 to 31. That's four bits of precision; the range is just displaced.

The information of the first bit is implied via the exponent.

Twenty four implied 1's would give you a precision of 76 bits, but you would restrict yourself to a very small subset of all possible numbers with that precision.

EDIT: Let's put it another way. Assume you have 3 decimal digits of precision for a float. The first digit cannot be zero. Would you then claim that you don't have 3 digits of precision, but rather log(10x10x9,10) = 2.95.., because the first digit only carries 0.95.. digits of information?

Strictly speaking, the 2.95 is right; but there is justification in rounding it up to 3. There is much less justification for rounding up 2.0 to 3.

> Those languages which print 0.30000000000000004 are doing something which is silly as a default

Yes, but the silly thing isn't what you say.

The silly thing is having binary floating points as the default representation for literals expressed as precise decimals.

> binary floating points as the default representation for literals expressed as precise decimals.

For better or worse, it is a widespread practice appearing in numerous programming languages, including some widely used popular ones.

To conform with the practice, if you want, in your language, a way to use decimals to write exact fractional numbers, it's better to have an explicit notation for that like, say, 10.5r (rational number, another spelling for 21/2). Or hey: 10R5. Electrical Engineers will love you to death. :)

> For better or worse, it is a widespread practice appearing in numerous programming languages, including some widely used popular ones.

Well, yeah, I've been programming since the 1980s. I know that.

> To conform with the practice, if you want, in your language, a way to use decimals to write exact fractional numbers, it's better to have an explicit notation for that

Sure, given that your goal is to conform with that practice. I'm just saying its a bad practice that reflects a generally premature optimization that often harms correctness, so we shouldn't conform to it in new languages.

> [implicitly approximating numbers like `0.1`] is a widespread practice

Sure, but the key point is that it leads to approximation that may cause problems.

It makes much more sense to interpret `0.1` as meaning exactly that number. Same with, say, `29/7`. If someone wants approximation, then have an explicit notation for that, like, say, `1.23e45`.

Defaulting to a Binary Encoded Decimal is also silly because its not as performant and loses compactness. Fundamentally. (Having a composite base comes with problems). What is your suggestion, having a default floating point have a Binary or hexadecimal representation for literals? Let's say I want to input a planet's mass as 6e24 kg - quick what is that in your literals?

It doesn't have to lose performance; it's just the default representation. For instance, if you have

   a + 0.1

where a is of binary floating point type, the 0.1 constant could denote an exact number. Because of the addition with a float, it gets coerced to the float's type. But that can be done at compile time. So effectively 0.1 is like a binary floating point constant in that situation.

But in another situation it remains exact, such as:

  0.1 + 0.2

this gets folded by the compiler at compile time, and uses exact math producing an exact 0.3.

Exactness is just the default.

If your language has rationals expressed as digs/digs, you can imagine that those are used in their place:

  a + 1/10;

  1/10 + 2/10;

In other words, suppose that 0.1 is just another "spelling" for 1/10.

This will be no less performant than using integer constants in floating contexts, e.g:

   double x;

   // ...

   x += 1;  // not slower than x += 1.0!

Here's a question: How do you code an arbitrary-precision-result (i.e. not an interpolated pade approximant) BCD log10 function?

The algorithm in binary is simple, because of special properties of two. It is harder for base-p, and base-composite, is very tough.

> Defaulting to a Binary Encoded Decimal is also silly because its not as performant and loses compactness.

Defaulting to correctness makes a lot more sense than often premature time and space optimizations by default.

> What is your suggestion, having a default floating point have a Binary or hexadecimal representation for literals?

There's lots of good solutions (Scheme's, of having a numeric tower with literals using the minimal-scope exact representation -- e.g., integer if there is no fractional part, decimal if its a decimal literal, rational if it is rational literal not expressed as a decimal -- by default unless they have an modifier specifying intent to use an inexact representation) is the best.

the point is, there are still unrepresentables (1/3 e.g.) in binary encoded decimal. If it's all about literals, just make the user put the correct literal in, and make them understand that floats are imperfect anyways. If you're doing anything remotely useful with Binary Coded Decimal, probably about two or three operations in, you're going to lose resolution on your exactness. It doesn't buy you much. GIGO.

Your comment made me look up 'txr'. Looks like a wonderful tool.

Do you use that often for data munging tasks? How big is this tool in that area of work?

Since I'm the author, I absolutely use it as a matter of "eat your own dogfood" and because it is actually convenient.

TXR is not "big" in this area because it is very difficult to get people to notice new things like this.

It is being rapidly developed. OpenHub stats: https://www.openhub.net/p/txr

In the last half a year or so, I added major features such as: structs with good OOP support, an interactive REPL (based on "linenoise" with a lot of my hacks applied), and delimited continuations.

I just have to post this:

http://www.smbc-comics.com/?id=2999

So it seems SQL does not give access to the secret robot Internet...

I think this sort of captcha could actually work, hah.

For a short while but I'm sure that bot makers would use a decimal library if this became common.

For real fun, use Excel:

    =0.1+0.2-0.3        => 0

    =(0.1+0.2-0.3)      => 5.55112E-17

(inspired by http://www.cs.berkeley.edu/~wkahan/Mindless.pdf)

Oh Excel... I'd love to see that up on the site, mind sending me a pull request? https://github.com/erikwiffin/0.30000000000000004

Tried it in Numbers and they both say 0.0000000000000000555111512312578

EDIT: Numbers is http://www.apple.com/mac/numbers/

That... is not a known hardware floating point representation. There are way too many digits. This is a just-plain-buggy conversion routine I'm guessing? What is Numbers, and why isn't it using a proper arbitrary precision library if it doesn't want to use doubles?

Edit: sorry, my brain was still reading the leading 0.3 from the linked article. There is indeed nothing wrong with that representation except the nonstandard leading zeroes.

What? No, it's a perfectly valid floating point number, represented in a way that lay people might understand.

Those zeros are not part of the mantissa.

uh, perl gives exactly the same answer as Numbers

> perl -E 'say .1 + .2 - .3'

5.55111512312578e-17

And python actually gives one more digit than Numbers

> python -c 'print(repr(.1 +.2 -.3))'

5.551115123125783e-17

So, I'm not sure why you are claiming "there are way too many digits" and that Numbers is doing something wrong.

A double can hold 15-17 significant base 10 digits. The exponent can be +-10^308 (11 bits for the exponent, 52 bits for the fraction, 1 bit for the sign).

Yes yes yes, I know how to represent an IEEE value. It was an interpretation error: the linked article shows a zillion variants of numbers with a leading significant of 0.3 (i.e. all the leading zeros are significant) while the posted string above did not.

Postgres gives the same response.

# select 0.1::float + 0.2::float - 0.3::float; ---------------------- 5.55111512312578e-17

Apple's brand of Excel.

LibreOffice Calc shows both as 0.

Try changing the cell to show as many floating points as possible (in Numbers it was to set it to "Automatic").

Sill both zero (dot many zeroes).

I saw the same thing in Numbers until I expanded the field to a larger width, which revealed the non-zero digits at the very end of all the zeroes.

What Every Computer Scientist Should Know About Floating-Point Arithmetic: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...

Here is the part that made me understand it all: "Floating-point representations have a base (which is always assumed to be even) and a precision p. If = 10 and p = 3, then the number 0.1 is represented as 1.00 × 10-1. If = 2 and p = 24, then the decimal number 0.1 cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4."

In Haskell/GHC it depends on what representation you chose. Default is Double, which exhibits this problem. But if I force it to Float I get the 'correct' result. Or if I use the 'Scientific' data type (which has arbitrary precision). Just like 'Int' vs 'Integer', where the former is restricted by the machine's word size, while the later is unbounded.

    Prelude> 0.1 + 0.2 :: Float
    0.3
    Prelude> 0.1 + 0.2 :: Double
    0.30000000000000004
    Prelude> :m +Data.Scientific
    Prelude Data.Scientific> 0.1 + 0.2 :: Scientific
    0.3

So effectively, in Haskell, it's both and neither.

A Float will round, as expected.

A double will return the correct value as it, should.

Shouldn't the Scientific, if it has arbitrary precision, arrive at the correct value too?

Clearly the only logical conclusion is that the Glasgow compiler, at its roots, virtualizes the entire quantum mechanics model as a compilation step, enabling superposition-as-a-feature.

My futile attempts at humour aside, this is quite an interesting thing to know and exactly the kind of intricate little detail I was hoping for.

Tip of the hat, friend.

The Scientific type does arrive at the correct value. 0.3 is the correct answer, and Scientific gives it to you. I don't know why you would expect something else from a type which implements arbitrary precision arithmetic.

Its important to understand that 0.1 + 0.2 is actually 0.3, and Scientific is giving the correct answer.

The Haskell type annotations aren't casting values that are interpreted as binary floating point (as might be the case in some languages), it is instructing the compiler how to treat the literals. Scientific is arbitrary precision decimal, so it can exactly represent 0.1 and 0.2, and return the exact result 0.3, rather than representing 0.1 and 0.2 as binary floating point approximations, adding them, and then producing some decimal approximation of the binary floating point result when asked.

I'm not sure what was the intended message of the table of examples. It is possible in many languages to get both types of ASCII representations for floating point: a nice human-readable 0.3 and the technically correct 0.30000000000000004. But some examples seem artificially biased towards one or the other. Only for Python both variants are shown. And then some examples use decimal representations, which are not floating point at all.

For instance, in Perl "print 0.1+0.2" will give you "0.3" just like in Python, while the printf format shown here will give you the non-rounded representation. And there are of-course decimal and rational number packages for Perl as well.

Yep, it's better to print the result of 0.1+0.2-0.3

Not generally!

    $ perl -E 'say sprintf "%.64f", 0.1 + 0.2'
    0.3000000000000000000108420217248550443400745280086994171142578125
    $ perl -E 'say sprintf "%.64f", 0.1 + 0.2 - 0.3'
    0.0000000000000000000000000000000000000000000000000000000000000000

My Perl is configured with -Dusemorebits. https://metacpan.org/pod/distribution/perl/INSTALL#more-bits

So I have a question.

For those that arrive at the right answer, is it by mathematical correctness in the implementation or by rounding down?

i.e. Which are right and which are flukey in their wrongness?

Under any other circumstance I wouldn't care about this pointless detail, but seeing as how this is the top post on the front page, I'd like an excruciatingly detailed investigation please.

Don't make me not get round to doing it myself, sir.

In PHP's case, it converts the value to a printable (string) representation, and in so doing rounds off to a precision of 14 digits.

http://lxr.php.net/xref/PHP_5_4/Zend/zend_operators.c#2142

The "mathematically" correct answer is 0.30000000000000004 according to the IEEE-754 spec.

If you want PHP to print it like the other languages you have to set the output precision high enough. The default, as you mentioned, is 14 but if you set it to 17 then you get the 'correct' output.

https://3v4l.org/TRcJB

http://php.net/manual/en/ini.core.php#ini.precision

Another option is to use printf, as like other languages it can take a precision argument.

    printf('%.17f', 0.1 + 0.2); // 0.30000000000000004

Scheme and Common Lisp arrive at the right answer to (+ 1/10 2/10) by correctness. They have internal representations for rational numbers.

Read http://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/pr... and, if you still need more info, a few of the earlier papers it references.

All of them may be correct as typically, multiple 'answers' can be correct. One could argue that given a number, one should print the shortest string that round-trips to that number, but that may mean that one prints "0.3" while the 'real' value is closer to 0.30000000000000004 or 0.29999999999999998 (a representation that one of the algorithms in the above paper gives) or possibly even equal to it.

Also, some of them may be incorrect for backwards compatibility reasons. In the eyes of some, that makes them correct again.

And don't get round to not doing it yourself :-)

Go is actually a lot more confusing about this than this than this page makes clear.

a := .1; b := .2

fmt.Println(a+b) returns 0.30000000000000004

fmt.Println(.1+.2) returns 0.3

Even worse, this persists into comparisons:

c := .3

a+b == c returns false

.1+.2 == c returns true

It's not a particularly big issue, mostly, but I'm sure it's confused someone before now, and will do so again.

Demo code: http://play.golang.org/p/FOv0JQRQJN

One should never compare for equality on floats or doubles, with the possible exception of '0' depending on the context.

Don't be ridiculous. It's perfectly fine if you know what you're doing. For example, when you have a limited selection of floats (e.g. hashtable keys, or only numbers between 0 and 100, accurate to 1 decimal) - in these cases, strings would work just as well, but floats are more efficient (and compare correctly).

You've just redefined your problem to using floats as the underlying representation. In that case you don't even need floats...

In most other cases the correct way is to check if the absolute value of the difference between one value and another is smaller than some arbitrary cut-off, a small number typically denoted with 'epsilon'.

I don't need floats, but they're useful. E.g. for comparison, as I wrote above.

Until your code gets ported to a platform where float == double and it turns out the algo feeding your clever efficient comparison delivers slightly different values that previously collapsed into the same bucket. Relying on the particulars of an underlying representation requires deep knowledge of how those particulars are implemented (and I would happily agree with you that such knowledge is a requirement but more often than not when you come across a float comparison with equality you're looking at a bug. That you're the exception to the rule is fine with me, you seem to know what you're doing so don't take it personal).

The real problem is: most people don't have deep knowledge of how those details are implemented and take 'equality' for granted, especially if the visual representation of the numbers they are comparing tells them they should be equal which is not always the case.

The old adage goes: if you need floating point you don't understand your problem :)

The comment that sparked this sub-thread is a nice illustration of what can and does go wrong.

Yay that's weird. Where's the magic?

Go is specified to calculate constant expressions with higher precision (256 bits of mantissa). https://golang.org/ref/spec#Constants

Numeric literals in Go are untyped until they need to be.

https://blog.golang.org/constants

try " fmt.Printf("%.50f\n", .1+.2)"

Let's start another round of DEC64 discussion? ;-)

http://www.dec64.com/

Here is a very nice website that gives solution also, not just describes the problem: http://floating-point-gui.de/

It helped me a lot when started learning.

Python's print method is actually fairly clever picking the "nicest" representation that's still correct.

https://docs.python.org/2/tutorial/floatingpoint.html#tut-fp...

Quote:

"In current versions, Python displays a value based on the shortest decimal fraction that rounds correctly back to the true binary value, resulting simply in ‘0.1’."

This doesn't seem true for me (Python 3.5.0 (default, Sep 20 2015, 11:28:25) [GCC 5.2.0] on linux).

    >>> print(.1 + .2)
    0.30000000000000004

Additionally, for exact calculations you might want to use the decimal module. https://docs.python.org/3.5/library/decimal.html#module-deci...

But be wary so you don't pass in floats when creating your decimals

    In [10]: from decimal import Decimal
    In [11]: Decimal(0.1) + Decimal(0.2) - Decimal(0.3)
    Out[11]: Decimal('2.775557561565156540423631668E-17')   

    In [12]: Decimal("0.1") + Decimal("0.2") - Decimal("0.3")
    Out[12]: Decimal('0.0')

Try the usual decimally-unrepresentable numbers (like 1/3).

As illumen pointed out, Python has also got fractions. For things like Pi however, there is no exact numerical representation and you'd probably want sympy or something similar.

Sure. But the inability to represent 1/3 in decimal is just the same as inability to represent 1/10 in binary. The binary case just seems different because we are using a foreign base (decimal) literal "0.1" to express the binary number.

I also posted this below as a reply.

Python uses David Gay's algorithm for printing the shortest equivalent floating point representation.

Implementation details and discussion are available from the Python bug tracker: https://bugs.python.org/issue1580

There are a few broken links. I believe the original poster references the following paper by Robert Burger & Kent Dybvig, "Printing Floating-Point Numbers Quickly and Accurately":

http://www.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf

There's also a reference to the following paper by William Clinger, "How to Read Floating Point Numbers Accurately":

http://www.cesura17.net/~will/professional/research/papers/r...

ftp://ftp.ccs.neu.edu/pub/people/will/retrospective.pdf

I believe this is the David Gay paper, "Correctly Rounded Binary-Decimal and Decimal-Binary Conversions":

http://www.ampl.com/REFS/rounding.pdf

By the way, this has previously been discussed on HN (which is where I think I first read all of this...):

https://news.ycombinator.com/item?id=2708983

There's also the GRISU algorithm: http://dl.acm.org/citation.cfm?id=1806623

What are the differences in implementation of the languages that cause some to show a rounding error and others not?

I suspect in many instances it's down to the method doing the printing being "helpful". The page documents this behaviour with Python. I've never specifically used C#, but it seems to default to the "G" format[1], the alternative "R" format shows the expected result. You can play around with it here: http://ideone.com/hxD96J

Printing out is kind of a misleading check here, maybe a better idea would be to test equality with the constant 0.3. Of course the site is just a neat illustration and not a technical whitepaper.

[1] https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).... and https://msdn.microsoft.com/en-us/library/3hfd35ad(v=vs.110)....

Yes, I agree this page is misleading. It makes it look like C# is using rational numbers instead of floating point. In fact the "G" format defaults to 15 digits of precision for doubles, which is few enough to avoid ugly strings in many simple situations, but not all:

> Console.WriteLine((.1 + .2)-.3); > Result: 5.55111512312578E-17

The printing routine. Internally, none of them can store 0.3 in a floating point variable because it is impossible to do so in exponential notation with base 2 (which is what IEE 754 uses).

Some languages may optionally use an exact representation (e.g., Scheme or Python), but that's not the general case.

There is one langauge that once I complained to the authors that it was broken: Lua.

Lua uses floating point by default, it can't make integer math at all, I made a game using Lua, back then DirectX had a bug where it would switch the floating point mode of the processor without permission, and sometimes the result was some extreme FPU imprecision.

Since Lua uses only the FPU for maths, even for integers, the result was that my game had lots of extremely bizarre results sometimes on Windows (while on Linux and OSX it worked as expected), it happened only once, but I saw 5+6 result in 13...

It was a really hard thing to debug, specially because of the flaming (when I asked about this on Lua IRC and forums, people flamed me endlessy, happily one guy in particular suggested me to check if it was the DX issue, and indeed it was).

> Lua uses floating point by default, it can't make integer math at all

JavaScript does this too. The fact is that with 64 bit floating point numbers, you can exactly represent every integer from -9007199254740992 to 9007199254740992, which is larger than the range of `int` on most systems anyway. 5+6 resulting in 13 is not an artifact of using floating point numbers. CPUs or GPUs don't randomly lose precision like that.

32 bit floating point numbers can exactly represent every integer from -16777216 to 16777216.

By the way, your reply was one of the most common replies to me that quickly turned into flaming (people offending me in all way they could to tell me Lua was perfect, and I was seeing things, because 32-bit is enough for the integers I was using...)

> it happened only once, but I saw 5+6 result in 13

That's not the result of standard floating point arithmetic. I don't know what it is, but it's not that.

Very interesting, I've never run into this, but I never used DX. For the simple example of 0.1+0.2, Lua does return 0.3 on Windows and OSX.

This table does not show which languages have support for Decimal numbers, which does the calculation exactly.

Can it calculate 1/3 exactly? You can in trinary: It's 0.1.

Binary, decimal, trinary, or any other numbering system is no more accurate or "exact" than any other. They all have fractions that they cannot represent.

For that you'd need a Rational.

Basically all languages have a Decimal number library you can use if you need it.

An explanation of the following useful fact:A fraction p/q is equal to a finite length floating point number in base b <=> q divides b^n <=> the prime factors of q are prime factors of b.

Proof if q divides b^n, then q * c = b^n => p/q = (p * c)/(q * c) = (p * c)/b^n = (p * c)e-n in base b.

Example: Express the fraction 7/18 (base 10), in base 60 and convert it to a floating point in base 60.

60 = 2^2 * 3^1 * 5^1, 18 = 2^1 * 3^2, look for c such that 18 * c is a power of 60. Observe that the factors of 60^2 all have exponents that are greater or equal than those of 18, so we can multiply 18 by the required factors to get (60^2). This way: 60^2 = (2^2 * 3 * 5)^2 = 2^4 * 3^2 * 5^2 = 18 * c = (2^1 * 3^2) * (2^3 * 3^0 * 5^2)= 18200, so we need to multiply by 200 the numerator and denominator of the initial fraction to get a new fraction whose denominator is a power of 60. Now 7/18 = (7 200)/(18 * 200) = 1400/(60^2) and 1400 = 23 * 60 + 20.

Finally 7/18 = 1400/(60^2) = 0.2360 in base 60.

We use the notation 00,01,02, ... 59 for the digits of base 60, so 2360 is a two digits number in base 60.

The asterisk used for multiplication is not shown in HN, don't know why.

The author forgot Microsoft Excel. There's even a Wikipedia page about it: https://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft...

Not that other spreadsheets (LibreOffice, Apple's Numbers) are better: They also use binary floating-point data types instead of a decimal one.

I remember, once upon a time... a bug where we were trying to find the average credit rating (AAA, AA+, AA, etc) of a portfolio of companies. And we had this issue where, in a simple degenerate case, the average of AA+ and AA+ became AA (AA+ is expected). In the end, it was an issue of floating point error accounting! (This was in C++ btw on a probably not-so-familiar platform).

TLDR; floor(13.0) != 13 floor(13.0) == 12

Each credit rating was mapped to a integer. We'd take the floating point average of all the integers and then floor it. The integral answer would be used to map back to a credit rating.

Let's say AA+ was mapped to the number 13. Well floor( (float(13) + float(13)) / 2.0) == 12. Not 13.

That's because floating point 13.0 is really 12.99999999999.

The fix was to add accounting for floating point error by adding in ULP (https://en.wikipedia.org/wiki/Unit_in_the_last_place) which in essence tips the representation of 13.0 to 13.00000000001 (or whatever the real number is).

This might be true on really weird platforms, but with IEEE-754 floating point this isn't true. Integers within a big range are exactly representable without any rounding error. Doing floor((float(13) + float(13))/2.0) should be exactly 13. It is entirely possible that the weird platform was supposed to be using IEEE-754 arithmetic but had a buggy math library (depressingly common).

Interesting that Swift and Objective-C give different results.

They do not. Open up a swift repl and try it.

I wish the full source for the Objective C case was provided. I'm curious to know how the value was printed.

FWIW

scala> BigDecimal("0.1") + BigDecimal("0.2")

res0: scala.math.BigDecimal = 0.3

Or as I wrote in 2008 http://codemonkeyism.com/once-and-for-all-do-not-use-double-...

Relevant smbc and xkcd:

http://www.smbc-comics.com/?id=2999 https://www.xkcd.com/217/

Recommended watch: "Demystifying Floating Point" by John Farrier at CppCon 2015.

https://www.youtube.com/watch?v=k12BJGSc2Nc

Why don't all these languages fold down to the same FPU add and hence produce the same output (on the same hardware)?

Are you telling me they implement their own version of the IEEE float standard? Surely that way madness lies.

The difference is in how the result is converted to a string, not in how the result is produced.

gotcha

In some cases, string conversion is different, and the same calculation is going on, but the output is different.

In some cases, the internal representation isn't binary floating point at all, so the errors resulting from approximating an exact decimal with a binary float don't occur.

The difference is between the length of their floating-point number (e.g. float vs double).

Rust: https://play.rust-lang.org/?gist=c02ab67dab3e180b5cf9&versio...

It's not quite that simple: https://play.rust-lang.org/?gist=7f6ba6bcd1c50868928a&versio...

I know that no one here cares but VBA returns 0.3

Edit: however a better test of ?.1+.2-.3 in the immediate window now returns 2.77555756156289E-17 I wondered about that first result...

I would like to see a language use decimal by default. Under the hood, it could use floating point for certain types of calculations where less precision is required. If I want to know the answer to 0.2 + 0.1, chances are I want the answer to be 0.3. If I'm writing performance-sensitive code, maybe I can drop down a level and opt for float(0.2) + float(0.1). Are any languages doing this currently?

Perl 6 uses rationals by default. They have the advantages of being base-agnostic, able to accurately represent any recurring digit expansion accurately regardless of eventual base, and also faster (since, especially if you normalize (convert to lowest terms) lazily, most operations are just a few integer instructions with no branching, looping, or bit-twiddling involved).

True. Also notice that if the user wants a floating point, the literal format is the one with the e<exponent> suffix:

    0.1e0, 1e-1, 0.2e0, 2e-1 etc.

You want COBOL. It did decimal by default 50 years ago.

Wouldn't you get other issues with decimal numbers? Consider 1/3 for instance.

What you really want, I think, is rationals by default, as in Perl 6.

> I would like to see a language use decimal by default.

So, Scheme?

Scheme has rationals, but decimal literals aren't rational by default. You have to type #e in front of it, or express it as a fraction.

It really ought to be fixed at the implementation level (how much stuff would it really break if floatin point errors went away?), but failing that I'd love a macro that did it for me. I can't figure out how to write one robustly.

Floating point math is fun. Here's a post I did awhile back the deals with similar-ish shenanigans. https://medium.com/forrest-the-woods/perfect-prevention-of-i...

anyone understand why is it 0.30000000000000004, why not 0.30000000000000002 or 0.30000000000000003 or something else?

Floating point is (usually) about "closest representable value". That is, the normal floats form a finite subset of the real numbers, and a calculation like x + y will pick the closest element to the true result.

In this case, adding the closest floating point value to "0.1" (0x1999999999999a / 2^56) and the closest one to "0.2" (0x1999999999999a / 2^55) gives

  0x13333333333334 / 2^54

which is exactly

  0.3000000000000000444089209850062616169452667236328125

(Rational numbers with a denominator that's a power of two have a finite decimal representation, and that's the full one for this number.)

However, when printing, one is either interested in a close-enough rounded value (e.g. like some languages print 0.3 by default, or how %.3f will explicitly limit to 3 digits), or an exact representation. The exact representation should be round-trippable, so when you read it back in, you get exactly the same value. In practice this means printing a float z as a decimal value such that the closest float to the exact decimal value (i.e. as a real number) is z. Obviously the full string above works because it is exact, but it is nice to be as short as possible. In this case, the floats immediately before and after our number are exactly

  0.299999999999999988897769753748434595763683319091796875
  0.300000000000000099920072216264088638126850128173828125

So, only printing up to the first 4 is enough to uniquely identify our float: it is the closest one.

The 0.2999* float is slightly interesting: it is the closest float to 0.3, and most environments will print "0.3" in exact mode.

There's a lot of trickery/research around printing[1] and operating on floats: rounding things correctly can be highly non-trivial[2].

[1]: e.g. http://www.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf

[2]: https://en.wikipedia.org/wiki/Rounding#Table-maker.27s_dilem...

It's the closest binary representation. See e.g. this: https://www.reddit.com/r/javascript/comments/2scikz/eli5_why...

FWIW, tried this with two less common languages:

    Tcl: + 0.1 0.2 
      0.30000000000000004

    Chicken Scheme: (+ 0.1 0.2)
      0.3

Same results under Windows 8.1 and FreeBSD 10.2. I suppose one is as "right" as the other, at least both answers equally popular among languages.

Also Chicken Scheme:

  (- 0.3 (+ 0.1 0.2))
    -5.55111512312578e-17

Of course you can always (use numbers), but then you have to use (/ 1 20) and (/ 2 10). "0.3" is still considered "inexact", because of the decimal point. I don't like this - finite decimals are perfectly rational numbers and trivially convertible, why can't the interpreter treat them as such?

Another less-common language. The docs suggests they use "C doubles" for all floats.

    Welcome to SWI-Prolog (Multi-threaded, 64 bits, Version 6.6.6)

    ?- Sum is 0.1 + 0.2.
    Sum = 0.30000000000000004.

This is more a comparison of output routines than anything else. In PHP, one of the languages that gives the "correct" result:

    php > echo .1 + .2;
    0.3
    php > if (.1 + .2 === .3) { echo "equal"; } else { echo "not equal"; }
    not equal

Errr that's exactly what the page implies; it does not say nor imply "those that display .3 are doing it correctly internaly".

One of the cardinal rule of floating point in computing is that you never do direct comparison between values, whatever the language, so your test should never ever be used. That doesn't mean a language cannot be allowed to infer what the correct display of a floating point value should be.

Recommended:

Floating Point Numbers - Computerphile https://www.youtube.com/watch?v=PZRI1IfStY0

Tom Scott's fun and simple take on Floating Points is a good video explaining why such weird errors happen

Along the same lines, but slightly different emphasis (fixed precision for accounting): http://roboprogs.com/devel/2010.02.html

Salesforce Apex:

  system.debug( 0.1 + 0.2 );
  09:25:37:027 USER_DEBUG [3]|DEBUG|0.3

However

  double a = 0.1;
  double b = 0.2;
  system.debug( a + b );
  09:26:57:043 USER_DEBUG [4]|DEBUG|0.30000000000000004

For C# i see a different behaviour as in his example. double test = 0.1 + 0.2; //will be 0.30000000000000004 if you check in the Debugger Debug.Print(test.ToString());//will print 0.3

Discussion of this issue from 70 days ago here:

https://news.ycombinator.com/item?id=10168799

The Common Lisp version is slightly misleading because computations are made by default with single-float types, whereas there is also short-float, double-float and long-float.

default for READ-DEFAULT-FLOAT-FORMAT is implementation dependent and can be changed by user.

In CL type float has subtypes single-float, double-float, short-float, and long-float. "Any two of them must be either disjoint types or the same type; if the same type, then any other types between them in the above ordering must also be the same type. For example, if the type single-float and the type long-float are the same type, then the type double-float must be the same type also."

If there is only one float representation it has to be `(typep x 'single-float)` but if the internal representation is different (double-float for example) it can be all of the types. At least this is how I understand the spec.

http://clhs.lisp.se/Body/v_rd_def.htm#STread-default-float-f...

http://clhs.lisp.se/Body/t_float.htm#float

http://clhs.lisp.se/Body/t_short_.htm

Indeed. Note also that there are recommended minimum precisions and exponent sizes, which are followed in practice by CL implementations. That means that if double and long are the same type, the precision and size is at least the one recommended for long-float.

Anyone care to explain the details on a Texas Instruments NSpire calculator (or CAS)?

    a:=0.1+0.2
    b:=0.3
    a-b     |>   0.

Using the non-exact mode.

Calculators have traditionally used base 10 internally, not base 2, what is the base of the numbers on that TI? That's the answer, if it's base 10, .1, .2 and .3 are all exact.

The Wikipedia mentions base 10 with TI Basic:

https://en.wikipedia.org/wiki/TI-BASIC

"Real numbers, using decimal floating point. These store up to 14 significant digits depending on the calculator model."

This is it.

An example: (1/3)*3.-1 gives -1E-14

Just a guess, but they're probably cheating the display. Try (a-b)*10^16

I tried some variations before I posted, but the result was just 0.

Obligatory SMBC reference

http://www.smbc-comics.com/?id=2999

So, best way to store currency amounts? Cents or decimal? Would the answer change based on platform or DB (i.e., Python, Postgres)?

Any SQL DBMS is able to store and handle NUMERIC data type. It is a binary decimal with any precision you may want.

Some of the DBMS's also have dedicated currency types.

Math tells me there is an infinite number of floats between 0 and 1. 64 bits are really not enough to address that numberspace...

They learn this in the first lectures of a CS study.

A bit unfair as Java has BigDecmial.

Bc and dc do exact arithmetic.

What a good domain name!

That's 90% of the reason this site exists.

AweSome Domain Name ... :-D :v :v :v !¡!¡

edit: The OP has accepted my pull request and this explanation is now directly available on the website.

It's a shame that the web page doesn't explain why floating point numbers are inaccurate when it comes to decimals. It's actually pretty simple. When you have a base 10 system (like ours), it can only express fractions that use a prime factor of the base. The prime factors of 10 are 2 and 5. So 1/2, 1/4, 1/5, 1/8, and 1/10 can all be expressed cleanly because the denominators all use prime factors of 10. In contrast, 1/3, 1/6, and 1/7 are all repeating decimals because their denominators use a prime factor of 3 or 7.

In binary (or base 2), the only prime factor is 2. So you can only express fractions cleanly which only contain 2 as a prime factor. In binary, 1/2, 1/4, 1/8 would all be expressed cleanly as decimals. While, 1/5 or 1/10 would be repeating decimals.

So 0.1 and 0.2 (1/10 and 1/5) while clean decimals in a base 10 system, are repeating decimals in the base 2 system the computer is operating in. When you do math on these repeating decimals, you end up with leftovers which carry over when you convert the computer's base 2 (binary) number into a more human readable base 10 number.

Spot on. I would add to this explanation that the real confusion comes from the fact that in most languages when you write `0.1`, it doesn't mean 0.1 = 1/10 – instead it means the closest number to 0.1 of the form n/2^k where n < 2^53. Namely 0.1000000000000000055511151231257827021181583404541015625, which we can represent in decimal because 2 divides 10. And, of course, the illusion that `0.1` is actually 0.1 is supported by the great efforts [1] made to print floating-point values with the least number of digits necessary to reproduce them, so `0.1` still looks like 0.1 when you print it even though it isn't.

[1] http://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/pr...

In decimal if you only have 5 digits for mantissa and you write 2/3, it becomes 0.66667 which is a little bit more than 2/3. Similarly, if you write the binary number 0.0000'0000'1 in decimal with 5 digits for mantissa, it becomes 0.0039063 instead of 0.00390625. If in some faraway land they use binary math, they would likely ask why such a simple binary number got rounded off.

Huhh, that was a totally unexpected encounter to Prof. Ramsey's page.

THANK YOU! That is one of the best explanations on that matter I have seen. I've never considered why floating points are inaccurate, I just took it for a given (which I hate doing). Now I understand why, and so my own wisdom has increased ever so slightly. Today is a good day.