Hacker News Reader

Comment by soegaard | Hacker News Reader

Anyone care to explain the details on a Texas Instruments NSpire calculator (or CAS)?

    a:=0.1+0.2
    b:=0.3
    a-b     |>   0.

Using the non-exact mode.

View on HN

Calculators have traditionally used base 10 internally, not base 2, what is the base of the numbers on that TI? That's the answer, if it's base 10, .1, .2 and .3 are all exact.

The Wikipedia mentions base 10 with TI Basic:

https://en.wikipedia.org/wiki/TI-BASIC

"Real numbers, using decimal floating point. These store up to 14 significant digits depending on the calculator model."

This is it.

An example: (1/3)*3.-1 gives -1E-14

Just a guess, but they're probably cheating the display. Try (a-b)*10^16

I tried some variations before I posted, but the result was just 0.

Reply on news.ycombinator.com