[1] https://docs.python.org/3/library/decimal.html#context-objec...
[1] https://www.stackage.org/haddock/lts-20.10/base-4.16.4.0/Pre...
[1] https://python-history.blogspot.com/2009/03/problem-with-int...
The next level solution is to apply generators so that either the decimal stream or the continued fraction is allowed to be infinitely precise, but I think this can have dangerous effects where checking whether a number is equal to 0 or maybe 1 can involve infinite computation? So that's where you really understand “oh, I do really need that epsilon, for comparisons’ sake.”
For continued fractions I think you can also just have your library bound the size of the integers involved? So “it’s an array of signed int32s, but if your continued fraction generates a number that would overflow that, we just truncate the stream at that point.” Then the library is able to say that these two things are equal because their difference is [0; int_overflow] which becomes just [0]. Something like that.
which is pretty darn amazing. pi and e are essentially first class, but a lot of transcendentals aren't. seems like a really neat approach.
That's actually a really interesting question - while this is obviously true for most functions which (in a mathematical sense) exist, I wonder if it's true for "all functions weighted by their use in computing applications"? That is - do boring old "addition, subtraction, and multiplication of integers" outweigh division, trigonometrics, etc.?
In 3D modelling/video games, almost certainly not. In accounting software...probably? Across the whole universe of programs: who could say?
[1] https://peps.python.org/pep-0465/#so-is-good-for-matrix-form...
Since addition, subtraction, multiplication, and modulus are each used more than division and exponentiation _combined_ (and since not every use of those last two functions would result in an "inexact" result), I think we can pretty clearly conclude that "most usages of mathematical operators in these libraries will result in an 'exact' result" (I'm hand-waving on the definition of "exact", I don't think it's at issue here)
Which is not, of course, a good justification for ceasing to worry about the problem, since a) those packages might not be representative of all libraries, and b) a small proportion of uses might result in a disproportionate amount of bugs.
This is what concerns me. Sure, using decimal floating point solves 0.1 + 0.2 = 0.3 (which I can't imagine ever writing in real code). But if you get used to that, then you start to expect 0.1*x + 0.2*x to be 0.3*x, and depending what x is this may or may not be true. Maybe it works for all of your test cases (because your test cases are things like 2 and 10^-4), but then you accept some user input and start getting weird bugs (or infinite loops). There is no good solution besides expecting and preparing for rounding error.
This reminded me of an article I read awhile ago, probably here on HN:
https://randomascii.wordpress.com/2014/01/27/theres-only-fou...
Though regardless of usage, I think that people doing stuff that needs floats are more likely to understand why they need them, and have the ability to use them explicitly without much issue. By using python, and most other high level languages, we're already making sacrifices to make things easier to use and understand, and in Python specifically we're told that explicit is better than implicit, except for this.
Sure, but how common is that? Exponentiation is almost always positive when I've seen it.
>In 3D modelling/video games, almost certainly not.
A 32-bit integer divided into a 16-bit whole and a 16-bit fraction would be limited to only representing values between -32768 and 32767 while also having worse precision than a 32-bit ieee std754 floating-point at values near 0.
>In accounting software...probably?
Representing money in terms of cents instead of dollars removes the need for real-numbers entirely outside of "Office Space" scenarios where tracking fractions of cents over millions of transactions adds up to tangible amounts of money.
>Across the whole universe of programs: who could say?
Most computer programs don't need real numbers of any sort, and the ones that do need to be written by people who understand basic mathematical concepts like precision.
...OK? I'm not sure how that relates to my supposition that trigonometric operations are likely to be more common in 3D modelling cases. I'm not arguing for or against any particular representation of numbers therein.
> Representing money in terms of cents instead of dollars removes the need for real-numbers entirely
I wasn't imagining dollars-and-cents, but rather rates - X per Y, the most natural way in which division arises in real life.
> Most computer programs don't need real numbers of any sort...
You're again arguing against a case I'm not making. I'm not making any claims about the necessity (or otherwise) of real numbers in programs, but simply wondering about the prevalence of particular operations.
> and the ones that do need to be written by people who understand basic mathematical concepts like precision.
A snide insult motivated by your own misunderstanding of my point. I understand precision, and it's irrelevant to my point.
jeez talk about arguing against cases i didnt make, i never said you dont understand precision.
And will break when the version reaches 1.10. While I agree we need a better way to teach this (e.g. inexact-exact distinction as in Scheme or more recently Pyret), that's as problematic as storing a telephone number as an integer (or worse, a FP number).
For example, I'm a huge fan of TypeScript, but it is hamstrung by the fact that javascript only supports a single `number` type (and, recently, `bigint`). Worse is the effect that since JSON is derived from javascript, it also has no built-in decimal type. So what happens inevitably when you want to represent stuff like money:
1. First people start using plain numbers, then they eventually hit the issues like this post.
2. Then they have to decide how they will represent decimals in things like APIs. Decimal string? Integers that represent pennies or some fraction of pennies?
3. Also, pretty much all databases support decimals natively, so then you get into this weird mash of how to not lose precision when transferring data to and from the DB.
Overall it's just definitely one of those issues that programmers hit and rediscover again and again and again. I'm surprised there hasn't been more movement towards a better language-level solution for the post popular language in use worldwide.
> The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.
So, in reality, the JSON "spec" is really how the most popular implementations interpret it. I'm not aware of a single implementation (though I could most definitely be wrong) that interprets number tokens as anything but floats/doubles by default.
https://learn.microsoft.com/en-us/dotnet/api/system.decimal?...
Really wish JS had added first class support for a bigdecimal class before bigint. After all, the first is basically a superset of the latter.
IEEE 64 bit floats are accurate to just past 15 decimal digits. For ordinary monetary amounts, the exact figure in cents is approximated with ridiculous precision. If you rub two pennies together, you are likely causing more of a difference in the amount of copper than the IEEE 64 bit float causes in the value.
You have to do many, many additions and multiplications before you get a result which has accumulated so much error that it is now closer to the wrong penny. E.g. if you don't deal with dollar amounts more than 7 figures, you have about 8 places past the decimal point; you need something like a 6 place error before the penny is affected.
You can counteract this problem by correcting intermediate results to that floating-point value which is closest to exact penny result. In other words, throughout your calculation, you truncate away the difference between the result, and the best approximation of the dollar and cent value.
Within this framework, you can implement all required rounding rules, too. You can take a floating-point result representing a fraction of a penny and round it according to banker's rule to the penny.
Of course, you can't just ignore the issue and just blindly use floating-point for money in a serious accounting system; but that's a strawman version of using floating-point for money.
Also, Microsoft Excel uses floating point. See here:
https://learn.microsoft.com/en-us/office/troubleshoot/excel/...
Vast armies of people rely on Excel for financial calculations.
> You can counteract this problem by correcting intermediate results to that floating-point value which is closest to exact penny result
The argument seems to be "floats are okay, as long as you're careful", but forgetting to round the number in between a large number of operations is a probable mistake.
Using decimals would make such a mistake impossible.
Decimals are software libs; what could go wrong?
More people use the floating-point instructions of a popular CPU than any given decimal library.
If you're starting from scratch, it's probably a lot less work to write (test and debug) a Money class based on floating-point, whose overloaded math operations do the required rounding (so that code using the class cannot forget) than to make a Money class based on decimal arithmetic.
(The last time I wrote an accounting system, I made a Money class based on integers. It could be retargeted to other representations easily. I could make the change and compare the ledger totals and other reports to see if there is a difference.)
Perhaps, but how many of the former are in a position to notice accuracy issues? I have more faith in a reputable decimal library than your average FPU, frankly.
Bugs in libraries do exist, but it's much easier to fix a bug in one place, than to track down every single line where floating point operations could misbehave.
> IEEE 64 bit floats
That should be IEEE 64-bit floating-point. The name `float` is a 32-bit floating-point.
> are accurate to just past 15 decimal digits.
FOR SOME RANGE. They're called floating-point because they can modify how much scaling is devoted to either side of the decimal point. The more scale it has to devote to representing whole portion, the lower its precision in the fractional portion. At some point, a 64-bit floating-point value cannot represent any decimal digits. Past around 2^53, a `double` 64-bit floating-point value cannot even represent every whole number, because at that point the available precision is 2 or more.
If I say "64 bit float", it's obviously not that one.
Here is CLISP:
[1]> (type-of 3.0)
SINGLE-FLOAT
[2]> (type-of 3.0d0)
DOUBLE-FLOAT
>>> type(3.0)
<class 'float'>
>>> type(3.0e300)
<class 'float'>
Rust has f32 and f64. Some historic languages have used type names like REAL and DOUBLE and others.
"IEEE 64 bit float" is almost unambiguous; it could be the binary one or the decimal one.
IEEE uses identifiers like binary64, decimal32, if you want to be pedantic, and not float and double.
> FOR SOME RANGE ...
Your comment is seems to be based on the wrong idea of what the number of digits means. An IEEE 64 bit binary float stores 15 significant digits without loss. In that C language that gives us the float type, the preprocessor constant DBL_DIG has a value of 15 (on IEEE floating point platforms).
This is a 15 digit number using E notation (in terms of digits of precision / significant figures):
1.23456789012345E-20
123456789012345.0
12345678901234500.0 (same as 1.23456789012345E16)
I think it's only for subnormal numbers that the 15 rule breaks down; but I might have mentioned that. These are special representations close to zero beyond what is reachable with the regular exponent and mantissa, which have the benefit of certain desirable behaviors in underflow situations.
if (walletBalance - sumOfWithdrawals < 0) { throw new Error('overdrawn); }
Point being there are tons of operations in the financial world where you check things against 0, or want to ensure that a breakdown of smaller transactions equals a larger amount. Those all can fail with floating points but succeed with decimals.
I would think that, nowadays, every child would learn that calculators do not always produce exact answers almost in kindergarten.
Also (nitpick), it’s not “float vs decimal”. “Floating vs fixed point” and “binary vs decimal” are orthogonal issues.
such a strange comment to make. the number of people that would ever bump into this situation is so small. like the difference of .1 + .2 = .3 and .30000000000000004
i just used my iPhone to do (√2)² and it displayed 2 as the result. same for 3 x 1/3 to receive an answer of 1. i can only assume that the default android calculator app would behave the same. between those 2 apps, we've probably covered the default calculator for the majority of people.
gotta break out of the HN is the world shell, and realize the majority of people do not suffer the same issues you might deal with on a daily basis.
(My Haskell-fu isn't deep, but I suspect it would even be possible to write it so that, for example, multiplication of two division operation expressions multiplied the numerators together instead of doing divide -> divide -> multiply...).
$ ghci
GHCi, version 8.10.7: https://www.haskell.org/ghc/ :? for help
Prelude> :m +Data.Ratio
Prelude Data.Ratio> :t (%)
(%) :: Integral a => a -> a -> Ratio a
Prelude Data.Ratio> 18 % 21
6 % 7
Prelude Data.Ratio> 1%10 + 2%10
3 % 10
Prelude> :m +Data.CReal Data.Complex
Prelude Data.CReal Data.Complex> let i = 0 :+ 1
Prelude Data.CReal Data.Complex> exp (i * pi) + 1 :: Complex (CReal 0)
0 :+ 0Then you need to do some analysis on your AST to try and restructure it in a way that preserves as much precision as possible.
You don’t really need Haskell for this though, in theory it will work in any language (but more ergonomically if you have operator overloading).
I think this is a pretty user-friendly compromise.
By default, when it not longer fits, it will convert to floating point (called Num in Raku). But this behaviour can be set: another alternative is for the numerator and denominator both being big integers. This gives you infinite precision rational numbers, at the expense of potentially needing infinite CPU and/or RAM.
So I caution against blind preference for rational representations as well. You really have to choose your numeric representation based on your use case. It's unfortunate that this can be so hard to control precisely in many programming languages.
To many, a rounding error makes the answer "wrong", and suddenly the tool has switched from a reliable one into an untrustworthy one.
By middle school, kids should have learned that you can't trust calculators. There are all sorts of numbers like pi, e, sqrt(2) that are impossible to represent. Once you start getting into trig, you have to accept rounding.
Explaining _why_ .1 isn't representable requires explaining IEEE-754 and explaining _that_ requires an understanding of binary numeric representation.
I teach college students who find this confusing, so I think it's fair that the average person finds floating point behavior confusing (in fact, I've had to explain to Physics Professors doing computation simulation work why their 1-<tiny number> isn't working out the way they expect -- though they initially tried using double doubles to get around the problem).
i=0
while(i<1):
<something with i>
i=i+.1
Very frustrating, but in retrospect very eye opening.
I think that's a good lesson for kids.
That's how I learned about it years ago.
When Python originally made the choice to have literals with decimal points in them be floats, the language did not have a decimal implementation, so floats were the only choice.
I don't know if anyone has proposed changing the default now that Python does have a decimal implementation, but I suspect that such a proposal would be rejected by the Python developers as breaking too much existing code.
What would be almost as nice, and would be backwards compatible, would be introducing a more compact way to declare decimal literals, something like "0.1d".
You realize that decimal numbers also have the exact same types of rounding issues as binary numbers right? The only difference is that the former allows you to divide by both 2 and 5 cleanly, whereas the latter only lets you divide by powers of 2. If you want to divide by 3, 7, 11, or compute a square root or an exponent, using decimals is not going to save you from having to reason about rounding.
But what if... what if they had a bastard child? What if we moved the point a fixed distance in decimal... and also a floating distance in binary?
The value represented would then be sign * mantissa * 2^exponent * 10^bias
With a bias of -6, you could represent every multiple of 0.000001 up to 9 billion if I did the math correctly.
Would it? I thought dealing with integers - a value, in binary - would be faster than floats - a value in binary, a decimal places value, whatever odd logic there is required to hide the leaky abstraction.
Edit: nevermind. Since the conversation was 'decimal versus float' I thought 'decimal' meant integers without floating points.
If decimals means a decimal point, I think a better suggestion would be to use integers.
Ironically, this is a much better description of `decimal` than of `float`.
IEEE float math is done in hardware. It is "one value in binary" that is added, subtracted, multiplied, etc etc with electric circuits.
The decimal abstraction requires manually keeping track of the number of significant digits, converting back and forth so that two different decimals can be added / multiplied, etc etc. There's a lot more that has to happen besides asking the CPU to do a single operation.
Its easy to think of "decimal" as one thing because every language provides a library called `decimal`, but there are a million subtle decisions and tradeoffs to make when choosing one standard binary representation. Most languages don't have a binary representation at all, and implement `decimal` as a high level abstraction with regular integers.
On modern CPUs, it's faster to cast a number to double, do a square root and cast back to int, than even the cleverest bithacking int algorithm.
b) Floats are efficient approximations of real numbers. Trigonometry and logarithms are vastly more important than having the numbers be printed pretty, so defaulting to rationals instead of reals is quite insane.
Citation needed. I suspect more people are doing accountancy than physics simulations.
f = float(closest_to=0.1)
But like the parent poster noted, hardware tends to not actually implement the decimal float parts (I mean, IEEE 754 doesn't care about how calculations are made or how fast they are, so a software emulation is perfectly acceptable from the standard perspective). I think IBM POWER has one of the rare HW implementations of decimal floats.
:)
(with a multiplication to enter and a division to exit)
Acceleration for binary fixed precision was (still is I guess) common in DSPs. Not decimal fixed though.
I lost track of which extensions Intel provides, but I wouldn't be surprised if something was available.
https://docs.oracle.com/cd/E19120-01/open.solaris/817-5477/e...
Why not rational numbers?
Money transfers tend to involve payments with fixed precisions. For cash, it's often rounded to the smallest available coin, for credit cards, you can add a few more digits, but it will still be fixed point.
When you settle a transaction, you typically need to pay exactly the amount defined by the transaction because some system is doing some check that the amount is exactly right.
So in my example above, if your friends are paying $33.33 you may have to pay $33.34 to make the transaction go through.
When you're writing systems involved in actual money transfers (or accounting) you tend to be better off using fixed point decimal, with predefined precision for each variable involved.
Within a calculation, it may be fine to use float, but the number going into ledgers or transactions need to have the predefined precision.
Fixed point (ie calculate everything in cents) is a good convention for lots of things to do with money. But it's not a good convention to have as the default in a language like Python.
> Within a calculation, it may be fine to use float, but the number going into ledgers or transactions need to have the predefined precision.
I was not suggesting the use of float. I was suggesting using rational numbers by default. See eg https://docs.python.org/3/library/fractions.html
let foo = 0.1 + 0.2;
But more to the point of your question, we use floating point math because that's what computers are good at, not because we want that for it's own sake. We want to figure out sales tax, or how long until our kid will need to buy new shoes, or what effect changing the speed limit had on the total number of accidents, or all kinds of other things that humans care about. Using a floating point representation may be the most efficient way to get some of those answers using the technology available, but it is just a step along the way, not what we actually want. That's what I mean by implementation detail.
Basically, I think that any literals typed into the interpreter should work the same way as a calculator. If you want something special for your implementation because it will work better or faster, then that should be explicit.
How your calculator handles rounding is itself an implementation detail. I have no idea what rules your calculator uses to round and it’s not in any standard. Does my calculator do the same thing? Unknowable.
It’s the conversation from an abstract model to a concrete instantiation where floats are used, generally out of necessity or ignorance.
The fewer details needed to do this conversion, the easier it is to develop programs. When I say easier, I mean it’s faster AND less buggy — since the conversation often involves introduces errors, subtleties, and logic not present in the abstract model.
It also appears that the logic for implemented something like this standard is indeed slower than standard IEEE754. Even if it’s only a bit slower, seems bad to make it the default.
All this just to fix something which is confusing to a novice programmer… and this is leaving aside the additional complications a fixed width decimal has which a floating point type doesn’t have.
What would be much slower is all the native code that Python apps use for bulk math, such as numpy. And that is because we don't have decimal floating point implemented in hardware, not because of some fundamental limitation. If decimals were more popular in programming, I'm sure the CPUs would quickly start providing optimized instructions for them, much like BCD was handled when it was popular.
That leaves vanilla Python. You’re right that compared to dynamic dispatch, a decimal op implemented in hardware is nothing, but the issue here (IMO) is death by a thousand cuts. Python is already slow as hell, and currently using decimal as a default would require a software implementation in many if not all places, which will be slow. Trade off does not seem worth it to me.