DEC64: Decimal Floating Point
dec64.com
dec64.com
There are situations where other decimal floating-point types are appropriate, but if you do not understand the tradeoffs you are making, you should be using decimal128.
I don't agree with everything in Jens Nockert's "silly review", but he's right about a lot of things: http://blog.aventine.se/2014/03/09/a-silly-review-of-dec64.h...
I made a few notes the last time I saw this type come up somewhere:
- It has significantly less exponent range than IEEE decimal64 (it effectively throws away almost three bits in order to have the exponent fit in a byte; 2^56 is ~7.2E16, which means that it can represent some, but not all, 17-digit significands; the effective working precision is 16 digits, which actually requires only ~53.15 bits).
- Even if you weren’t going to use those extra bits for exponent, they could be profitably used for other purposes.
- Lack of infinity is a mild annoyance.
- Rounding is biased for add/sub/mul, broken for divide.
- It's not significantly more computationally efficient than the IEEE formats, despite handwaving by the author.
----
Edit: I would be remiss not to note that Intel has made available a well-tested complete implementation of IEEE 754 decimal64 and decimal128 under 3-clause BSD: http://www.netlib.org/misc/intel/
I don't know how much effort Intel is allocating to this project; I sent a bug report to the author listed on that page in 2015. The bug was acknowledged and fixed, with a new release out in "several days", but nothing yet.
Maybe DEC64 is flawed, but can we stop pretending that we like binary-based default systems for any reason other than the barrier to entry it creates for newcomers and visceral feeling of communing with computer internals.
Binary has a place -- in low-level languages. Like assembler. There's nothing wrong with any language providing a non-decimal system as an optional number type.
The system we have in many high-level languages is a frankenstein of a binary system that you usually interact with through decimal representation.
The proposal gets at least one thing right: the languages of the future will have a numeric system default that has less insanity. But it will take the passing of a generation or two of developers.
In the real world, there are natural numbers, integers, rationals and real numbers.
Computer languages are designed with types that mimic this real-world number stack. Low-level binary implementation details don't leak unless you're overflowing or using bit operations.
What you're really complaining about is the fact that rationals aren't a first-class type in any popular language. With that I agree, it's a shame that corners were cut we three number types instead of four.
They are at least in Clojure, which seems quite popular.
No sense in special casing the number 10 when you can have proper rational numbers with almost equal effort.
Every newbie programmer tries to avoid thing about this by using IEEE floats. They discover, usually years later after some anal auditor has come down on them like a ton of bricks for the hundredth time because the dropped low order bits from 1/3 of a cent hit that 1 in a billion case and effected a significant digit, and then it finally dawns that 1 in a billion isn't really 1 in a billion because thousands of such calculations get combined into a single profit and loss figure that is out by 2 cents and chasing that 2 cents for 2 weeks really only to discover it was caused by a computer can't compute, really, really pisses off the auditor, that you realise if aren't thinking about those fractions of a cent as hard as a C programmer focuses on malloc(), they will have gone to whoop whoop in half the code you have written. You will have nightmares about divide signs for the rest of your life. Crockford seems to think Dec64 allows the programmer to avoid thinking about the problem. He is just as wrong as every newbie programmer.
There is only one safe format for currency that accurately reflects this reality, that forces you to accept that you must think about those fractions of a cent. It is the humble integer. You put in it the lowest denomination: cents in the case of the USA. And then when you write total = subtotal / 3 * 4, and you test it, your error stands out like dogs balls, and you fix it.
This could be solved with a rational-style data type, but I consider the fact that real-style data types don't capture that to also be the implementation leaking.
No, they don't. Floats are meant to represent real numbers reasonably accurately.
Real numbers aren't fractions, and the idea of 'rounding' doesn't even apply to reals.
What you want are rational numbers, they solve your use case perfectly.
Floats don't represent rational numbers properly, and they are not meant to.
"Binary number" as used by grandparent really refers to dyadic rationals (https://en.wikipedia.org/wiki/Dyadic_rational), which are a perfectly well-defined dense subset of the rationals. Similarly, "decimal number" is really "terminating decimal expansion" (or whatever you want to call the decimal analogue of dyadic rational), which is again a well-defined dense subset of the rationals. This is a perfectly valid mathematical distinction; the numbers that people work with day-to-day are much more frequently the latter.
There are reals, they do exist. In the "real" world (as poorly defined as that is).
The issue is, fundamentally, what programming languages call "real numbers" are not real numbers. They are an approximation to a subset of the reals. This approximation has holes, and the implementations work to some degree to define regions of applicability, and regions of inapplicability. Usually people get hung up or caught in the various traps (inadvertently) added to the specs for "Reals".
Its generally better to say "floating point numbers" than "Reals" in CS, simply because floating point is that subset of the Reals that we are accustomed to using.
I definitely agree with the comment on rationals. I am a fan of Perl6, Julia and other languages ability to use rationals as first class number types.
Sadly, as with other good ideas that require people alter their code/libraries, I fear this will not catch on due to implicit momentum of existing systems.
From this, it’s fairly non-controvesial to say that only the computable reals exist; these are a tiny (measure-zero) subset of the reals.
If you go further and assume a fundamentally discrete universe (much more controvesial), then all you can really measure are integers.
[1] http://people.sju.edu/~jhodgson/ppl/lisp/lispman.txt [2] https://dspace.mit.edu/handle/1721.1/5600
Javascript programmers don't need a number type designed for accurately computing trig and square roots and representing pi, but they got one anyways.
He said it's the biggest mistake he's ever seen go down professionally.
1. "rounding modes and overflow behavior are not addressed", which is incorrect.
2. "Where's exp(), log(), sin(), cos()?", which you can find in dec64_math.c.
3. "There are also 255 representations of almost all representable numbers", which is incorrect.
4. "it will take around FIFTY INSTRUCTIONS TO CHECK IF TWO NUMBERS ARE EQUAL OR NOT" in the slow case, which is probably true but is unlikely to be common given the design of the type. It can undoubtedly be improved with effort.
5. "The exponent bits should be the higher order bits. Otherwise, this type breaks compatibility with existing comparator circuitry", which seems like an odd comment given the type is not normalized.
6. "With both fields being 2's compliment, you're wasting a bit, just to indicate sign", which AFAICT is just false.
7. "there's no fraction support", which I don't understand.
So the only valid criticism I saw in that whole thread is #4, and even that is only a tenth as valid as its author thought it was.
Not really here to defend these criticisms, but if #4 is only a tenth as valid as the author thought it was, then does that mean that you know the worst case instruction count to compare two DEC64 numbers is five? Also, we're talking x86 here, how many μops and static instruction bytes are we talking on either side of this? Are all of the branches usually predictable? Can we ever afford to inline something as critical as the comparison operation?
As for the validity of #5, I think this might make more sense in the context of hardware implementations, where a low-order exponent position could increase complexity (requiring adjustments to reuse any of the FPU datapath) AFAIK.
Re 2: these are at best toy implementations, at worst dangerous (in the sense that they may provide wildly inaccurate results, due to convergence
Re 3: agreed. Though there is still a gargantuan number of values which have 2 or more representations, and that makes all kinds of comparisons more complicated. Dealing with that efficiently in SW is difficult, and more so in HW. It might be worth it if the advantages outweigh it, but at least I personally don't see it.
Re 4: if it can be "undoubtedly improved with effort", why hasn't it been done in several years? Sure, it may be possible, but I will keep my doubts for the time being :-)
I agree with you on 5, 6 and 7.
My inexpert understanding is that modifying rounding modes is super niche and poorly supported by most things, so this doesn't strike me as much of a problem. A saner replacement to rounding mode flags would just be to offer different operations for those rare cases they are wanted.
> Dealing with that efficiently in SW is difficult, and more so in HW.
Not really; you never really need to normalize values and not doing so makes basically everything other than comparisons cheaper. I don't see how normalizing around every arithmetic operation would make the hardware any simpler.
> if it can be "undoubtedly improved with effort", why hasn't it been done in several years?
Because this is one guy's project and it hasn't seen much (any?) use.
4. To implement compare quickly you can probably short circuit by first checking that the values are normalized, and normalizing them if not. From a hardware point of view you just added two registers for compare, in addition to the logic for normalizing. For hardware it is probably cheaper to unconditionally normalize.
All math operations will also need to normalize inputs first because otherwise you throw away precision for no reason. For similar reasons all outputs will need to be normalized anyway. Again, given this why would you choose to not just have an implied leading bit?
5. Uh yeah what? If you’re making dedicated hardware you could randomize the bit ordering that’s what making hardware is all about ;)
1 in 10 values have two representations, 1 in 100 have three, etc., since you can only change the exponent when the (normalized) value is 0 mod 10.
> For hardware it is probably cheaper to unconditionally normalize.
Given normalization is going to be infrequent (most values will either just be integers or have a single representation), and that arithmetic is more common than comparisons, I don't see why this would be. Normalizing a binary value is much easier, so you can't carry floating point intuitions over.
> All math operations will also need to normalize inputs first because otherwise you throw away precision for no reason.
No, you just conform the exponents and handle overflow.
That's false. The author do not understand the limitations of binary floating point.
0.2 is a periodic number in binary floating point.
People need to first understand why 0.2 is a periodic number in binary floating point before writing a blog post.
Anyway, IIRC you can round-trip 15 decimal digits correctly with an IEEE double. Don't know if it's floor(15.955) or something else, but it's close enough for me to consider DEC64 quite useless compared to existing, quite well-designed and widely used implementation of FP.
Edit: it does say "A later revision of IEEE 754 attempted to remedy this, but the formats it recommended were so inefficient that it has not found much acceptance." at the end. Hmm.
Addition and subtraction on anything other than matched exponents is going to need rounds of multiplication-by-10, however you implement it. Using IEEE-754 packed decimal, you only need a handful of gates per digit to unpack into BCD.
Software multiplication is pretty fast? BCD is comparatively hard to do in software.
Intended by whom? A lone voice (however correct), a standards body, or an industry consortium? This article should really carry some authorship information.
The explicit leading bit directly halves the space of addressable bits (which is how you get space for multiple representations)
If you want insight to how awful this is, x87 has pseudo infinities, pseudo /nans/, unnormal values, and pseudo denormals
The solution intel eventually took was to recognize that the leading bit was useless but require it to be as though it were implicit and treat any case where that is wrong as being invalid.
So yes you do want a format that requires normalization, because the alternative is one that requires normalization anyway, but is also insane for any kind of comparison, and wastes precision needlessly.
(Disclaimer: I don't work with any financial figures and I have not checked whether the dec64 proposal is a sound one.)
In Python, the story looks slightly better:
from decimal import *
>>> getcontext().prec = 6
>>> Decimal(1) / Decimal(7)
Decimal('0.142857')
But I think it's important to make sure programmers don't end up with the idea that they might as well store money amounts in binary floating point because the language they are using doesn't support decimal.
.getcontext().prec = 6
AFAIK (and I don't do this for work), after every calculation you manually need to call quantize as in
cents = decimal.Decimal('.01')
money = price.quantize(cents, decimal.ROUND_HALF_UP)https://en.wikipedia.org/wiki/Decimal_data_type
Also, I don't try to endorse the Python API. I don't even use it. I just pointed out their api is easier since the operators work with their decimal data type, as opposed to the Java version.
In general you need to know when to round and to how many digits. But my complaint was that languages have a natural API for binary floating point, which is useful for scientific number crunching, but rarely anything comparable for commercial calculations.
I have not tried to design a natural API for fixed (but parameterizable) decimal precision and the implicit rounding required. But I would be surprised if it's impossible to come up with anything less verbose (explicit constructors everywhere) and less error prone (forgetting to call rounding).
If you're on the JVM, Groovy is nicer in that regard. It uses by default BigDecimal and BigInteger for literal numeric types, and has operator overloading: http://groovy-lang.org/syntax.html
Apache Groovy uses by default BigDecimal for literal decimal types, but only uses BigInteger for literal integral types when it would overflow the largest available primitive type for that literal. You can force a BigInteger by using the suffix `g` on the integral literal. Perhaps misunderstanding this is the source of some hard-to-find bugs in some of your Groovy code.
That works fine for addition and subtraction. When you divide (or do percent calculations) integer cents is no longer, you need to deal with rounding manually/some custom class with every calculation.
Also, what's with the aversion to traits/types? This is the whole point of having operators/types.
We need a decimal format because people who read the code are sometimes not coders how know about the various tricks to pack decimals in uint64.
So, u64 is perfectly doable, sure, but it doesn't scale with the various types of people who write/read the code.
(same thing for floats : you can do money computations with them, but that'll work only if the people understand the trade offs of the representation; and believe me, they're not legions)
*Ok not that one.
https://dev.mysql.com/doc/refman/5.7/en/fixed-point-types.ht...
I have never done any SW development in the commercial domain. Do you say in banking system the amounts never make it to the host language, they are always calculated in the database?
A cashier system most likely doesn't even have a database. At least not used for every item.
For the average cashier / checkout system I would doubt that, although again I have never seen the source.
It’s provided in most higher level environments these days (java, .net - where I think it might even be a vm primitive?)
I still question the value of it but it does allow “perfect” representation of values common in decimal systems. But by the same token it can’t represent everything in all other bases. Shrug.
Non-compliant implementations are widespread for performance reasons, but that is a completely different story.
Also I'm not sure about GPU, but FPU are typically compliant in HW, and it is typically the compilers that have faster approximate modes, and I think for mainstream ones only when you explicitly enable such optimizations, which are disabled by default.
Sure, but to get there it'd need to interface with the current generation of languages which still use double.
Has anyone ported a non-trivial application to use DEC64, and compared the results?
Boundless arrogance, minimal information.
If you want a decimal floating point type, use the IEEE formats, there are high quality implementations everywhere, and if you're using C++ you can probably switch without much more than a string replacement and a couple header includes. Douglas Crockford can be forgiven for having apparently no concept of computing at the limits of the machine (in terms of cache latency, memory size, and CPU throughput), but if you get comfortable with this level of ignorance and aren't famous, you will be highly replaceable.
Added:
Moore's law is effectively dead. This means that for a given CPU microarchitecture family and your monetary or spatial budget for memory, you have limited resources to achieve your goals. If you write an inefficient program with no regard for performance at the numerics level, your program will no longer automatically get much faster and cheaper to run, you will instead be contributing to the pile of performance debt your successors will be cursing and shedding tears over.
Furthermore, with his "loss of accuracy" comment, he seems to imply that his 64 bit decimal types are even remotely large enough that common users will not lose accuracy (by which I suppose he really means precision).