Hacker News Reader

Comment by xlii | Hacker News Reader

I glanced, and I found this handbook shallow and - in some areas - even bad advice.

E.g. If I ever see a monetary value stored in something else than integers I'm going to run away screaming (thank you Rust decimals represented as JSON floats). It's always integers unless you have a VERY good reason to do otherwise (though exported view can be in anything, even in weird bitcoded formats).

FX exchange. Resolution of FX isn't a point-in-time thing, things like buyer rate-in-time, seller rate-in-time, agreement, agreement tolerance, agreed upon resolution timestamp come in the effect.

Immutability - that's why you want to have event sourcing everywhere that touches money:

    # Resolved stream
    A -> B -> E

    # Actual stream
    A0 -> Edit(A0, A) -> B -> C -> D -> Rollback(B) -> E

Though in the end Fintech != Fintech. I worked at Fintech where money was treated like a baggage, and in other where money was a central point of everything.

View on HN

> E.g. If I ever see a monetary value stored in something else than integers I'm going to run away screaming (thank you Rust decimals represented as JSON floats). It's always integers unless you have a VERY good reason to do otherwise (though exported view can be in anything, even in weird bitcoded formats).

That really overstates the issue. Whole domains of finance run just fine on doubles.

If you're doing Monte Carlo options pricing over interest rate paths, and you're interested in the risk metrics, like durations, convexity, vega, and so on, no one cares what your rounding convention is. doubles are just fine, thank you. How are you going to force `exp(-rt)cashflow` to be an integer? Or the normal CDF?

Yes, there are domains where ints make sense. But it's certainly not universal, you just need to make the right engineering choice.

I love how this sensible take is followed by a tornado of comments that boils down to "NEVER use floats JUST BECAUSE".

Never use floats for financial calculations. Because hard won experience.

"Floats" are simulations of Real Numbers, and Reals are uncountable. Not what you want for finance, where everything is counted.

I suggest reading the comment I was replying to since it contextualises the answer quite well. Hard to find absolutes in real life outside of thermodynamics.

It can be really frustrating viewing threads like this sometimes. I've not once seen an interest rate swap priced in anything other than float/double, and that's relatively simple even compared to some of the crazy instruments out there.

Like, sure, probably don't use floats for everything, but what are the odds that your greeks are gonna be nicely expressable as simple rationals?

It doesn't matter for quant finance to use floats because approximate is perfectly fine. Tracking the exact amount to accounting standards is the job of the brokerage team not the quant devs. Floats are fine for modeling as long as the numerical drift is accounted for.

Basically whenever you are acting as a custodian e.g., a bank. Then you have to be super careful.

No one cares if you’re mortgage calculator is off by a penny

Quick browse over Stripe API (i.e. not a simplistic payment provider) will show exactly otherwise.

Metrics/durations/convexity/vega aren't monetary value, these are calculation values and I'm fine with them being whatever. If you're calculating averages to make decisions based on this I don't care if you're counting dolars or tomatoes.

Monetary value is $4.50 and it's representation of actual money passed through system. In practice this value is almost never undergoing any operations except for addition and subtraction. Any multiplication/divison etc. is left to specialized systems that return results wrapped in 800 pages of standarization documents, regulations and so on; systems where term "bit flip" is not a joke, but a risk.

As I wrote fintech != fintech, and I know many organizations aren't at that level of scrutiny, but a huge domain of people dealing with others' people money know that charging someone 1 cent too much is a tens of thousands of problems coming their way.

What's your preferred solution to dealing with FX conversions?

My issue with using integers everywhere is that FX conversions (or other rates) always come into play, and at that point I'm forced to use something else anyway (e.g. arbitrary precision decimals).

I never had to deal with that. At one place it was so complicated that specialized system took reconciliation over (it got baggage full of context data including exchange agreement details, with boundaries and time-regions plan).

In not-a-fintech we just went with doubles and rounded up (worst case - we get a cent more of a customer).

Though if I had to design today for that, I'd look for non-string serializable decimal, so not "10.123 == (10, 123)" but more like (10123, 3) and serialize in JSON as a value and precision separately.

Yet that's only cause I saw Decimal(10,123) sent in JSON as "10.123" which JSON reader red as 10.122 float and inserted to "1012" to database.

I worked on a trading system dealing with transaction limits expressed in USD but trading in many currencies.

FX rates and prices were expressed in fixed point because it was more efficient to handle in FPGA. A lot of thought went in to making the system accurate enough based on real FX rates while falling on the right side of the limit

> you're interested in the risk metrics, like durations, convexity, vega, and so on, no one cares what your rounding convention is. doubles are just fine, thank you.

Doubles are not just fine, but required there. If you compute risk, as you allude to, by finite differences, then you need the precision of floats rather than rounding to integer cents prematurely.

Rule of thumb here is that investments will work fine using doubles, but transfers need to run on int or fixed point.

What are you referring to with the integers/floats comment? The article says clearly that the rule of thumb is not to use floats and that they’re “almost never” a good idea, that they cause unpredictable precision loss, and recommends integer or BigDecimal types in multiple places. Are you also talking about rationals? So what is the bad advice here, exactly?

For FX, it seems like you’re reinforcing what the handbook says, that there’s no canonical rate. Aside from that, it’s talking about post-resolution records and you’re talking about how to resolve, no? That’s valid nuance of a separate goal, and it’s a fine goal of yours, but doesn’t seem like a demonstration of something missing or wrong.

The article appears to make the very same point about immutability? What are you saying that’s different?

With integers/floats, he's saying it's not opinionated enough. Anything other than integers with minor-unit precision, unless you have a very good reason, is a bad idea. So "floating point is almost a bad idea" doesn't go far enough, and the other alternatives are presented somewhat equally.

The FX critique is saying that it's glossing over a lot of the complexity. I'd say the same is true for the treatment of DE ledgers, and it borders on bad advice (e.g. "Balance is never stored. It’s derived from the movements of money.")

> Anything other that integers with minor-unit precision, unless you have a very good reason, is a bad idea.

The article clearly communicates this sentiment, no? What else needs to be said? How much further does it need to go, and why?

It might be a mistake for either us or the handbook to be absolute or dogmatic about floats. It’s not a sin to mention that they exist, and it’s a fact that some people in fintech use them for some reasons that have a defensible engineering position and well considered tradeoffs. I’ve been on the side of assuming people don’t use floats for money and then been surprised when I bumped into people here on HN who report using floats in finance routinely.

BTW, is your quote “almost a bad idea” a typo? There’s a world of difference between ‘almost a bad idea’ and ‘almost always a bad idea’. The actual words in the article, if we’re quoting the article, are: “almost never a good idea” in reference to using floating point types.

> it’s glossing over a lot of the complexity.

Of course it is, that’s a good thing. It’s not pretending to be a spec or rules, it’s an introduction and general principles. The article is already introducing new complexities that people outside of fintech might not be aware of. But do we really have to mention ALL complexity? The biggest problem with Wikipedia is that it’s overrun by nuance and complexity, so much that you often can’t read an article on a topic without already being an expert on that topic. This is why experts are often bad teachers. Being unable to gloss over some complexity is not good for learning and doesn’t make a good environment for newcomers. Let’s allow people to write for non-experts and make room for learning. We don’t have to avoid glossing over some of the complexity; it’s useful to get the general direction and gist correct while leaving out some of the detail.

> it borders on bad advice

Be specific. What’s wrong? Note that contributions are invited.

For me quickly scanning over the article, the fact that floating points were even presented as a possibility was an immediate red flag. And I pretty much stopped taking the rest of it seriously.

"Monetary value must be stored in integers" is the much stronger statement that the article doesn't make. Obviously there are exceptions, but you're going to need a much longer side discussion in order to justify why you're using floats.

It's like saying "don't write your own crypto algorithm". Of course write your own crypto algorithm, that's how you learn about cryptography. But you'd never put your homegrown cryptographic algorithm into production until after several PhDs worth of understanding of cryptography has been put into it by many other people.

If you render a number using JavaScript it has been temporarily stored as a doublr. The article was likely allowing for that common use case.

>"Monetary value must be stored in integers" is the much stronger statement that the article doesn't make. Obviously there are exceptions, but you're going to need a much longer side discussion in order to justify why you're using floats.

It doesn't make that statement because arbitrary precision decimals exist. Binary float is not the only and not the most plausible alternative to using integers.

> it borders on bad advice (e.g. "Balance is never stored. It’s derived from the movements of money.")

Don't think that's bad advice, in particular as the article clarifies:

> Derived state can be cached. Balances and projections can be cached or snapshotted for performance.

Of course you can cache or snapshot. But don't mistake that with storing the golden source.

> It's always integers unless you have a VERY good reason to do otherwise

I don't really agree. This seems like one of those outdated "greybeard rules" which people love to cargo cult, but to me it just comes with its own set of trade-offs, like now you have to think about exponents everywhere and getting them wrong comes with orders-of-magnitude consequences.

If you have a modern DB and your languages handle its decimals well then I'd use them. You can translate as needed for imports and exports, but the core of your system is always sound. Switch to the more basic formats if and only if there was some perf problem with decimals, which for most fintechs will be a VERY long time with modern DBs.

What really pushes me towards decimals is the consequence of getting something wrong. Relying on comparing raw ints, with variable exponents baked in, can lead to catastrophic errors with multi-order-of-magnitude gaps if that implicit exponent isn't carried along correctly. It's a massive footgun always waiting to happen. Decimals avoid that whole class of problem. So the goal is to maximise the "safe" decimals as source of truth everywhere you can, and have very well-tested "in and out" paths for any producers or consumers with different representation preferences. To me, that's good system design. And when, not if, you're manually inspecting DB records to track down a bug, having everything normalized in a glanceable, obvious format like decimals will let you recognise errors faster and more intuitively.

This goes extra for crypto, especially stablecoins, where one USD stablecoin might be exp-6 and another might be exp-9. And you're representing them in the same DB! Off by a factor of a thousand if you miss that exponent! Decimalize that immediately says I.

> thank you Rust decimals represented as JSON floats

What do you mean? JSON doesn’t have floats, it has numbers, and how they’re used after being parsed is not part of the spec.

> If I ever see a monetary value stored in something else than integers I'm going to run away screaming

That’s good, then we’ll likely not be working on the same system :) I consider running from “amounts as integer” systems these days (but usually unfortunately can’t). In an idealized codebase that only seasoned financial programmers are allowed to touch, it can go well, but such a system is usually either overly exclusive or risks becoming brittle.

> I consider running from “amounts as integer” systems these days (but usually unfortunately can’t).

In the context of Fintech, how do you otherwise resolve floating point rounding issues if not representing amounts with integers?

Native decimal types, if your system has them. Many languages and databases used in financial contexts do.

But native decimal libraries are almost always floating point.

Do people not know what a floating point number is?

That’s alright, the important part here is that they’re decimal, not binary. You don’t want 0.1 + 0.2 to equal 0.300…004.

There are pretty trivial ways to use binary floating point values that don't result in 0.1 + 0.2 producing 0.30000...4 and it saddens me when this topic comes up and people go to such extreme lengths to recreate a second hand buggy reimplementation of a subset of floating point numbers to do it.

> There are pretty trivial ways to use binary floating point values that don't result in 0.1 + 0.2 producing 0.30000...4

Not across all architectures and operating systems there are not

Listen to those who have done this. Use integers for finance.

Yes across all architectures and OS's that use IEEE 754 floating points.

What are those pretty trivial ways? And how are they better than storing 10 cents + 20 cents = 30 cents?

By not thinking in terms of strict rules and dogmas and instead focusing on the actual problem you want to solve.

If you want to guarantee that adding cents together results in an exact value without any loss of precision, and you also want a tiny memory footprint and very high performance, then use a binary floating point to represent cents, just as you would use an int.

Also the question of how it's better depends on your use case and my argument is not that one representation is universally better than another, it's that money is used for such a diverse range of use cases that you need to actually understand what you're doing, what the goal is, what the potential issues are etc... in order to pick the right representation for your use case. At my trading firm we have three different Money classes optimized for three different use-cases (with functions to allow interoperability between them).

For the use case where the representation needs to use little memory, is fast, and needs to be used to perform complex financial calculations at an enormous scale, then you use binary floating point where a value of 1.0d = 0.000001 dollars. This gives you exact precision when working with cents, and lets you perform all of the usual financial computations that most quantitative applications need to perform with excellent performance.

We also have a use case optimized for I/O, where no calculations are expected to be performed but the data will be transmitted over a network or to/from a database etc...

If this isn't your domain, maybe you're just writing a GUI application/web app, then by all means use a big decimal or use an integer... I don't know, but it's very sad seeing how many people who are presumably professionals don't take just the bare minimum amount of time to think things through and instead just reason in terms of strict dogmas.

> you use binary floating point where a value of 1.0d = 0.000001 dollars

Is that actually better than just defining one dollar as 1.0? In my (pretty limited) understanding of floats, this doesn't seem to add any precision or prevent any edge cases.

Depends on your use case, once again. If your use case requires that arithmetic involving cents is exact, then yes defining 1.0d = 1 cent guarantees that, just as using the int value 1 = 1 cent guarantees that. Floating point numbers can represent and perform exact arithmetic on integer values, no loss of precision.

The original example was being concerned that 10 cents plus 20 cents should equal 30 cents exactly, not 30.000000...2 cents. Well if you set 1.0d = 1 cent, you get that exact result. In fact for any arithmetic performed on values greater than or equal to 1 cent, you are guaranteed to get exact results just like you get exact results using an int to represent cents.

Now if your use case does not require exact arithmetic on cents, then by all means stick to using 1.0d = 1 dollar. Evaluate your use case and decide on an appropriate scale to use by default.

Where the benefit of floating point kicks in, for certain use cases, is that you can continue to use the same representation with operations that don't work well with fixed precision, like interest rate calculations, financial derivatives, or more advanced financial modelling techniques and you get these operations at significant orders of magnitude faster than you would using decimal representations.

And I'm not talking like 20% faster or 50% faster... the difference between using a binary floating point representation versus decimal ranges from 10x to 100x, and the difference between using a double versus a big decimal is on the order of 1000x.

In order words, a financial report that can be generated in 1 second using binary floats can end up taking almost 2 minutes using a decimal representation, and 16 minutes using a big decimal.

Aren't decimal types BCD coded?

An integer for the value (scaled by number of decimals) and an integer value for the number of decimals. Different systems may use different values, even for the same currency or asset.

What exactly do you think a floating point number is?

Usually that, but in binary, which causes lots of headaches for financial math.

For those who don't understand what they're doing, yes. But those people end up having headaches and causing trouble no matter what.

So all the people who write financial software and avoid floats don't know what they're doing? That's would end up being just about everyone. You should teach them, you know so much more than they do.

All of the data we control (in the database, in our apis, etc) is integer cents. When we have to interface with a system that represents money using JSON numbers as dollars.cents, we parse or serialize it into an arbitrary precision decimal type. Hasn't been much of a problem.

This started out fine for us except that int32s were a bad idea and upgrading was painful

That's definitely valid, for b2c applications though a lot of the time you will never deal with numbers of that magnitude.

I'm sure there is something I don't know here, but how is working with integers "brittle"? The only issue I see is rounding down by default, not sure if that is even an issue or not. At any rate, it seems a lot less brittle than floats or bigdecimal style number classes.

The brittleness comes from the fact that the number of implied decimal digits per currency isn't always well-defined across all stakeholders and systems.

If you're only working in a single currency, there's usually no issue.

As a general rule, you always include the currency code (EUR, SEK, USD etc.) and if possible also the amount of decimals, when using minor units.

Currency codes can be found in ISO 4217.

Yes, definitely always include the number of digits, but at your system boundary you still have to pray that whoever you're working with isn't silently dropping that number and re-deriving it from their own, almost-4217-compliant currency database.

Redundancy can be great, but it's not a panacea, since it's not guaranteed to be used in an optimal way.

How you expose you present your data to third parties is not how you represent it internally.

Ideally it’s not, but I’ve seen it leak so often that I’d rather be safe than sorry.

ISO 4217 also defines the number of significant digits after the decimal separator if Wikipedia is to be believed.

> JSON doesn’t have floats, it has numbers, and how they’re used after being parsed is not part of the spec.

I think that's the problem they were trying to describe. Without a formal spec, systems won't agree on how to handle floats. JS engines treat numbers as 53 bit signed floats, so passing a well defined decimal there through JSON means losing precision at the edges.

Money stored in integers gets around the issue by simple virtue of not really needing more than 53 bits to accurately represent the values anyone is going to encounter.

There are downsides like all the extra math or functions to handle doing the math everywhere money is manipulated or displayed, but this is the sort of thing where static typing is really helpful, and isn't too hard for juniors to understand that they should always use money functions to work with money data.

With I need to represent monetary amounts through JSON, I encode it as a base 10 string and wrap it in quotes so that the JavaScript engines treat it as a string. Conversion back to int happens after the string has been parsed and validated.

I do this with HTTP GET and POST form requests as well. In HTTP, everything is a string (even if that string is JSON).

> JS engines treat numbers as 53 bit signed floats

Please do not use Javascript for finance applications. Just do not do it

Save it for user interface elements and dancing whizzimagigs, where it will do less harm.

Excel does the same thing, and has been used to track money for decades. Somehow, we're all still alive.

Financial applications need user interface elements too, and doing these poorly has caused quite some [1] harm in the past...

[1] https://financialregulation.linklaters.com/post/102j8af/the-...

You can't do everything you need with an integer. There are values you might want to display or calculate with that are smaller than cents. In some places you'll need things like BigDecimal, which are immune to floating point errors in most cases.

It's also safe to return decimal values for displaying values.

The integer `1` can mean whatever you want, it doesn't need to be a cent. Haskell's `Fixed` type is a good example of this:

https://hackage-content.haskell.org/package/base-4.22.0.0/do...

Its a wrapper around an `Integer` where you declare the scale in the type. So if you use `Fixed E2` as your type then `MkFixed 1` is 1 cent. If you did `Fixed E3` as your type then `MkFixed 1` is 0.1 cent. In both cases it is entirely an integer encoding.

But it's a good idea to keep those values distinct from actual money. You can't pay a fraction of a cent to anyone.

But the price of some unit of a commodity may well be expressed in smaller fractions to allow for small price changes.

You could argue that a price is not really an amount that anyone can actually pay and that's true. But it seems complex and error prone to store unit prices and amounts using different scaling factors.

If you're calculating stock gains, taxes, etc. You'll want to do it on BigDecimal on line items, then sum it up. Cents could lose precision leading to a large difference on millions of line items.

If you're doing static payments or displaying a number, it doesn't matter if it's float amounts or integer cents since you're not doing any calculations anyway.

Stripe uses cents mostly because it's the most foolproof way that won't cause massive issues as a public API.

That was the first thing that popped out and made me distrust the whole wiki; there's only One Right Way to store money (as integers[1], as you said) and it should have been explicit about that.

You can also use fixed-point if whatever you're using supports it but it's still technically integers.

Reply on news.ycombinator.com