And correcting the invoices meant playing with numbers so that at least the PDF and accounting software agreed on the total value and tax.
She also said they had at least 2 different employees and an external company look at it and not able to fix it. She also told me not to bother because she does not believe the problem can be fixed (that's what she was told).
I looked at the software, it had two separate copies of the invoice calculation (separate for on screen and for printing to PDF). And of course it would send the invoice to the accounting software which calculated the invoice in a different way still.
I ran couple of experiments to reverse engineer how the accounting software did the calculations -- the exact order of them and the exact rounding rules. Then I built a small module that captured those calculations. Then I changed all doubles to arbitrary precision.
It took two days and the problem was fixed but it took couple more days before accounting department actually believed it.
That's actually kinda normal, in some industries.
For example, an amazon marketplace seller's warehouse management system might not be tightly integrated with Amazon's basket/checkout display logic.
In some situations the results of recalculating are supposed to be different. For example, if there's a "5% off when you buy 3 widgets" offer and you check out with 2 widgets in your basket, the offer doesn't apply. But if you checked out 3 widgets, thus getting the offer, then the seller found they were low on stock and could only send you 2, you should get the 5% discount on those 2.
Those 3 people that now didn't had a job probably weren't all that happy xD
Your setup made it sound like something crazy and inane, like "the PDF printer used on those machines changed floating point rounding modes" or what have you.
In this particular case we had two separate pieces of code, one running on the client (for on screen presentation) and one on the backend (to create the PDF on a shared location and produce a download URL).
My career advice is to work in a field / company / job where you can be somewhere in the top 10-20% of all employees. Just don't overdo it, if you are top 1% you are probably aiming too low and could be working for better paying, more rewarding field / company / job.
For a lot of my career I was working for financial institutions like banks. A lot of really badly managed projects with definitely not top level developers. Easy to be a top performer. I really like helping people and projects and it was working well for me especially when it was easy for me to provide valuable help.
I got hired once for a really good company with really top performers and suddenly I lost the status that I was so used to. I was keeping up with my work, sure, but I was no longer a shiny star. I got back to working for banks.
I run a Commercial Real Estate Servicing platform, where we are accruing interest on large balances daily. Our method is to not do the rounding daily, but add up all the numbers for a given period, say a month, and then round to the penny and create a single adjustment rounding transaction along with it. Accounting departments love us for it.
If we rounded daily before storing the amount, the adjustment for accounting is usually a few pennies at least every month they have to make. Our method, it's roughly $0.01 per year with monthly periods, adjusted usually at the very end. Which on a $20MM loan, is very well within the bounds of acceptable.
I brought up specific math problems that floats couldn't handle and they weren't phased
Or it is because rounding errors happen and accounting is a bitch. The first option makes for a better movie plot so.
AGI is 40,001 to 40,025 then your tax is X dollars.
Being accurate to the penny isn't worth the trouble.
round in favor of the bank / financial institution you are working for
You issue an invoice for 1.44 USD (aka, amount due), then the 1.44 USD is used as a basis for accounting and is all consistent.
Then, if you are a nice company and the situation applies in your case, you may issue a credit in favor of the customer for 1.44 USD - 1.433 USD that will be used as a discount on a future invoice
The best part is that the moment where you decide to issue the credit invoice or not, is the perfect moment to track the rounding errors and even keep a very detailed journal of the entries (e.g. for auditors).
A good accountant so will sooner or later investigate those rounding errors, as they will show up somewhere ultimately. And a general policy of rounding in one direction is the last thing you want an auditor to find.
Eg: fuel is usually priced 3 decimal, so 4 gallons x 25.5444 usd/gallon gives = $102.1776 to 4 dp, but will be billed as $102.18
[1] https://globalmarkets.cib.bnpparibas/app/uploads/sites/4/202...
Where issues could come in is when these things are multiplied with a bunch of other numbers (each number with defined rounding but not after each operation) and then have some defined rounding at the end. There different computer numerics could in give slightly different results, but those can easily be resolved on settlement (for small stuff that stays well within the back offices - at least that is how I remember it).
Also, not totally unusual for one or both participants to forget about some rounding they might have agreed bilaterally if it was some one off etc.
I assure you, I would have had a million bugs filed on that before it even hit production.