You should have seen my face when debugging this.
Here's what I'm seeing:
---
JavaScript
(function() {
let inc = 0.37;
let times = 5000;
let total = 0;
for (let i = 0; i < 5000; i++) {
total += inc;
}
console.log('expected:', inc \* times);
console.log('actual:', total);
})();
actual: 1849.9999999997679
---
Java
public class MyClass {
public static void main(String args[]) {
double inc = 0.37;
double times = 5000;
double total = 0;
for (double i = 0; i < 5000; i++) {
total += inc;
}
System.out.println("expected: " + (inc \* times));
System.out.println("actual: " + total);
}
}
actual: 1849.9999999997679
---
update: Changing `double` to `float` in Java yields:
expected: 1850.0
actual: 1849.9778
and maybe that lines up with what you meant by "the differences should be a LOT farther than 3 digits", though it's hard to tell what you mean by "differences on the order of 10-20".
declare @i float =0.37,@times int = 0,@total float = 0
while @times < 5000
begin
set @total+= @i
set @times +=1
end
select @total
or
select sum(t) from(
select top 5000 convert(float, 0.37) t
from sysobjects a cross join sysobjects b) z
Do the sum two ways:
- export them in excel and do a sum
- select sum(value) from t
and you get two different values, with a difference of something like 15.
Replace double(10,2) with decimal(10,2), and the sum(value) works correctly.
I still have no idea why this happens. I could expect a tiny difference, and code like your above will indeed give differences like this, well under 1.0, but my particular scenario had differences many orders of magnitude higher. Still can't explain.
Oh, that's something entirely different. If some of your other numbers are much larger than 0.37, you will run into a different kind of problem.
You know about exponent and mantissa? Adding a value with a small exponent to a value with a big exponent will cause imprecisions. In extreme cases the result is just the bigger number.