When you're doing floating point arithmetic on a computer, it will approximate and round certain values in ways that don't match the way humans do it when they're, e.g. doing accounting.

So you need to run a massive physics simulation really fast? Yes, floats are great.

You need to calculate taxes on a massive corporation's fiscal year? Bad idea.

Some libraries advertise "arbitrary precision", many computer systems have a "decimal" type intended for currency, etc. and then they won't make all the same mistakes, but as the OP said you still need to control rounding rules and make sure they match the law.