Well, every base has issues with non-termination. In base 10, we have problems like 1/3 = 0.33333... . Base 2 has its own set of non-terminating decimal expansions, including 1 / 3 = 0.01010101... and 1 / 10 = 0.0001100110011... . Since we only have finite storage space, we have to cut the repeating string of digits off somewhere, no matter what base we're in. This can cause rounding issues.

You can't dodge all your rounding problems by changing base. There are binary-coded decimal (BCD) systems that store numbers as strings of decimal digits. You can generally find these in calculators (ever notice that a TI-84 overflows after 9.(9)e99) or some financial software. However, you'll still have problems like the 0.1 * 0.2 issue in binary floating-point. For example (assuming shorter-than-usual numbers):

2/3 = 0.66... ~ 0.66667

2/3 + 2/3 ~ 0.66667 + 0.66667 = 1.33334

However, 4/3 = 1.33333... ~ 1.33333, which isn't the same result.

Basically, you can't get away from this problem, you can only push it around to cases you care less about.

(You probably can't find an API for binary-coded decimal in $LANGUAGE unless $LANGUAGE is often used for tasks where you really really don't want any float-related gotchas since most everyone else doesn't care too much).