The problem with floating point numbers is that most mathematical operations on them are approximations. People expect the same kind of correctness from floating point arithmetic that we've come to expect from integer arithmetic, when you really should be thinking in error bars.
I mean, just ask the average programmer to do currency math and I assure you they'll go straight for the `double`. Most of the time, not a catastrophic error, except for when it is.
For the curious, this is a long but very thorough overview of floating point arithmetic and it's quirks: http://docs.sun.com/source/806-3568/ncg_goldberg.html