It's confusing because it is very difficult to look at a decimal number and know whether it can be represented exactly as base-2 floating point. It's especially confusing because you get no feedback about it! Here is a Ruby session:
irb(main):001:0> 0.1
=> 0.1
irb(main):002:0> 0.2
=> 0.2
irb(main):003:0> 0.3
=> 0.3
irb(main):004:0> 0.4
=> 0.4
irb(main):005:0> 0.5
=> 0.5
They all look exact, right? Well actually only 0.5 is (IIRC?). All the others are actually approximations internally.