Geocoder's api feels a little bit more modern Geokit (rails3 vs 2). Having read the code for both, Geocoder was cleaner and would be easier to tweak if necessary.

So really it's just a preference: they do the same thing.