I've used GeoNames (geonames.org) for a few projects.
Here is a JavaScript library I wrote for it, along with some scripts for munging the GeoNames dataset:
https://github.com/lucaspiller/offline-geocoder/
If you need higher granularity, take a look at this OSM project:
I think most free datasets are probably just the areas that Census derived for TIGER (or at the very least, rely heavily on that data).