Hacker News Reader

Comment by tc | Hacker News Reader

I was trying to find a free SQL database of IP geolocation with country, city, region, latitude and longitude for a project but none was accurate and up to date so I decided to create my own and now I offer it for free.

He makes it sound as though this isn't particularly difficult or expensive. Does anyone want to speculate on the method he is using to compile this data? For his zipcode database he seems to be running mass queries on the Google Maps API.

Simply doing mass WHOIS queries to the regional registrars only gets you data about the corporate headquarters of the ISPs, which I suppose could be enough depending on the level of granularity that is acceptable to you.

(presently a similar question on his forum, posted by someone else, is unanswered: http://blogama.org/node/105)

View on HN

I checked his ip->city database against a dozen class C's with which I am intimately familiar, and in every case they were either correct or as correct as possible (i.e. someone in that /24 is located in city X, even if most people in the block are located 25-100 miles away). I specifically looked at cases where the ARIN information would lead one astray, so it appears that is not the primary source of location data.

Someone on reddit claims it's an exact copy of this:

http://www.maxmind.com/app/geolitecity

I haven't verified that claim.

From page (quoted in some other comment below): "Very accurate. The database is updated during the first week of each month."

This is also the frequency at which Maxmind releases updates. So...

.. so perhaps he's pulling the data from some of the same sources as Maxmind does and hence is on a similar update schedule, is what you're implying, right?

Anyway, I'd still like to know what those sources are.

Or perhaps the implication is that he's pulling some of his data from the Maxmind database. (Which would be illegal, unless I'm misremembering Maxmind's licensing terms.)

I'm guessing that you're merely pretending not to understand what's being implied, but I don't understand why...

This DB has close to six times as much location as GeoLite:

  wc -l ip_group_city.csv 
   1195012 ip_group_city.csv
  wc -l GeoLiteCity-Location.csv 
    238957 GeoLiteCity-Location.csv

If you want locations of zip codes or US cities, you can easily get that from the US Census's TIGER data.

The method: dynamic programming algorithm, triangulation and tracert.

I don't believe this is what Maxmind does; I have no direct knowledge but I believe their data is based on IP registry data (location of the registering org) along with inference from per-user registration data (surveys, social networks, panels, etc.)

Akamai has a (much more expensive) geolocation product that's based on doing triangulation, since they have a lot more information on routing (all those edge servers) but I'm not sure it's appreciably more accurate.

Sorry: I was actually joking. I should have capped with an emoticon. But it's cool to know Akamai actually has a triangulation DB: I didn't know it was feasible. So I gave you a karma point:)

Reply on news.ycombinator.com