Hacker News Reader

Comment by buckhx | Hacker News Reader

I'm glad to see more geocoding offerings, but if you need more than a few thousand addresses, you'll find no offering will. Running your own geocoding really shouldn't be that scary of thing to do.

The biggest pain in running your own OSM geocoder (nominatim) is kind of a bear to set up. I imagine there docker images that could make things a bit easier and allow you to grab a region extract, but I could be wrong there.

On a side note, I was working on an open source geocoder based on OSM that was easy to build as part of a mapping util I've been working on (https://github.com/buckhx/diglet). I lost some traction on the geocoding side of things and have been focusing more on map building/serving, but did get a stable version for US addresses. There are lot's of interesting problems that come along with geocoding.

View on HN

> Running your own geocoding really shouldn't be that scary of thing to do.

> The biggest pain in running your own OSM geocoder (nominatim) is kind of a bear to set up.

You said it shouldn't be scary then immediately call it a bear. I am scared of bears.

Here is the Docker image for the geocoding stack that foursquare uses, with all world data already set up (would take you about a day to set up yourself):

https://github.com/doda/docker-twofishes-geocoder

Why does it take so long?

Processing the raw data and generating a search index is quite resource (IO/RAM/CPU) intensive. Some geocoders (not twofishes) need 100GB of raw data.

Not sure about your first sentence.

I'm founder of the OpenCage geocoder, we'll gladly work with anyone large or small, we have clients doing millions of requests per day. We use OpenStreetMap but also other open geo data sets. https://geocoder.opencagedata.com

Looks pretty good. We currently use Google but would love to switch.

One thing our app needs is a way to only get street address matches. Basically, I already know that an address is a street address or at least a very small side road. It's not an entire county or city, and never a very long road like a highway. Google lets me filter out such bogus results by returning a "bounds" field in the results. Using this bounding box I can calculate the area, and ignore anything less than 200x200m, regardless of the type of feature returned (which is sometimes a "route" even though it's very small). I see your API returns a "bounds" in the results, but it's not documented. Is that what it's for?

I'm also looking for a good autocompleter. Basically, if my app is about Montana, then the user should be able to type "M" and get "Missoula" as the top hit within 50ms. Not Mississippi or Montreal or anything outside Montana. It should also be possible to misspell names, type partial street addresses and so on. Google's geocoding API is useless for this since it doesn't do fuzzy matching. They have an API called Google Places, but it has restrictions and doesn't seem designed exactly for this stuff.

The Google Geocoder API has viewport biasing[1], and will work perfectly for your example of typing "M" to get Missoula, and should should handle typos. Not sure what you mean exactly by fuzzy matching in this context. Note that you still need to create your own clientside autocomplete widget, and an average user would potentially require numerous queries (instead of just 1), which would hit your quota.

Places API autocomplete can be limited to only return results of type 'geocode', which will give you the autocomplete of just geocodable locations, and was designed for this purpose.

[1] https://developers.google.com/maps/documentation/geocoding/i...

We used to use viewport biasing, but I forget why it's no longer enabled; it's been a long time since I worked on this. I know we do use component filtering, and I can tell you that with "components=administrative_area:MT|country:US", typing "M" will yield "Montana, MT" and that will be the only result. You have to type as far as "Missou" to get Missoula as a result.

Generally, the geocoder's substring matching has limited range, which is why it is fairly useless for autocompletion. Its partial matching seems limited to misspellings or nearly-complete terms. For example, I just tried typing "mead", and asked the API for 10 results. I got 5 results, all things like "Mead Lane" or "Meade Ave", but no "Meadowood". So it stopped at 5 even though there are more sharing that prefix. If I type "mea", I get just _one_ match, about some random place with "Mea" in the name. In other words, it's doing something else (something more sophisticated, but less appropriate for this use case) than mere prefix/substring matching.

Again, it's been a while, but from the last time I looked, I recall the Places API being better for this type of incremental autocompletion, but it had some issues related to geofencing that made it almost as bad.

The only way to know if our service will meet your need is to try it. Please give the free trial a go.

We don't offer an autocomplete.

Geofabrik can set up a nominatim geocoding service on your own servers for a one off fee, if you don't want to do it yourself. http://www.geofabrik.de/services/server.html

Just wanted to call out that we regularly process datasets in the 10s of millions and 100s of millions range at geocod.io. Our service was specifically built with batch geocoding in mind.

Reply on news.ycombinator.com