Gisgraphy – open-source geocoder based on open data
gisgraphy.com
gisgraphy.com
I can't stress enough that it should be one of the job of the state to do this or make this happen (e.g. in France we have https://fr.wikipedia.org/wiki/Base_adresse_nationale which, restrospectively, is probably why I never had any problem with addresses in France).
In the worst offenders, you have UK, where Royal Mail charges you for postcode data [0], or China where private mapping is simply illegal [1] and you have to go through some state-approved cash register (i.e. NavInfo).
[0] https://www.poweredbypaf.com/licence-our-products/licence-ag...
[1] https://en.wikipedia.org/wiki/Restrictions_on_geographic_dat...
That is a very bold statement? Do you have a source for that?
https://geo.api.gouv.fr/adresse https://www.data.gouv.fr/fr
But you're right, other countries could do as well, in their language. It's hard to be aware of all the initiatives.
Sibling parent already mentioned BAN, but you can also check cadastre [0] which is the main source of polygon data for France on OpenStreetMap. Under the general subdomain [1] you have access to all sorts of stuff, but geography wise, there are plenty of geotagged aerial photographs, both historical and current. You also have shapefile data for towns, regions, postal codes, you name it.
No shenanigans, all official, verified, and regularly updated.
[0] https://cadastre.data.gouv.fr/
[2] https://datanova.legroupe.laposte.fr/explore/?sort=modified
I guess it is groundbreaking for most countries.
It would be nice for property prices to be included the central government data. It is free and publicity available but from each municipality or many websites that republish the data along with analytics.
Actually, the UK is kind of a funny situation. A number of years ago the Ordnance Survey began publishing postcode data under a free (CC) licence. Royal Mail wasn't too happy, but as Royal Mail's APIs were a bit simpler and more direct (OS service was SPARQL-only) so I guess they didn't feel threatened enough to change their pricing. I worked for a company that had an account with Royal Mail at the time and I was tasked with parsing OS data.
That's many years ago and now that I revisit the OS offerings have changed. They appear to have relicensed, and split their offerings between a free db[0] without full addresses and a commercial db[1] with full addresses.
Either way, Royal Mail may have the established customer base but I wouldn't look to them exclusively to see who's "leading the way" in this space in the UK.
[0] https://ordnancesurvey.co.uk/business-government/products/op...
[1] https://ordnancesurvey.co.uk/business-government/products/ad...
It can be accessed as a map layer at http://geopunt.be/ or as open data at https://overheid.vlaanderen.be/informatie-vlaanderen/product...
On the other hand, having lived in a country where such databases were notoriously unreliable, such automated processing didn't exist, so I didn't encounter that problem.
Sometimes, having something work "way too well" can lead people to believe it's perfect, and enforce it too strictly.
Are paid options any better? Like the one from Arcgis? Or is google still more accurate?
What regions? I work in GIS and have tried Nominatim in several projects, and it always ends up being so error prone we have to abandon it.
But that's for cities, in anything rural/mountain Nominatim takes the lead. Google seems not to care for addresses that aren't residential or a business. Arguably most geocoding uses are shipping things and business info, so most commercial efforts go that way.
Then some areas are just abandoned by Google. When I lived in Ukraine I tested some addresses and about 1/10 of the requests didn't work or were just wrong, and that's leaving aside the fact that it always showed 01001 as postal code on every reverse geocoding within Kiev city that I threw at it. The only information that was reliable was business data, and mostly because of businesses themselves updating it. The transliteration was software based and often wrong. Russia and Central Asia are also a big mess, though big cities tend to have OK data, but my feeling is that Google dropped the ball against Yandex and they just don't care anymore; not their turf.
It might have changed for Ukraine since, since there has been a push to drive away all Russian techs, maybe someone can shed some light on the subject.
I had been dabbling with the idea of a geocoding service that is highly accurate based on open data, but gave up because data quality is all over the place even for countries with excellent open data access (France being the gold standard in my book). I found that to have accurate geocoding I had to have an ETL stack that pulled data from 2-5 sources per country, with country-specific code for basically every country, and that's just to store usable data. Even keeping it to the major economic powers felt too daunting, nobody wants a partial coverage geocoder and I don't want to be a one man data janitor gig, which is what the whole thing felt like. And still large swaths of countries are totally uncoded for all sorts of reasons.
A quick search shows some of the problems I found in my quest, and some new.
1. Wrong postal codes for certain addresses, e.g. Place de Gaulle, Antibes, shows 06160 but it's 06600. It's probably taken from cadastre or nearest OSM tags, but it's wrong, different parts of the town have either of the postal codes. For France, the postal code relations/polygons from OSM are accurate, just don't take postcode info from anywhere else, really, unless for some reason your point falls within France but somehow out of any postal code polygons.
2. Missing addresses. For France, I did some analysis on open data to find what street names were the most common, and found ~15k steets/squares/etc named "de l'Église", due to basically every town having at least a church and naming their street/square after them. Not sure if a problem with the data or with the parsing, but it only shows a few dozen.
3. While OpenAddresses is doing an amazing job, I only found the coordinate data to be accurate, and not always. Most of the fields in their own JSON are empty (postcode, district, region, and sometimes even city), but Gisgraphy takes them at face value, e.g. first result for Улица Деревянко is just that, it only shows street name. Also the coordinates are broken: 0.140642,34.6784995 falls either in Kenya or Algeria (coordinate swap is also rampant in many sources, which is a another topic). That is the official data from Kharkiv, Ukraine, which probably geocodes the locations in a different system than WSG84 for, you know, reasons. OA tends to focus on easy to process data, and can't blame them, since the state of the whole geocoding world is an absolute clusterfuck.
4. Related to 3, OpenAddresses only has (had?) two towns from all of Ukraine (Kharkiv and Dnipro) with really terrible data. But OSM data is also fed to their model, so data overlaps, sometimes with different scripts, or languages. So the street above, "Улица Деревянко", which is in Russian btw, appears again as "Дерев’янка вулиця" which is Ukrainian. The whole l10n/i18n part of geocoding is another tough problem to solve, because letters written to Derevyanka Vulitsa will arrive as well, transliteration data is spotty at best, but for many languages it can be transliterated through software, again, depending on many factors.
That said, I'd love to work on a project like this as part of a team.
That said, their jobs page doesn't have much for now, but you may want to keep an eye on it.
Disclaimer: the founder of the company and I are acquaintances, but my assessment is only based on the quality of their service. I've been using it in production for reverse geocoding for a few years now.
Ed from OpenCage here, thanks for the kinds words! It's true we don't have any open positions right now. But anyone who is into geo stuff in general and geocoding specifically can dive in to OpenStreetMap and the open source libraries we (and many others) rely on and contribute to. Most notably Nominatim https://nominatim.org
Here's a podcast interview I did last summer with Sarah Hoffmann, the lead maintainer. https://thegeomob.com/podcast/episode-35
https://www.fossjobs.net/ https://github.com/fossjobs/fossjobs/wiki/resources
While your API service is stellar, if not the best with open data, unfortunately the data quality is always the limit and one can only extract so much from it. While I noticed a lot of sanitization when running some queries, it didn't take a long time to find hiccups, mainly because I know the types of warts open geo data has.
But from my quick tests, there are two issues.
1. Spain. Like, the whole of it. OSM Spain is lacking a lot of number information. Even Madrid (city) alone is missing a lot, and some reasonably large towns are basically unnumbered. E.g. 40.309452, -3.730451, the whole of Getafe (180k people) lacks numbering.
All that information is available in the catastro, but names are often shortened, missing prepositions, lacking accents ("Calle de la Pasión" becomes "CL PASION" in the catastro) and is a horrible mess overall with no 100% proof way to cross correlate data, but here I don't see any cross correlation happening at all.
2. Searching for "Place de Gaulle", because it's a solid no-strange-characters way to obtain an endless supply of points within France, shows a mysterious result at rank 10: 47.63341, -83.04979, in the middle of nowhere, ON, Canada. No info whatsoever. Why would that rank that high, vs thousands of French counterparts? It doesn't appear in Nominatim either, nor in any of the datasets I've worked with; not sure where that comes from. Now I am curious, what's that?
thanks for the kind words.
You are right that a geocoder is only as good as the data available to it. Happily OSM is great for many use cases and getting better literally every day.
Whether it is good enough now for your use case will depend ... on your use case. Not everyone needs comprehensive house numbering in Getafe. Until the local OSM community decides to add those numbers we do the best we can for the use cases where open data is a viable option today. As an aside, I am not sure the catastro qualifies as "open" data (even if it may be public), and even if so, as you correctly note, someone with local familiarity for all the abbreviations and common usages will need to help with adding it. Local knowledge is key.
re: "Place de Gaulle", of the top of my head I couldn't say, I would have to a detailed look. It's complicated, which is what makes geo fun.
"It's worth noting the mass download service of cadastral information, available since 2011, that makes it free for companies and individuals said information, including the possibility of it being reused."
Translation mine.
I'd love to hear about the origins of such mysterious Ontario spot!
[0] http://www.catastro.minhap.gob.es/esp/usos_utilidades.asp
https://opencagedata.com/tutorials/adding-an-address-to-open...
re: Ontario, I will eventually have a look, but the list of projects is long and priority goes to bugs reported by customers.