Hacker News Reader

Comment by jcrawfordor | Hacker News Reader

My husband built https://wikimap.wiki which shows every geotagged Wikipedia article on a map at once. It was actually quite complex to do this because of the large number of articles. The UI makes some compromises to try to keep it usable but it's a tough nut to crack with the highly variable density. It also has filtering by category and some features of that nature.

The major trick is that the article icons are a layer of prerendered tiles; the client can't handle that many objects. Clicking does a request to a geojson server backed by postgis to find what's in the region you clicked. I'm not sure that it's actually updating right now either, ingesting updates takes hours and is pretty brittle because of issues with the format of the dumps Wikipedia provides.

View on HN

This is impressive. Very cool.

Poking around a bit, I noticed that some of the geotagged articles are from other planets. For example, north of Alaska are a bunch of craters from Mercury. Makes me think that an interesting feature could be choosing which planet to render the map from :)

Unfortunately I don't think there's any structured way to determine which planet an article applies to. Broadly speaking, Wikipedia geotagging is inconsistent and the data quality is fairly poor. I think a lot of that is because it's surprisingly difficult to make a viewer like this, so few people have. That makes the data quality issues nonobvious.

Wikidata offers the promise of more structured data, but I know my husband experimented with it and decided not to use it. I think the problem was that mapping Wikidata objects to Wikipedia articles was too difficult (there is a structured way to do this but in practice very few articles have the annotation).

The globe: parameter should give you the information you need: see https://en.wikipedia.org/wiki/Template:Coord#globe:G for full documentation on this.

Awesome! But it should probably include local wikipedias as well, as it's pretty thin outside the US/GB

What about Canada, Ireland, Australia, New Zealand, South Africa, India, ...? (If you want language-independence, there's Wikidata.)

I think you misunderstood me

I mean, what are the "local" Wikipedias that would improve the coverage in the countries I listed?

I'm still unsure what you're getting at. But what I suspect: if there is no local wikipedia version (e.g. the German wiki for Germany), then you possibly can't increase the number of data points, can't you? All depends on there being a Wikipedia in the first place.

I suppose I didn't (and still don't) understand why you singled out US and GB. Here's my take of the topic:

1. Wikipedia editions are not defined by region but by language, and the relationship between regions and languages is many-to-many. English is spoken (as a dominant language) in US/UK/CA/IE/AU/IN/... among others and German is spoken in DE/AT/CH/IT/... among others.

2. English is one of the language editions. The edition is special mainly in that it has the most diverse regional coverage.

3. All editions have an uneven regional coverage dependent of the speakers' distribution but also population density, literacy, social economics etc.

4. There is overlap between language editions so it's unclear whether given two their union would improve coverage and how.

5. Wikidata is the language-independent knowledge graph that includes the sum of the article metadata regarding all the Wikipedia editions (and more).

Now, a global map of Wikipedia articles in English is useful in its own right if you want to see what you can read in English. To improve the coverage, you would write articles about more topics in English.

However, if you wanted to see more dots on the map, you could base it on Wikidata to include all articles on all editions, or to include all Wikidata items even with no corresponding Wikipedia articles.

Whereas, if you wanted to see what you can read in German, you would need a map that includes the German articles and excludes the English articles.

And if you know multiple languages, you might want to see the union of those languages.

I start to think you may have in mind people who speak two languages: English and a regionally dominant language. If they wanted to see which articles they can read about the region, they could use a map of those two languages. If those two Wikipedia editions were significantly complementary, it would be a significant improvement in coverage for them.

Still, I'm unsure why to single out US/GB when there are other countries where English is the majority (or practically the only) language, there are other countries where the coverage in English is already fine, US/GB also have minority languages that could potentially improve their coverage etc.

>why you singled out US and GB

let's take him at his word, "it's thin outside the US and GB." OK, hmmm, how do we get more global coverage? oh, of course, France would be fr.wiki and Germany would be de.wiki, that would really increase global coverage.

now you come in and say "but what about Canada, Australia, etc? there is no nz.wiki et al?" And then you have some ideas about how to do that, ideas which are not strictly "wikipedia" as we see in the headline idea.

Perhaps a relevant question: Are you describing a business need, a (supposedly) good way to implemented it, or how are you combining the two here?

> OK, hmmm, how do we get more global coverage?

Like I said, combine all the Wikipedia editions using Wikidata.

> oh, of course, France would be fr.wiki and Germany would be de.wiki

And of course, what language is e.g. Belgium (in your model)?

> And then you have some ideas about how to do that, ideas which are not strictly "wikipedia"

In this strict case, how do you remove duplicates (between the different language editions)?

Yup, that's it, thanks! My list of English-speaking countries just wasn't exhaustive and were the first I noticed with a higher density than the rest on first glance.

The language picker doesn't just change UI language, it changes which language Wikipedia is displayed.

There isn't really such thing as a "local Wikipedia" but of course coverage for a given country tends to be best on the Wikipedia in the language spoken there.

Not every language is supported (I think processing the fifteen Wikipedia dumps takes several days as is) but I believe the supported languages are the top 15 by number of articles.

Cool service! You could get rid of prerendering as well as handle clicks client-side if you switched to MapLibre and vector tiles (e.g. tippecanoe). On small zoom levels it doesn't really matter, but I think it would further improve the experience of zooming in.

This is really cool. The thing that strikes me immediately is all of the shipwrecks.

It mistakingly incorporates coordinates from other planets/satelites in our solar system. Seems a large fraction of those point nemo locations are meteorite impacts on nearby satelites.

When looking through shipwrecks I found a mexican school https://wikimap.wiki/?base=map&lat=-5630577.9494&lon=2565608...

That's great! And, unlike nearbywiki.org, it correctly places the wiki articles (nearbywiki seems to be off by hundreds of meters for some entries).

That is a really cool method I would have never thought of. Did he have a lot of experience with this type of thing?

I like it. What is missing, or I couldn't find, is a "find my location" option.

This is great. I remember thinking something like this would spice up travelling.

Oh no, there goes the rest of my morning ;)

Fun site, give him a high five for me

Reply on news.ycombinator.com