Hacker News Reader

Comment by ks2048 | Hacker News Reader

ks2048

I don't have a problem with having paid special services, but the "machine parsable formats" is a bit troubling since I think that should be a core part of the open wikipedia project.

I submit this link after coming across this site while Googling for info on parsing wikipedia "infoboxes". I plan to check out their "Article Structured Contents (BETA)" API. Improving infoboxes to be machine-readable seems important. And it would be bad if didn't do this because it's a revenue stream for them.

View on HN

Machine-readable infoboxes are surprisingly painful because an "infobox" isn't a top-level thing in mediawiki. Rather, it's something that emerged from community practice -- a template that's slightly-standardized, and has a sprawling network of subtemplates for specific categories of article. Then the assorted other language wikipedias have their own variants of these templates with their own names, etc.

Pulling it into a mediawiki extension (or core) and making it part of the page-level metadata gets suggested pretty frequently, but it's a bit contentious amongst the hardcore editors who'd need to actually adopt such things. The template-based nature of the current infoboxes mean that they're very accessible to the community, and it's easy to spin off new variants or make changes without getting programmers to help you.

There's slow movement towards getting the sort of data that winds up in infoboxes into wikidata, but it's still somewhat spotty.

(If you've never done it, it can be quite edifying to install mediawiki for yourself and seeing how much of the surrounding infrastructure of wikipedia is absent because it's all templates.)

I once tried to export a simple infobox template with dependencies and it exported 30-40 different template/module pages.

There’s also Wikidata which has machine readable data for everyone

https://m.wikidata.org/wiki/Wikidata:Main_Page

> Wikidata is a free and open knowledge base that can be read and edited by both humans and machines.

> Wikidata acts as central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wiktionary, Wikisource, and others.

Yes, but it is different dataset, many(majority?) infoboxes are not in wikidata

DBPedia has been providing machine-readable info drawn from Wikipedia infoboxes for a long time, and that's absolutely an open project. But there are drawbacks to that approach, nowadays Wikipedia users are broadly expected to input that info into Wikidata (a language-independent project, providing its data under the CC0 license) and the Wikipedias are gradually moving towards fetching the information from there as well. The English Wikipedia is somewhat behind on that, so DBPedia might still be useful for parsing info from it.

Just fyi, i think the plan for machine readable infoboxes is stick the data in wikidata, which has its own dumps in machine readable format as well as a publicly accessible SPARQL query endpoint.

Its actually quite cool. If you have never played with it i encourage checking out some of the example queries (there is a button labelled examples) on https://query.wikidata.org/

Reply on news.ycombinator.com