Hacker News Reader

Comment by robryan | Hacker News Reader

Has anything that has built on top of wikipedia as the primary product got much traction? I think it is something that would be hard to really get widespread without it.

Can a 3rd party even maintain a fully up to date set of wikipedia content? Not sure what their policy on scraping is?

View on HN

Wikipedia also has an API (although this does little to address tptacek's concerns of database meltdown, unless you cache frequent requests):

http://en.wikipedia.org/w/api.php

http://www.mediawiki.org/wiki/API:Main_page

http://www.mediawiki.org/wiki/API:Query

For example:

http://en.wikipedia.org/w/api.php?action=query&prop=revi...

brings back:

   { 
     "query": { 
       "pages": {
         "1666533": {
           "pageid": 1666533, 
           "ns": 0, 
           "title": "Y Combinator (company)", 
           "revisions": [
             {
               "*": "{{Infobox company\n| company_name = Y Combinator\n 
                     [...redacted full wikitext content of article...]"
             }
           ]
         }
       }
     }
   }

No need to scrape, Wikipedia has full dumps for the articles available for download: http://en.wikipedia.org/wiki/Wikipedia:Database_download#Eng...

Yeah I am aware, but wikipedia changes daily. There would have to be a pretty impressive feature set built on top before many would consider using an out of date wikipedia when they can easily see the latest version.

Hm... for most pages it shouldn't really matter, they don't change drastically within days. For developing or current subjects I guess it would...

There is a feed of changes though - both a "realtime" IRC feed that is restricted access and some sort of batch feed (daily? Hourly?)

This dump is updated once a month, though the API is live.

One problem I've found with the dump is that you still need to render it (unless someone knows of another dump) which has caused us a few problems with the current thing I'm working on.

Reply on news.ycombinator.com