After I wrote my post about Apple's dictionary files, I got a mysterious email showing up in my inbox. The email was from someone who's spent some time writing code to do the same thing, but doesn't want to post it under his own name in case he falls fowl of his country's DMCA equivalent. Crazy. He said I could post his code under the condition that I took his name off it.
Indeed, it depends on the language and your goals - I had a very high success rate plucking out Russian grammatical tables from English Wiktionary with a few hours of scripting the data cleaning (https://github.com/thombles/declensions). I have a theory that you could get better results using an offline archive of the page sources but haven't tried this yet.
Q: Is it possible to download Wiktionary?
A: Yes. https://dumps.wikimedia.org/enwiktionary/ should have the latest copy of the main namespace. The cleanest navigation page is https://dumps.wikimedia.org/. Just download a -articles.xml.bz2 file and some software to read it (for nix, for Windows).
Q: Can I use data from Wiktionary in my program?
A: As long as you meet the conditions of the GNU Free Documentation License or Creative Commons Attribution/Share-Alike License, certainly.
Latest dump for English is from September 1. I wouldn’t know whether it has all the data or how easy it is to parse it.
It also seems to have some unevenness in coverage. For example, the entry for the word “anecdata” (a word discussed recently at [1]), has five illustrative quotations, which are quite handy [2]. The entry for the more mundane “anecdote,” however, has none [3]. Such unevenness might be inevitable in volunteer dictionary projects, as volunteers like to work on the more interesting words.
[1] https://news.ycombinator.com/item?id=28375767
We're translating some strings on our software user interface, and checking the abbreviations and acronyms used. Sometimes there are amusing or [nsfw] connotations in other languages! Thank you Wiktionary for warning us about abbreviating "low pressure" as "LP" in Taiwan.
I'm personally using the route of digitizing an existing dictionary, albeit manually because that can't be automated in this case.
A paper I wrote nine years ago on related issues is here, in case you are interested:
“Kokugo Dictionaries as Tools for Learners: Problems and Potential”
https://researchmap.jp/multidatabases/multidatabase_contents...
I browsed some entries of your dictionary and indeed their are (sometimes quite elaborate) explanations. I could easily why it ran out of steam, especially even a basic compiling is a daunting task. Also contributors are fews. On the Jibiki.fr (Japanese-French) project, most of the corrections are made by two members. They're starting from an existing dictionary and it still took years to just check the headwords.
Thanks very much for the paper. I'll read it. I actually already had one of your paper on my machine (Asialex 2011 proceedings) but haven't read it yet.