Hacker News Reader

Comment by tony-allan | Hacker News Reader

I went down a bit of a rabbit hole and noticed that wiktionary.org has an API, for example:

https://www.mediawiki.org/wiki/API:Main_page

A bit off-topic, but I feel like Wiktionary is a hidden champion on the internet. Some time ago I noticed that I use Wiktionary more often than Wikipedia these days. I am not an expert in the space, but I get the impression that it's content (especially the multi-language aspect) outperforms any other dictionary in existence. It's an absolute treasure, without many of the problems and conflicts an encyclopedia cannot really circumnavigate.

In my usage, Collins dictionary is the best for explaining and giving background to English words. Wiktionary can't compete with that. I use it with an ad blocker and read all their articles for a given word.

IME their articles are written with learners in mind, too. I would probably count as a proficient English user that is still learning new words (well, who does not).

Arbitrary word for example: https://www.collinsdictionary.com/dictionary/english/gremlin

Wiktionary is great. I use it many times every day - for languages I'm learning, but also for learning more about my mother tongue. I've never been able to figure out the mediawiki API but had success just scraping/parsing wiktionary HTML files. For individual languages there are normally standard templates which structure in a nice machine-readable way (much of this can I think in principle be accessed directly with an API but it's in my experience hard to discover how to do any given task with the API vs having the html right there in front of your nose begging out to be parsed).

I've generally found that things like the part of speech type is inconsistent, so the wiktionary files require a lot of work to extract the information from them if you want to use the data for natural language processing tasks.

I've also found that things like thesaurus/synonym information is incomplete.

I don’t find the definitions to be as good as e.g. American Heritage, though.

The Mediawiki API is pretty powerful.

The annoying thing about Wiktionary (for this purpose) is that it's explicitly not machine readable, so the entries can only really be handled as free-form HTML. Parsing the Wikicode, despite heavy use of standard templates, would be very hard to do robustly.

Once upon a time i had a project to do that. and it was miserable (but also it was when i was first learning to program... so i did a lot of stupid things)

Wikidata has lexemes now, which, like the rest of Wikidata, has great potential that's largely ignored both in general and by the parent organisation.

Yeah, we'll see how the lexeme thing goes. There is also the earlier (non-wmf) omegawiki project that tried to do something similar.

You might find this other api more suitable: https://en.wiktionary.org/api/rest_v1/#/Page%20content/get_p...

Much better than the API I found. Thanks.

Reply on news.ycombinator.com