NLP concepts with spaCy tutorial

I've always understood nltk as a teaching tool, and been accordingly surprised when I see people use it in production.

SpaCy really fills an important gap

Thanks! I'll answer this, since Ines doesn't have an account here (I'm Matt).

We're really happy with how Prodigy's being received. It's only been on sale two months, so I'm looking forward to hearing more success stories as people finish their projects (and of course, feedback to change what needs to be changed!).

You can read how FullFact used it to train claim identification models for fact-checking here: https://fullfact.org/blog/2018/feb/how-we-customised-prodigy...

Probably the best place to follow the progress is the support forum: https://support.prodi.gy/

We're also working on more tutorial videos. This video shows the workflow for training a new entity type: https://www.youtube.com/watch?v=l4scwf8KeIA . This is one of the bits of the tool we're particularly proud of --- you can start off with a couple of seed terms, use word vectors to build up a larger terminology list, and then turn that list into a set of pattern rules to start boot-strapping a classifier. Prodigy will suggest phrases that match the patterns as entities, and your answers are used to train the statistical model. As you keep annotating, the model will start suggesting phrases too, which you'll say yes or no to. Eventually the model basically takes over, and you're mostly correcting its suggestions.

You know, I'm still not 100% certain whether there's a substantive difference between the "computational graph" perspective and this "functor" approach. Actually the feeling is sort of eerily familiar, because I spent most of my PhD confused about whether these grammar formalisms I was working with where really just notational variants, or whether there were significant differences. About the grammar formalisms, I ended up deciding that in theory there wasn't, in practice there sort of was.

About these neural networks, I think it's "just" implementation. Here's the linear layer implementation in Chainer: https://github.com/chainer/chainer/blob/master/chainer/funct...

We have the forward and backward pass organized as class methods here, and the intermediate state from the forward pass is saved into attributes in the instance. So on each call to the network, we make an instance of this LinearFunction class.

In terms of what's being computed, there's really no difference between this and what happens when you call a layer in Thinc. It's just that the state gets captured in the outer scope of the closure. Maybe Thinc's way has a little less overhead, if there are fewer levels of indirection. Thinc uses the Chainer folks' GPU library --- so, unsurprisingly if you define the same network, the benchmarks are very similar.

On the other hand...I do think the implementation matters! Here's a difference for you: if the library approaches it as "we're going to build a computational graph, and execute it", then the library is going to steal the control flow. If the library tells you "here are some functions, and some higher order functions to compose them", you have more access. PyTorch and Chainer doesn't steal the control flow to nearly the extent that Tensorflow does, but they still build up and tear down the state in their objects, and that makes it harder to intrude.

(I'm the author of spaCy and Thinc)

This is a kind of tautology. Words and phrases exists not because of the whims of grammarians but because they are psychological realities for people. They are concepts represented in the mind and brain which exist whether or not people are explicitly aware of their existence--and this is demonstrated in a large psycho- and neurolinguistic literature. This is different from a man on the moon where we are talking about the anthropomorphism of bits of rock and dust--whether the ontology of such a thing is a human face or just bits of rock and dust.

I would be interested in the research you mention in case you have links handy.

At datamaran [1] we do some of that stuff for assessing non-financial risks with our internal ontology. Also, we are using spaCy and trying out prodigy. Send me an email and I can put you in contact with one of our NLP scientists if you want (they're planning on releasing a paper somewhere soon btw).

[1] https://www.datamaran.com

That's definitely deployed in production, e.g. to link entities to a summary page in news articles, or in information extraction systems.

In practice it's very driven by information retrieval, especially the coverage of the synonym list provided in the ontology. Relevance scores for each ontology entry are also super important.

Disambiguation is getting better with neural networks, but it's still really hard. If you have two entities matching for some text, picking the most frequent one gives you a very strong baseline.

There's a number of tools that do entity linking (that's the phrase you're looking for), including some open source ones.

We've evaluated and used quite a few over the years: there's Dexter [0] by Diego Ceccarelli, Semanticizer by UvA [1] and DBpedia Spotlight [2] and a few others. We've used them for various linking tasks, such as detecting "work skills" in plain text (HR domain) or detecting drug names (medical domain).

The amount to which these tools allow "customization" (ease of plugging in your own ontology, support for input format and disambiguation signals) differs. Either way, even though this research includes open source code, the code is more of the "research prototype" kind. Don't expect a plug&play optimized production tool.

[0] http://dexter.isti.cnr.it/

[1] https://github.com/semanticize/semanticizer

[2] https://github.com/dbpedia-spotlight/dbpedia-spotlight

> I prefer the approach of link grammar / relex which are based on dictionaries / grammars, it seems easier and less error prone.

Can you link to something open source that outperforms spacy on the basic NLP tasks (NER, POS, dependency parsing)?