Practical Text Classification with Python and Keras | Hacker News Reader

Practical Text Classification with Python and Keras

realpython.com

60 pointsendlesstrax7 comments

Neural networks often perform worse than bag of words for text classification. They compared to a bag of words baseline, but they didn't tune the regularization parameter, which is important.

They should probably have also tried tf-idf and n-grams.

Definitely not true. Transfer learning a model trained on a corpus then fine tuned on labeled classes is current SotA approach [1].

[1] http://nlp.fast.ai/category/classification.html

You mention a single counterexample, but is it "definitely not true"? I think there is a strong publication bias: papers will report when they improve on the baseline, but that's likely not representative for the common case (e.g., limited data, no pretrained model available, no time for extensive parameter tuning).

I'm ignorant. I thought "bag of words" was a representation (the idea that a sentence has no grammar, it's just represented by its words as columns, maybe weighted by tf-idf who knows), not a specific model.

Where can I read about this bag of words?

You're right that it is a representation, but also an instance of the vector space model of language. Coupled with a linear model for prediction it is a strong baseline for text classification problems. See e.g. http://scikit-learn.org/stable/tutorial/text_analytics/worki...

So "bag of words" = "count/tf-idf vectorizer + logistic/ridge/lasso regression"?

Also: a vector space is a set of things that can be added and multiplied by a scalar. So a vector space model should be the proverbial representation where "queen - woman + man = king".

Am I being an insufferable pedant? I follow text analysis only very lightly and keep losing the thread.

Yes, in the context of text classification a bag of words model will refer to that, or combined with some other linear model like linear SVM or naive bayes.

The queen - woman example is when you try to make a model of word semantics, such as with word2vec. In a document classification task the vectors represent documents.

Reply on news.ycombinator.com