Hacker News Reader

If the features are independent, it is better to use NB, because then you don't need to find it out the hard way. So your hypothesis of maxent being always better is flawed.

Moreover NB can be seen as a joint maxent method, so don't compare it on problems with conditional structure. This website explains nicely the connection between them: http://www.denizyuret.com/2010/11/naive-bayes-is-joint-maxim...

View on HN

Assumptions such as independence are always dutifully mentioned when discussing statistical methods, but I never see a discussion about in which sort of practical problems such assumptions hold. In the case of independence and text classification, it seems to me that it doesn't hold, certainly not on the scale of a paragraph. As the blog post mentions, naive Bayes performs well despite violating the independence assumption; I'm curious to understand why that is.

In practice, I would say that it is a very challenging task to determine whether variables are totally independent or not: It is more about the strength of independence. Every time something is "estimated", you'd introduce an additional source of error, and in cases of weak independence the violation may be negligible. Modeling the conditional dependences is basically the idea of Bayesian Belief Networks, and you might find the paper "Comparing Bayesian Network Classifiers" interesting in this context (includes comparisons to Naive Bayes) http://link.springer.com/article/10.1023/A:1007465528199

Reply on news.ycombinator.com