Designing Machine Learning Models: A Tale of Precision and Recall

nerds.airbnb.com

17 points·kilimchoi··5 comments

How does one write an article which summarizes the process of building a machine learning model, and not discuss cross validation? Not a very good article, to be honest.

They hint at withholding data ("The same logic applies when it comes time to splitting our data into training and validation sets"), though they don't outright mention cross-validation. Seems like a bit of an oversight for an article about building ML models, I agree, but the article seems like it's mostly surface-level.

I come from the world of biostatistics, where diagnostic tests are usually measured in terms of Sensitivity (probability of Predicting Evil, given actually Evil, same as Precision) and Specificity (probability of predicting Not Evil, given actually Not Evil). What is the reason for choosing to balance Precision/Recall, versus Sensitivity/Specificity? They are definitely similar, so why prefer one tradeoff versus the other?

Different domains name the same thing differently.

Really the entire ROC curve should just be generated and a fitness function applied to chose where along the ROC curve is the most profitable.

As an aside, I'm currently chilling in a AirBnb. Love their service. Half the price of a hotel and at least twice as good. Feels like a home away from home. Most of the hotels I've been to smell like chlorine and have an air conditioner that is so loud I just turn it off, even in the summer.

Reply on news.ycombinator.com