Hacker News Reader

I think K nearest neighbor is an underrated learning model.

Assuming you can define a good distance measurement how cool is it to use every piece of data. And with no training.

I wonder how they compare against tree based models on tabular data just in accuracy.

I like using KNN as much as the next person but tabular data can be quite wide, and KNN suffers from the curse of dimensionality.

The next course of action would be to split your distance calculation into multiple steps - a few dimensions at a time.

Then we need to decide on an order to these comparisons - what about using entropy derived from the training data?

Then as a further improvement, what if the order need not be constant, depending on which group the data point is closer to? And what if not all dimensions need to be considered?

Before you know it, woah! You've reinvented trees!

This comment is meant as a joke

That’s a great way to explain it though.

Reply on news.ycombinator.com