Hacker News Reader

Comment by tptacek | Hacker News Reader

Is k-means really what people are doing in serious production machine-learning settings? In a previous job, we did k-means clustering to identify groups of similar hosts on networks; we didn't call it "machine learning", but rather just "statistical clustering". I had always assumed the anomaly models we worked with were far simpler than what machine learning systems do; they seemed unworthy even of the term "mathematical models".

View on HN

Is k-means really what people are doing in serious production machine-learning settings?

Yes, that is one option. Ask me over Christmas and I'll tell you a story about a production system using k-means. (Sorry guys, not a story for public consumption.)

More broadly, a lot of machine learning / AI / etc is simpler than people expect under the hood. It's almost a joke in the AI field: as soon as you produce a system that actually works, people say "Oh that's not AI, that's just math."

Most of the classifiers I've seen used in industry aren't even k-means. They're fixed heuristics (often with plausibly effective decision boundaries found via k-means).

Consider the Netflix Prize, where the most effective algorithm couldn't actually be used in production. Commercial applications tend to favor "good enough" classification with low computational complexity.

> It's almost a joke in the AI field: as soon as you produce a system that actually works, people say "Oh that's not AI, that's just math."

I think that's the standard "insert joke here" any time someone has a "What is AI?" slide.

We often sell our machine learning as mathematical optimization, because ML & AI have bad connotations to industrial customers.

See: http://en.wikipedia.org/wiki/AI_winter

You might want to retest that assumption. Call it a "Big Data" tool and see how you go.

The AI Winter was a long time ago now.

Ha yeah welcome to the field, linear regression is now machine learning. Formerly known as pattern recognition > artificial intelligence > statistics.

K-means is simple, easy to implement and it often works quite well. It's unlikely to be state-of-the-art for any particular problem, but if you just need to do some straightforward clustering as part of a larger data-processing pipeline, K-means is a reasonable default choice, at least until you figure out that the clustering step is your bottleneck in terms of overall performance.

For example, this paper http://www.stanford.edu/~acoates/papers/coatesng_nntot2012.p... demonstrates that using K-means to learn feature representations for images (i.e., use k-means to cluster small patches of pixels from natural images, then represent images in the basis of cluster centers; repeat this several times to form a "deep" network of feature representations) gives image classification results that are competitive with much more complex / intimidating deep neural network models.

Whatever problem you are solving, if you don't try k-means before spending a huge amount of programmer and computer time on more complex algorithms, you are doing it wrong.

Just as most programming is CRUD, most machine learning in practice is k-means or linear models.

Many previous k-means applications are now done using support vector machines. They are a bit more advances, but not a lot.

In my opinion, K-means wouldn't really count as machine learning, learning vector quantization is close counterpart.

Reply on news.ycombinator.com