Hacker News Reader

Comment by inertiatic | Hacker News Reader

Search is now machine learning? Interesting introduction to the topic otherwise.

I would say this isn't machine learning, but relevance in general is an interesting topic to apply supervised learning. Of course the training data is the hard part,

An article on the topic, https://opensourceconnections.com/blog/2017/08/03/search-as-... (disclaimer I wrote it...)

I also work on this field so I do have an idea of what's possible if you apply machine learning techniques to improve relevance rankings.

But to my intuition, basic search doesn't feel like a machine learning task. After reading some of the responses to my post however I'm trying to come up with a meaningful reason why I wouldn't consider IDF to be machine learning, given that it is updated as more documents enter the corpus and your system "learns" to re-rank existing result sets based on these new documents.

These categorizations tend to be arbitrary and inconsistent because what is an intelligence is probably subjective. People consider kNN and naive bayes to be machine learning. One is "just" sorting and the other is just counting. The learning in naive bayes and even in some higher order bayesian networks is of a similar flavor as the count structures generated for IR.

Because we understand how these algorithms work, we can always reduce them to just this or that. Prediction in many linear classifiers is just dot products. relu neural networks are just lots of clamped dot products. Random projections on simple count data can generate word embeddings.

Whether or not something is merely model-fitting, super-scaling, compression or AI, AI art and machine learning will depend on the field it originated from. It's indisputable that the algorithms are so reducible but I tend to think that we should care more about functional capabilities when compared to an appropriate subset of a known intelligence's abilities than details of implementation.

In general search can be machine learning. Google certainly uses machine learning as part of it's ranking.

I guess you can make the argument that even tf-idf as described in the article is a form of unsupervised machine learning because you obtain ("learn") the idf from the data.

TF-IDF is a feature extracted from the data, much like a simple count of words, but it is not learned. It is simply computed. An example of learned features are word embeddings where it is necessary to train on data to obtain them.

If you want to apply machine learning to search then you need clickstream data, embeddings, or learned feature weights.

Word embeddings are also "simply computed." If you use GloVe, then the vectors are obtained by factoring a matrix of co-occurence counts.

The difference between machine learning and "simple" feature extraction is mostly just in the choice of metaphors used to describe the computation, not in any fundamental properties.

Right. Naive Bayes is considered to be a machine learning algorithm, but also consists of just "simple counting".

Your distinction between TFIDF as simply computed vs embeddings as learned is odd and artificial. Both are computations from data, but TFIDF has an understandable closed form and word embeddings do not. As for machine learning, it has to do with improvement and doesn't even necessarily need data.

I think you’re right. It’s the improvement in the learning process that’s the important bit. TF-IDF lacks that.

Search is often formulated as 'learning to rank'.

AFAIK learning to rank refers to things more advanced than simple TFIDF.

Reply on news.ycombinator.com