> How do people build recommendation engines?
One method, collaborative filtering with latent factor analysis, popularized by its efficacy in making recommendations on Netflix, is to use matrix multiplication to solve the problem.
E.G. Let’s say you have all users (rows) x all books (columns), in a massive sparsely populated matrix, where the value is the rating that user gives a book.
To make recommendations, the goal is to “guess“ what a user would rate a book they haven’t read, and if your guess is they would give it 5 stars, and then you recommend it, and the user gives it 5 stars, it’s a good recommendation.
The “latent factor” idea is breaking the problem up, so in order to compute the rating matrix that is the final size N users by M books, you split it into:
N users x D latent factor
(cross product)
D latent factor x M books
=
N users x M books
It then becomes a machine learning problem, using a loss function plus gradient descent, to solve for that D latent factor.
Customers with a similar latent factor, will have similar taste.
Once you have the latent factors, you find the nearest neighbors (the closest other latent factor vectors measured by dot product or cosine similarity), to compute the nearest books. The vectors that multiply together to give the highest rating, will be the best recommendations for the user.