Hacker News Reader

Comment by sadikkapadia1 | Hacker News Reader

I wrote the recommendation system at Netflix (still in use after 5 years). Primary problem was company politics. Many groups were not happy that one person could write a system that was better in A/B test, had more uptime and cheaper to run. All of it (ML, production, monitoring), was custom code.

View on HN

Almost all of your top-level comments mention you did this

Interestingly, his Medium post from two years ago [1] also says "5 years ago", and happens to be the only activity there.

[1] https://medium.com/@sadikkapadia/i-wrote-the-recommendation-...

I don't keep track of time. That system is old technology. I did confirm from a Netflix employee that they still use it a few months ago. Deep learning, LDA (even one of Xavier's pet projects - k-means), did not do better.

hard to fault him in this particular case as it's directly pertinent to the original topic.

This is what working in a big company is like sometimes. Imagine a person like MaxLeiter working in Netflix. You would think that all staff would be happy that their work became easier. But a minority seem to have a zero-sum mindset.

When I finish my current work I will talk about that also. Some of my older work is speech recognition. Download my thesis.

I always wondered how do you find the best artwork for the movies, is it multi armed bandits with thompson sampling?

In my company navigating politics is always the hard part, the marketing team would love to spam everyone all the time and the product and sales team would love to sell some kind of upgraded recommendation, its hard to push back but with metrics of coverage, ctr, precision, etc we usually kept them quiet with this metrics

@sadikkapadia - Any idea why Recommendation Engine as a Service has not picked up? I realize that building a use-case specific recommendation engine is unique.

However, I am wondering is there a recommendation engine as a service, which is similar to algolia available/possible.

I see only 2 players - yusp and recombee.

I'd appreciate any thought you have on this.

> Any idea why Recommendation Engine as a Service has not picked up?

These kind of services aren't so much exposed to the public and likely don't start at <100 bucks a month, which could be why those services are not that visible. However, there are some e-commerce services going in that direction, such like AgilOne...

It is hard to sell technology to companies when they have their own teams (often using free libraries). Embedded teams are always experts and will often discredit better technology. Often the only method of testing is A/B. These can easily be manipulated. For instance at Netflix (ignoring more blatant practices), P-hacking (run thousands of simulations and report ones that worked), and HARKing (come up with a hypothesis after the results are known) are rampant. That is part of the reason the recommender has been degrading over the years.

> That is part of the reason the recommender has been degrading over the years.

Netflix is clearly promoting its original shows. Do you think your system is still in use now that they've moved to thumbs rating and percent match score?

That is what I have been told. It is however clearly messed up.

I feel for you. It must be a huge disappointment to see what they've done to your work. Netflix recommendation system used to be very good a few years ago.

https://news.ycombinator.com/item?id=15607383

Amazing story! I don't know if it's true or if you're delusional, but I'm inclined to believe you as it would explain why Netflix recommendation system went downhill instead of improving.

I worked for a recommendation engine as a service company called aggregate knowledge in 2007 when they got their second round of funding, 25m from kleiner. It was a remarkably lousy business, and they didn't do well.

There was really nothing wrong with the concept. A little JavaScript on the page, a bunch of back end magic, the ability to use a larger data pool because you are collecting from multiple sites.... But people wouldn't pay, and the engagement of recommendations was never that good.

The secret we learned was, after a bunch of math and research, was that 'best in this category and 3 closest adjacent categories works so well in retail that a naive algo did very well.

Better than the fancy math, which once associated the Koran with the sports illustrated swim suit edition ( and vice versa ).

There is a massive data sufficiency problem, and no company with the real data would go into this low end business. Small companies can't get enough data to be relevent.

The world is 10 years later, ml is better understood, so it might all be different now.... Pm me if you are interested in further detail

What was the tech stack? Did you use a graph db? I've built my own in neo4j for about 500M data points, but would love to know what you used.

Same situation with a smaller project. I still struggle to pull off it for the problem I face today. How did you succeed?

Reply on news.ycombinator.com