I would expect with all the effort that has gone into these, and all the progress in machine learning, these systems would be fantastic and provide recommendations that I really enjoy. But they don’t.
YouTube seems to have a massive recency bias and music, film, and TV recommendations rarely end up being things I enjoy.
Supposedly ML should be able to figure that out, by monitoring millions of other people's listening habits. We are not as unique as we think we are. Apparently the models they use are not very good.
That's a name I've not heard in a long time. I had no idea they were still in business, I may give it a try.
Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.
Amazon is not an idiot:
> You need two insights here:
> 1) Conditional probability is a mathematical technology that does exist.
> 2) Buying X is not entirely random across the population.
> The ways X is not random vary based on the good [math]. Consider refrigerators.
> You probably buy one every ten years. If I don't know where you are in your refrigerator cycle, my prior estimate should be 2.7e-4 probability of you buying it tomorrow.
> Suppose I know you bought yesterday. In my life on earth, I have realized that buyer's remorse is a thing.
> What's a SWAG for how often a purchase immediately goes wrong? Not right color? Fridge DOA? Shoot I mismeasured my kitchen? Wife just hates it? Call that 2%. If I fix it within a week, then 2% / 7 = 2.9e-3 probability of purchasing a new fridge.
> That's a 10X relative risk.
Source: patio11 (Patrick McKenzie)
So many people complain about this behavior of recommender systems and here comes Patrick dropping some math and saying: “Well, actually there’s this and this probability for this to happen”.
Don’t piss on me and tell me that it’s raining. Before I make a purchase, especially online, I do my research and make a choice, that’s it.
I’ve never returned anything online and I’ve never needed to buy a second item after buying the first one.
If you insist that your math is right, give me a button I can press so I don’t have to care about your probabilities.
Maybe it is, but I'm not totally convinced. For one it doesn't explain why Amazon would recommend literally the same SKU so often. Also, does Amazon really want to incentivize returning large items like fridges?
And the poker thing from the original post
> People who are compensated strictly based on their ability to predict the future, like poker players... tend to be much better at high school math than Twitter users.
Professional gamblers in games like poker usually don't try to predict the future. They just have a small set of hands with known probability and mainly have to focus on things like sizing their bets. (Poker players also have to read human signals, but I'm not sure they explicitly assign probabilities to these).
Another explanation is just that the cost of recommending you a fridge you already own is lower than the cost of tuning the algorithm better or having multiple algorithms depending on tuples of factors (price, customer, customer behavior). If that's true, then we should expect Amazon to do less of this in the future. If Patio11's explanation is correct, then we should expect Amazon to continue to recommend these items even as the algorithms are updated to the more recent generation of AI.
Whether poker players are predicting the future or not, I would argue Amazon is doing the same thing:
- Each potential customer can be considered a hand with a certain probability.
- Amazon is sizing their "bet" (ad budget) according to the probability that customer will convert.
I also agree with this observation about poker, and can imagine Amazon using a similar strategy: https://hw.leftium.com/#/item/39462238
Successful marketers were already teaching this decades ago: https://thegaryhalbertletter.com/newsletters/direct_marketin...
But yes, it's all just a coincidence and does not affect the quality of recommendations.
TikTok recommends 9 good videos and then 1 where they explain the political topic of the day from their perspective because you are located in Europe.
So If I like and disliked 10 movies just don't show me movies from users who also liked these. First, filter or downgrade all users who liked what I disliked and then create my recommendations.
Results these days seem worse then the old days of Altavista and Lycos.
Right now it feels everyone is ready to ditch Apple products, Twitter, Tesla, Amazon, YouTube, Gmail once something better comes around because of all these small quirks and weird UX things where you notice that the vendor's interests are just not aligned with you as a user/customer.
Example: rap music and hiphop. For the most part, I don't enjoy it that much. There are a few things though that will make a track palatable to me (or instantly turn me off despite anything else positive about it):
- Sentimentality or romance in the lyrics
- Backing track or samples that are harmonically interesting
- No egregious sexism, misogyny, glorifying of violence, thug/gangbanger culture, etc
- Beats featuring stereotypical trap hi hats kind of annoy me
The way I assume recommendation systems are traditionally designed does not account for this. It sees me listen to these tracks, and thinks I'll probably like something by similar artists or the same artists. As far as I'm aware, Spotify's recommendation system is not aware of things like tempo, meter, tonality, themes of the lyrics, harmony, etc. and so there's no way it can pick tracks like this out from the crowd.
And why would they bother? Those are all much more technically difficult things to implement than forming correlations between IDs in a database.
This is likely intentional to encourage more content creation. Competing with two decades of content is almost impossible, so they make them compete with just 2 weeks of content.
Does it mean at peak hours IG switches to a simpler algorithm?
Otherwise, why would the quality of a RS drop during those hours?
It is extremely hard to predict human behavior beyond simple schemes such as most popular items (or most similar items to those you’ve seen before).
(bio: six years of xp in a leading recommendation company)
Let's say for example that you've enjoyed the recent hit game Baldur's Gate 3 and you'd like to play something similar. You check out the Steam page, see that the game is tagged as a "RPG", so you click the tag and expect to get something similar. What you get instead are games that are not only very different but also so far removed from the genre that no one will ever list them in a forum thread talking about RPGs. Examples include titles such as Dota 2, Warframe, Palworld and Horizon Zero Dawn. There are genuine RPG games as well but the fact that there are so many titles that you need to ignore is pretty bad.
Tags aren't the only way Steam recommends new games. Going back to Baldur's Gate 3 Steam page there's a section called "more like this". I'd expect it to match more closely to BG3 and in many cases it does. But when it doesn't, it shows up ridiculous recommendations like The Sims 3 or Tom Clancy's The Division - games that have nothing to do with what Baldur's Gate 3 is.
And all of this is for an extremely popular game that at the same time doesn't do anything revolutionary. Trying the same approach with a more unconventional title that you've liked is a quick recipe for failure. I've just checked the recommendation page for Undertale and it's full of random games that have nothing to do with the title.
There were a lot of little things that added up:
1. Everyone interprets the 1.0 - 10.0 rating scale differently.
2. Most users just rate the same, universally known games.
3. For the other users, the games they've played are usually really different. It's a sparse matrix.
I managed to find a few dozen similar users. Found some hidden gems by going through their pages manually. Fewer than I would have hoped though.
For example, if you have not seen The Shawshank Redemption, chances are you will like it. It's #1 among IMDB top 250 list. But a recommender does not know if you've seen it. If you've seen it already, it's a bad recommendation.
So the same recommendation for the same person can be good today and bad tomorrow, depending on something recommender engine does not see. That makes it very difficult to tune and measure performance.
YouTube: Is that recommendation for good content or for the highest value content you will consume?
Netflix: The more you use it, the less they make. There is a perverse incentive to put just enough good content in front of you to stay subscribed but not use it more.
Amazon: They dont give a fuck what you buy, the sellers are now in a race to the bottom and that business pays for it self. AWS makes all the money.
Find the perverse incentive and optimize for that.
When it comes to relaxing and watching something, I don’t like _my own recommendations_.
Maybe recommendation is like that friend who invites you to watch a movie—you know it’s a gamble. Haha.
All machine learning algorithms struggle at the edges — they’re very good at predicting aggregate behavior.
If you have eclectic tastes there’s probably not enough data on your demographic.
...and I'll never stop saying this: they have some sort of monetized recommendation system in place. Can't prove it but I can see it working almost every week. A video of a big company, "celebrity" or TV channel that I'd never watch, find it's way in my feed.
A passable analogy: you buy a car and get hassled, often hard-sold for a pre-paid maintenance package, tire insurance, financing insurance, undercoating, bla bla. You don't want any of it but it's what they push the hardest.
It is counter intuitive but yes companies do what is best for them. Often it also happens to be aligned but also quite often not.
Like loyal customers more often are ripped off because of vendor lock where new customers get huge discounts.
Is it? This is how any (public) company works - they will do whatever it takes to make the shareholder return the highest. This inherently disregards what's best for the "customer." Any beliefs contrary to this is a naive belief that doing what the customer wants == the most revenue, e.g. "the customer is always right."
Maybe in some fields, but definitely not in circumstances like this where the "customer" is the product (or more accurately - the advertisers are the real customers).
By extension, "companies don't always do what's best for their customers" also applies to their employees, maybe even more so. I'm very certain that the largest US companies would kill their own employees if it was legal and resulted in the most cost savings/profit for them.