Ask HN: People with direct knowledge of YouTube Algorithm-why the repeats? | Hacker News Reader

Ask HN: People with direct knowledge of YouTube Algorithm-why the repeats?

64 pointstomcam84 comments

I get that YouTube and virtually all social media sites want to maximize clicks. I get that YouTube really doesn't give a rat's ass about what I want in that context: they think it's worth far more to override my preferences and add all kinds of channels to my feed than I want. But if you have direct knowledge of the algorithm, please tell me why I can watch a video to the end, then have it recommended for days, weeks, or months after, even when I have dutifully upvoted or commented. What's in it for YouTube? Just because I liked a video by Kyle Kulinsky or Jordan Peterson in 2022 doesn't mean I'm still dying to watch it in 2023. And if you want to force shit down my throat, why give up valuable screen real estate to videos I already saw?

View on HN

I don’t know anything about the algorithm, but my dumb lizard brain take is that I often find comfort in watching the same stuff again. Whenever a 3blue1brown video pops up in my feed again, it’s like seeing an old friend and I’ll watch to the end again.

I can think of many videos like this, that pop up in a feed after many years and you’ll see the comments like “see y’all again in a year when the algorithm brings this back” or “I always click when this video is recommended”.

I have several friends who’ll rewatch whole shows again and again for a similar reason. There’s the comfort of something familiar.

So I guess I don’t mind when youtube does this

I like falling asleep to Adrian's Digital Basement. He's got enough content but I often just restart a video anyway.

Obviously, you have fantastic taste in videos. It drives me crazy to get repeats. But it’s really good for me to know that some people think the opposite way.

Many people use Youtube for music, and that lends itself pretty strongly to repeated watches. It could be the algorithm doesn't perfectly differentiate between music watching habits and non-music watching habits, and applies lessons from one to the other.

Easy to filter for those.

Very very unlikely that youtube isn't perfectly clean on which videos are purely music.

I don't have direct knowledge but I know that many people re-watch Youtube videos. I know I've watched the Rickroll video many times (don't hate me; I actually like it).

Here is an old, deleted, reddit thread about people who re-watch the same Youtube video over and over: https://www.reddit.com/r/TooAfraidToAsk/comments/h7fqv9/is_i...

My guess is if you look at the usage statistics "watching a video all the way to the end" is one of the single strongest signals that a person might be willing to watch the same thing a second time.

Damn son! That makes a ton of sense.

> I know I've watched the Rickroll video many times (don't hate me; I actually like it).

It sold a million copies worldwide on release, and was a Top Ten single in 30-odd countries (number 1 in 24), so you know you're not alone in liking it.

Rick Astley is most famous for this one but he had something like eight top ten singles in a row. It can't *all* have been down to Stock, Aitken, and Waterman...

Hypothesis: There are two significant use cases you may not be considering, which make up a significant portion of YouTubes use cases:

1. Music, which often lends towards repeat content

2. Children as a demographic, which any parent will tell you will watch the exact same movie/show/video on repeat ad nauseam.

Because of these, it's likely that YouTube has found that repeats (in general) drive more engagement.

But guess what, YouTube knows most of the time exactly what videos are music, and what videos are targeting children, so in that case they would suggest to repeat those. For me, YouTube suggests I watch whatever, not just music or videos for children.

I'd say the contrary; repeats don't impact the algorithm negatively, so nothing is done about it.

Another data point. Sometimes I like a video, and later when I am watching with friends and see it again, I watch it again to show them. Apply this to partner, roommates, family, etc.

When i say ‘see something once why see it again?’ people look at me like i’m a psycho killer. The paradigm of doom scrolling and reload-scan-reload-scan-reload-scan are entrenched in western social media to the extent that foreign social media that ‘thinks different’ is considered dangerous, subversive, and both political parties are falling over each other to ban it.

i think people do like watching the same video more than once sometimes though.

If people are looking at you like a psycho killer, I think it's probably for some other reason than you saying "see something once why see it again?". Were you holding a very large knife when you said it, perhaps?

I blame David Byrne.

Qu'est-ce que c'est?

Oh I get it now. Ai yai yai.

I don't, what's the joke here? I looked him up and see he's an artist but I don't see any direct jokes related to repeating or large knives.

David Byrne was lead singer for a band called the Talking Heads, and they had a song called Psycho Killer. One of lines in the song is “say something once, why say it again?”

It’s a pretty obscure reference and I did not get it when I made my initial response.

I feel you, and where the algorithm is completely insane for me is in the YouTube Shorts section. I literally get the same videos in the same order multiple times during the week. It makes it feel like there's about 100-200 videos in total on Shorts, and I've seen them all. As a rip-off version of TikTok, where there's an endless stream of new content, you would imagine discovery would be the thing they would focus on in the algorithm the most.

YES!!! that bothers me a whole lot

You are not alone, if you watch three videos by Kyle Kulinski, you realize how rarely he has an original take.

But coming back to your question.

This question reminds me of a complaint that I heard from multiple people about Amazon: why is Amazon recommending me a product I just bought? Or a less generous manifestation of this sentiment "duh, Amazon is so stupid, it recommend something I bought give minutes ago".

In reality, people don't realize how often they buy the same and similar stuff again and again. You bought a usb hub for home, you like it, so you but it for the office. If you didn't like it, you may buy three different usb hubs to make sure you find a good one. You like a book, so you but the same for your sister as a gift. Bought some socks, you enjoy using them, so you buy another 10 so that you don't need to worry about socks for the next two years. You realize you love funny socks, so you keep buying more socks.

The same happens with videos. I don't mind watching a video about streams by Venkat Subramaniam every now and then. A conference talk about regex is recommended to me again?? Hmm, actually, I might watch it again because I don't remember half of it and I remember she prepared a great talk.

I suspect this is it. As an example, someone who bought a new washing machine yesterday is still more likely to buy it or another one tomorrow then a random person on the street. This is why you get a lot of targeted ads for items you just purchased

> In reality, people don't realize how often they buy the same and similar stuff again and again.

Yeah... this is not true for the vast majority of non-consumable products without some sort of time delay.

If you bought 20 Zebra gel pens (best pens on the market, no affiliation just satisfied consumer) you don't need 20 Zebra gel pens for a sale refraction period of at least an hour. Immediately re-advertising is a poor advertising model, conceptually, and I'd really need to see a lot of evidence that this is wrong across a wide variety of products to move away from that prior.

And like, a great salesman wouldn't offer you more of the same product but try to either expand into similar products (you like gel pens? here are some mechanical pencils) or hit you up with complimentary products (what good are gel pens without a nice squishy writing mat? and i bet you're going to need a lot of paper!).

Amazon sort of handles these things in the Amazon UI with "people who bought X also bought Y" but they really should be showing that for their ads: instead of "here are results for your recent search" (which likely shows you the thing you already got) it could do "here are things people who buy stuff near the top of your recent search also buy".

Regardless, as far as I am concerned, the best commentary ever on this topic was CollegeHumor doing the Targetted Salesman skit, where the ad engine is represented by a human salesman shoving products at you all day: https://m.youtube.com/watch?v=KbKdKcGJ4tM

Well I guess statistics don't agree with you or Amazon wouldn't make that much money

Amazonians and other big techies would absolutely agree that the strategy space is undersearched, which puts your assertion's assumption at risk.

Took several years to even consider the statistical power of ad effectiveness.[0]

[0] https://www.jstor.org/stable/26372642

Wouldn't an sufficiently smart algorithm segment (users, items) into reconsumable vs one-offs?

If you buy a USB hub you might like it enough to buy another.

If you buy a washer, you probably won't like it enough to buy another.

But if you're a landlord, and you buy a washer, you might like it enough to buy 10 more.

So your point is that it's a comfort TV thing. We like to watch safe content we know is good.

Whilst I get that's the easy answer from YouTube perspective, I refuse to believe that there is nothing on the entirety of YouTube they can't recommend. It's just sad as a viewer to think how much I'm missing because they're recommending that "This is how empires fall" video for the tenth time.

Or, you were watching music videos and it figures you do want to see it multiple times.

There are some items which I occasionally repurchase — N95 masks, HP printer cartridges (next time I swear I'll get a Brother laser printer or something), mouthwash, handwash, toothbrushes, and yes sometimes even clothing articles such as socks.

So, most "consumables".

Boggles my mind how hundreds of hours of content id enjoy must be uploaded to YouTube a day yet to seems very intent on showing me things I’ve already watched or just the same videos for days on end that I didn’t click on the first 5 times they showed it me and I don’t intend to click on it on the 6th.

Then you check new for you and it’s weirdly just like normal fresh browser yt recs, not my algorithm.

I watched _one_ video of someone on a sailboat. One. Now all YT wants me to watch now are those stupid videos of people travelling the world on their sailboats. Their algorithm is so fucking stupid I can't look at that home screen anymore, I only look at the subscription one, at least that one I have some control over.

I am starting to question whether I want to click any video now, because I have what seems to be an extremely wide range of interests. Yet I f I decide that a single video on berry picking in the Himalayas might be interesting, I fear I will get nothing but Himalayan berry picking suggestions for the next hour or day.

Had a similar thing with stupid videos of people reviewing airline first class, I found the solution for when it does that, you have to go into history and find the video that caused it and remove it from your history and any others you accidentally clicked on since, solved it most times.

Hate how persistent it is with some of the lowest forms of content, also no idea why I can't just type in a channel name and block that channel from ever appearing, I can only do it when they get recommended.

There were dozens of videos about sailing in my history, no idea who's watching on my account. I checked my sessions with Google and nothing looks suspicious. Very weird. Thanks for pointing me to the history!

It actually works really well for some viewing patterns. For myself, it's the best recommendation algorithm.

When I'm listening to 80s rock, it gives me a bunch of other 80s rock, notably including things I've enjoyed listening to before so I can just yoink that right into my playlist. Also throws in some 2000s dubstep and some 60's country that I like and a couple of things that I've never heard before in case I've started out on 80s rock but just am not feeling it tonight.

When I'm watching thought-provoking mini-documentaries, it gives me more by the same creator along with some random related stuff. Some of that is stuff I've watched before but might want to re-watch or share with someone. Or to compare the thoughts of this guy vs that guy.

Your usage pattern might be different. But for many of us, it works quite well. I would assume they're optimizing for the most common use cases. And yes, I liked that video, I do want to see it again.

I always use the "Not interested" and "Don't recommend channel" options when I visit YouTube. You really have to babysit the algorithm to get value from the recommendations beyond time wasting.

Watch this scene: https://youtu.be/LXaHbJhKWLc

I think that what he says pretty much applies to almost everything that seems sophisticated to us such as health tech, privacy tech, fin tech, anti-terrorism tech and of course user behavior prediction tech. A lot of people are convinced that what these companies do is highly sophisticated and very difficult to achieve in terms of algorithmics. Most of the things that look sophisticated are just mediocre at most, with a handful of companies just having successfully delivered on a handful of problems and pretending to have solved many others.

One of the reasons I suspect i they're failing at so many things is that these companies have focused extensively on hiring the brightest minds to solve the most difficult problems they had to solve 10-15 years ago.

During all this time they failed at solving simple problems that do not require the brightest minds in the world but just people who actually can discuss with other people and solve simple problems with their technologies.

You're perfectly legitimate in hoping that YouTube would have solved these problems but no, they haven't, they're actually quite weak at solving many simple things and it is crucial that the large audience doesn't realize that.

I have no knowledge of the algorithm

I do however watch “Picard make it so (let it snow)” every year. Some videos I’ll watch more than once throughout the year.

On the other hand it’s rare I’ll watch a Numberphile or Tom Scott video more than once, and when I do I deliberately seek it out

If recommendations had any intelligence they'd maybe list the next video in the same playlist, by the video author, as the top choice. It's infuriating. Getting to the playlist interface from a video is not even easy.

Youtube would benefit from going MVNO with a subscription model. Let others develop the algorithm as a subscription basis, be the content hub.

Charge for ads, get longer engagement, drop the nasty rabbit holes as other subscription models are more informative, etc. Encourage better content to boot.

There are a LOT of ways to organize their content model, their current model pains me as a consumer, an economist, and with due respect would be indistinguishable from a catastrophe of ego tripping product owners.

YouTube's recommendation engineers just aren't as good as you expect them to be.

I wouldn't go that far, but I do think it's true that YouTube's recommendation system is simply horrible.

At least, it borders on worthless for me.

I am beginning to think it borders on malice or something worse.

They're good for making money off of ad revenue.

Why would they care what your experience is? You're the product, not the customer.

For a long time, I have been curious why overfitting their recommendation model generates more revenue. My guess is that this is stark for kid-friendly content. If parents have noticed that a certain set of videos is safe for their children, they are likely to not deviate from that set.

Today, when I found this thread, I started looking for past HN threads discussing this topic. I came across a 2019 thread (https://news.ycombinator.com/item?id=18999326) where a similar discussion was held. In that post, a user mentioned the following:

> "Read the YouTube recommendation paper (https://ai.google/research/pubs/pub45530) and it will become clear why it recommends a ton of high-engagement, clickbait-y content based on a minimal set of recent watches."

I skimmed through the paper, and they have some interesting stuff in there! Maybe I should read it more carefully later. Here's some I found:

> We consistently observe that users prefer fresh content, though not at the expense of relevance. In addition to the first-order effect of simply recommending new videos that users want to watch, there is a critical secondary phenomenon of bootstrapping and propagating viral content.

> Consider as an example a case in which the user has just issued a search query for “taylor swift”. Since our problem is posed as predicting the next watched video, a classifier given this information will predict that the most likely videos to be watched are those which appear on the corresponding search results page for “taylor swift”. Unsurpisingly, reproducing the user’s last search page as homepage recommendations performs very poorly.

This is probably the most interesting one!

> We observe that the most important signals are those that describe a user’s previous interaction with the item itself and other similar items ... As an example, consider the user’s past history with the channel that uploaded the video being scored - how many videos has the user watched from this channel? When was the last time the user watched a video on this topic? These continuous features describing past user actions on related items are particularly powerful because they generalize well across disparate items...

I am finding that I use YouTube a lot less these days. Back in the late 2000s I remember when I watched a music video, the recommended videos would be other videos by the same artist. Now it's just videos I've watched before and unrelated videos. The front page is mostly videos I've watched before and ones that it has shown before that I didn't click on. So there is not much difference between the front page and the recommended videos section. It doesn't even really show new videos from channels I've subscribed to, I have to go to the subscriptions page and look for them there. So I just get bored easily and do something else. I tried the shorts page, but I have no interest in the toxic political and far right garbage from the US, since I'm from Australia, so that was useless. I even tried looking for a page that showed what live videos are currently playing, like a twitch page, but the only live page only had a few videos listed.

Similarly, if I only watch the first 10 minutes of a Netflix video and then turn it off, why does it then recommend similar content forever after that? I accidentally watched 5 minutes of a Polish show six months ago (I don't speak Polish and I don't like subtitles) and now I constantly get recommendations for Polish language shows.

I also find it odd when I buy something on Amazon that people typically only ever have one of (at a time), then I immediately get pushed to buy another. Like a vacuum cleaner.

In my experience the only thing that really works is "don't recommend this channel" option in the ellipsis menu, however recently I must have reached some limit because it is now showing me channels I've already blocked. My guess would be somewhere between 50~100 is the limit for blocked channels.

There is definitely a limit to this list, I have run into that too. Also lately it doesn't always listen like it used to. I have to hit the channel block multiple times sometimes.

Yes, it seems like one signal is no longer enough for it.

In recommendation systems in general, it's well known that recommending things based on recent activity far outperforms other methods (e.g. product category, past liked items, subscribed items, searched items).

For example, Hulu showed that recommending shows similar to "recently watched" rather than "watched in the past" increases CTR by 188%. In particular see the graph at the end which compares "recently watched" recommendations to other types: https://web.archive.org/web/20120130230618/http://tech.hulu....

I find myself often doing the, "don't recommend this video -> I've already seen it" option many many times.

I wish they would improve the algorithm to distinguish between things I want to replay like music versus things I want to see once like a documentary.

This is a great question and I have no idea why. But if I liked the video, I'm afraid to click "not interested" because I don't want Youtube to not recommend similar content in the future.

If you do, you can select a reason from “I already watched the video” or “I don’t like the video”. In theory that should help signal it. It hasn’t ruined my recs yet (afaik)

My problem with that is marketers understand that disliking something can be a strong signal to make you watch more of the same, because social media sites like to rile you up. Engagement is engagement, and I suspect YouTube often feeds me things I don’t like for that reason.

You're absolutely right. They're trying to get clicks and engagement, not the best possible curated experience.

They DGAF if you like what you see, only that they can hit you with more targeted ads from paying customers -- and boy howdy does the rile-em-up lobby pay big money.

As mentioned in a previous thread: you're the product, not the customer.

My YouTube recommendations list is constantly polluted with Joe Rogan, even though I've never watched a single video of his. One theory is it's because my friends/family watch a bunch of his videos, and the algorithm basically concludes "Fuck your personal tastes! You like what your friends like!"

Nothing against Joe Rogan. It's just weird how I get so many video recommendations for him, even though I've never watched one.

My exact experience, and it happens with other prominent Youtubers as well.

I have several untested theories regarding YouTube recommendation system. One of them is: is better to promote a video of a channel who's already monetized than it is to promote a video from a small channel (creating a new monetized channel). I don't know the bureaucracy behind paying channel owners, but it seems the algorithm favor those videos, even if you've already watched them.

Clever thought. And it makes sense from YouTube‘s perspective, because IMHO they often demonetize for reasons that have little to do with the interests of the listener.

Probably.

Tbh, I'm often mindlessly watching stuff on youtube, it's mostly time I consider wasted anyway, there's rarely something really productive to do there. And, in that mode, I've watched many videos many times over. Comedians are a prime example of this. I think e.g. Rick Gervais gets rotate in my recommends once every couple of months, and I watch quite many of them.

I wonder if the censorship algorithms are getting too aggressive and that's messing up the suggestion algorithms.

I get the same short suggestions repeatedly but sometimes they're mixed in with very new, low quality stuff with no comments or likes.

My youtube time has gone down precipitously, which is a good thing for me, but not for youtube.

My gripe is that YouTube constantly shows me the same videos I didn't click on. A clue: if you've shown me this thing 3x and I never clicked on it I am not interested!

I agree: it makes it seem like there are a couple hundred videos on YouTube, and they keep trying to shove the same ones down my throat.

People often watch a video, and then show it to a friend. Hence, as far as the algorithm is concerned, thats a re-watch.

Obviously the algorithm has no way to know your friend isn't standing right there right now and demanding to be shown something you've already seen but think is cool...

I would also like to know the answer to my problem too:

Not interested ->Tell us why ->I've already watched the video

. . . . . . .

I would just like a way to block certain topics entirely.

I would assume that all the minecraft, sports, and vlogger stuff is what I get because it’s trending, but i also have zero interest in any of that stuff at all.

I have my watch history set to autodelete, and sometimes videos I’ve watched come back after they age out of the history, which makes sense.

I was unable to tell the difference between using auto delete and not.

From the history page, click Manage History and it shows the setting.

Thanks! I meant it did not seem to affect the algorithm.

>then have it recommended for days, weeks, or months after

Because it works. I'm ashamed to admit I rewatch a lot of videos on YouTube.

I am oddly comforted by your response. I would rather believe that your habits are more likely to appear in people than mine. The alternative is to believe they are just stupid or vindictive at YouTube, and I prefer not to believe that.

I just came here to say that now since there’s precedent, YouTube should os it’s underlying algorithm code.

Funny videos and music video are good candidates for repeats. I see these all the time in my feed.

eventual consistency?

Could it be something to do with Kyle Kulinksy or Jordan Peterson specifically? I.e. maybe other viewers who watch those videos have tendency to rewatch at some point in the future?

Scrolling through some of the comments on Jordan Peterson videos I have found that his fanbase does seem to be a little on the fanatical side. (Just to be clear this is not a knock against Jordan Peterson - I actually find him pretty interesting. Nor do I think his fans are uniquely fanatical - lots of YouTubers have fanatical fans. But because he is in the realm of self-help and politics, and he tends to be very opinionated, he does tend to attract some really hardcore fans who I could imagine rewatching his videos from time to time).

I’m not sure. I used them because they are well-known figures, but the same thing happens to me with many other Youtubers as well.

you liked a jordan peterson video, i think youtube assuming you want shit forced down your throat is not entirely crazy

Touché! However, the same phenomenon occurs when I watch videos, and either actively dislike them, or don’t finish them, or don’t interact with the videos at all.

Reply on news.ycombinator.com