I created an alternative to the YouTube algorithm to stop me wasting time | Hacker News Reader

I created an alternative to the YouTube algorithm to stop me wasting time

towardsdatascience.com

343 pointsChrisLovejoy153 comments

Really cool project, going to play around with it, since I wanted to experiment with alternative algorithms for these kind of platforms for a while, but didn't found time to get started.

My biggest gripe with YouTube recommendations, or any service of that kind (Spotify, Amazon, etc.), is that they don't understand what aspects of the content I value. So it is a huge hit and miss. Sometimes it works, because I guess I just happen to value the same things that the majority of people do, other times, I might watch an hour long very in-depth video about a topic, then YouTube for a week or so tries to get me to watch 5 minute, rather shallow introduction videos that are way more popular. There is nothing wrong with those videos per se, they are often really well made, but it is not what I'm looking for.

Other times it picks stuff up from the long tail that seemingly gets into a feedback loop, suddenly reaching millions of view, with practically zero relevance to me.

Then there are some personal tastes of mine, e.g. that I can't stand the, I guess I'd call it the "Youtube Voice" where creators go completely over the top instead of talking like a normal person.

I wonder what metric could be used as a proxy to get better recommendations? Or if a better approach would be, to just build a better custom search so I can tweak it to what I'm looking for at this moment in time.

I wish I could explicitly tell YouTube “don’t ever recommend videos I’ve seen.”

Awhile back I started using a video's 'like' button simply as a mechanism by which I could tell if I had watched something before since I did get repeat recommendations. Then, if I clicked a recommended video again later, I'd know I had already seen it. Over time, as far as I could tell, YT stopped recommending videos I'd seen before as there were fewer and fewer videos I would check out that I had already seen.

May also be attributed to something else, or I'm getting old, but it seems to have been a fortunate outcome.

inject those two styles into YT website:

    a #video-title { color: red !important;}
    a:visited #video-title { color: black !important;}

They have two options under the triple dot menu (at least on Android feed) that you are not interested in this video or this channel.

May help curate once you start getting those recommendations.

I once watched a few videos about US politics. After that YouTube just kept spanning half my recommendations with vids about USpol. While they were addicting, they didn't really interest me. I then took the effort to flag tem as 'not interesting' (about 3x, refreshing the browser) and it works, I have gotten very few recommendations about that since then.

Then you cut out videos you were interested in the first place.

When it asks you why you're not interested, one of the options is "I've already watched this video".

I used to do that, but it was a lot of work & I didn't notice an improvement to the search results.

I experienced exactly the same. I suspect that since they're optimising for engagement, already having watched a video is some indication that you'll engage with it again.

Also, if a video thumbnail/title displays on my screen and I don't click it, it is a sign of disinterest. Stop suggesting videos I have repeatedly scrolled past. Use the lack of engagement as a feature.

Once I tried disliking or dismissing a video as enforced dislike. Then it said "ok disengaging the entire category" and I was like wtf no. That's not at all what I wanted. And there was no way to undo. So its frustrating trying to teach those algorithms things I don't like.

It was on google somewhere. It was like I wanted to dislike a particular author or that particular programming language they were talking about, but took it as me not wanting to hear about anything technological ever again. It was frustrating as heck.

This option would single-handedly fix their completely broken music recommendation algorithm that only seems to auto-play the same 5 or so songs that I recently listened to and thumbs-upped.

I get the same behavior, only I've never thumbs-upped anything.

amen! I wish you could crank up exploration cf. exploitation in your settings or something

It would be cool in general if recommendation algorithms or search algorithms were more customizable. I would imagine you could put some code into your settings that controls which recommendations you see. The platform, like YouTube in this case, could compute features that enable different kind of recommendation algorithms. There could be communities of people who tinker with this and share their recommendation algorithms.

Steam added this feature recently (https://store.steampowered.com/recommender/ lets you add weights for "niche/popular" and "older/newer") and I've never had so much fun browsing a store. I hope other platforms take note

* Except for music.

Don't ever recommend videos from 9 years ago please.

This is the part I don't understand, how is Google doing so bad at this. You get slightly better experience if you use Chrome. It (on purposely?) keeps asking you to click 3 buttons to play a video Edge, and won't even show comments.

Our tastes are so different. I'd much rather watch a great video from 2005 than a good one from 2020. I'm irritated by the recommendation algorithms pushing new content. Why would I care whether the video was published last night or last year?

Yes, recommendation should be personalized anyway, so you can get videos from 9 years ago and I don't.

If e.g. you want political thinking in current events than you'd probably value recent videos more.

If you wanna see a great video on some obscure implementation detail of some electronics gear from the soviet era it doesn't really matter if it was uploaded a year ago or 8 years ago.

What I mean is it displays a button to click after I clicked play on video when I use Edge, which is my main drive nowadays. And the comments are not displaying. It cannot be only my experience I hope :S so I get downvoted because my taste not liking old videos :/

agreed, it's strange that with all their legions of machine learning PhDs Youtube recommendations still suck.

The worst part is when it gets stuck on particular video/topic and just keeps recommending the same thing for months.

I’m sure the YT algorithms are optimized just fine for whatever revenue metrics they’re maximizing. You and I might just be part of the edge cases that they haven’t bothered improving their models on just yet

This is such an insightful comment. I find myself falling into the trap of complaining about things that are "broken" all the time too, but you're right: if you're on HN then you're not the customer that most companies are targeting.

unless one can prove that there is something special about being on HN that makes a person intrinsically different than other consumers I don't think this is a reasonable conclusion. It may be that this can be proven but I would like to see the proof.

If someone posts on a crafting forum, complains about youtube being broken is the conclusion that they are not the customer that most companies are targeting?

or any number of other fora, I doubt the predictor of a single interest is enough to differentiate the person so significantly.

Yes, just posting on a forum alone puts you in the tiny minority of consumers. There are huge swaths of people that barely like to read content, let alone write.

Look at how many youtube videos dominate search results with hundreds of thousands of views for “how to” X when X is trivially described in a few lines of text.

>Yes, just posting on a forum alone puts you in the tiny minority of consumers. There are huge swaths of people that barely like to read content, let alone write.

My stepfather was an idiot, a drunkard, a gambling addict, a drug addict, easily conned by direct marketing and ponzi schemes, with not a large number of friends, and poorly educated.

He read, as far as I have been able to figure out, 1 book in his life and he read that book 75% of the way finished (It was some sort of crime book - pretty thick though so about the 600-800 pages length)

But when the internet came around he got himself an online identity and was out on forums of people who shared his interests all the time posting his views on things.

There are really not very many people out there less capable than my stepfather, percentage wise. I do not believe there are huge swathes of people that he beat.

>Look at how many youtube videos dominate search results with hundreds of thousands of views for “how to” X when X is trivially described in a few lines of text.

given that youtube is owned by the Google, not sure what to take from that other than they put results from one of their properties at the top for purposes of their own.

although I often take a how to X video over text because even though the X is trivially described there are a lot of extra things you can find out from the video really quick - for example if you search for how to do X in Y

and you read a tutorial, it can be that the tutorial is slightly off from your version of Y, and you spend time trying to figure out if it is the tutorial that is wrong, if you have misunderstood their trivial few lines of text because the text might not be clearly written and so on and so forth, but when you look at the video if it shows a menu entry being clicked on that just does not exist then you know they have something different than you do and if you can't immediately figure out what your system change has been try to go find some other solution.

It is the extra information that a video gives that can make the video useful. As to which to choose is dependent on a lot of things.

Their recommendations are not designed to make you happy, but to optimize their business. They are driving you to ads, not joy.

The ML PhDs could be working just fine. You won’t know.

I pay for YouTube premium, so I don't have ads. Still, I'm seeing many of the problems other people are talking about, especially 10 minute videos where the content could be explained in 10 seconds.

Though that's a reason why many people don't pay for YouTube premium, not why you somehow deserve better treatment.

I want this for Spotify. Especially the ability to say "wipe your recommendation model you have on me for the weekly list and start from scratch."

This is a great start, but there's a huge portion of data you're just flat out missing from this--comments.

There's so much useful information inside user comments.

I have a theory that I'm truthfully too lazy to test out, but I'm presuming that lower quality videos are going to have an average comment text complixicity number lower than the higher quality videos, especially as your topic gets more specific/technical.

Take a look textstat(1) for some examples.

(1). https://github.com/shivam5992/textstat

Unfortunately comments are known to be important for ranking. I see more and more Youtubers asking their audience to write comments.

The carwow channel for example is definitely asking to answer stupid debates in the comments to improve its ranking. The vilebrequin channel does the same but as a joke and explain that it's to improve the ranking. The community then write a lot of long and stupid comments to improve the ranking.

writing a comment, and doing likes, both improve the ranking.

Writing long complex comments that are still readable would be a lot harder to ask of your audience, you would have to inspire it. Therefore something like this is maybe better for the ranking https://medium.com/glose-team/how-to-evaluate-text-readabili...

on edit: changed wrong comments to ranking

The youtube algorithm is working extremely well, to the point where I find hard to believe it does not have any human in the loop or active moderation.

I have discovered so many interesting types of music and documentaries and subculture/subgenres on it. It manages to hit my interests so well, that I rarely go to netflix nor spotify anymore for discovery. I 'll either directly go watch something that I 've chosen on amazon prime, or let the youtube algorithm handle recommendations. (I know this reads like a fake review - but I swear I have no affiliation for youtube btw. Just amazed how well their recommendation engine works)

Are you for real?

YouTube seems to do terrible at the recommendations. It cannot even recognize that when I’m behind the computer, I’m not interested in watching documentaries and when I’m behind my Apple TV, I’m not interested in listening to music videos. It’s such basic stuff, let alone the actual recommendations; it just keeps rehashing the same channels. “Oh you watched a video about the US elections, let’s bombard you with US election videos for the next 3 weeks”.

If I compare the music recommendations of YouTube versus Spotify, the latter is miles ahead. YouTube always seems to either descend into obscurity, or you’ll end up with the same old stuff, where Spotify seems to balance things out fairly nicely.

In all honesty, for all the AI expertise and “intellectual excellence” Google possesses in this area, I am surprised how bad they are doing here.

I actually agree with OP. YouTube recommendations on my primary account are quite good.

I attribute this to careful curation of my logged-in viewing habits.

> you watched a video about the US elections

That's the problem. As a rule, I never view political, celebrity, or clickbait videos when logged in. There is such a vast amount of dreck in these categories, you're sure to be disappointed sooner or later.

If I see a political/clickbait video that catches my eye, I open it in an Incognito tab.

> I attribute this to careful curation of my logged-in viewing habits.

> That's the problem. As a rule, I never view political, celebrity, or clickbait videos when logged in. There is such a vast amount of dreck in these categories, you're sure to be disappointed sooner or later.

I think you've just pointed out that YT's recommendation algorithms are very poor. If a user needs to carefully curate to get good recommendations, then it means Youtube's algorithm is not very robust.

I've had a similar experience. Anything I don't want to litter my recommendations I view from a private window. If I accidentally click on something I don't particularly want to watch, I remove it from my history.

It's not perfect, because as you go down the YT rabbit hole, you'll invariably get fed "top ten X" type clickbait videos if there's anything that's remotely related to what you're watching, but it does reduce the noise level.

Curating your subscriptions also helps.

I think this is the problem: it's really hard to understand how to get better recommendations out of YouTube.

I did try to curate my watching history, at one point I erased it and started to keep track of what stayed there or not, it doesn't matter, if I watch 2-3 news/political commentary I start to get crap recommendations. If I watch 2-3 videos of any subject I start to see a flood of those recommendations.

It is absolutely useless to me, even more that I have some very non-overlapping hobbies that I jump around.

I tried curating my subscription list, at some point that worked but since YouTube dropped those from their recommendation algorithm my subs are just to check videos from channels I like, some weeks I will see the same videos on my recommendations that I have already watched.

It's a mess, I can't understand it as a SWE to be able to tailor it to my usage. By now, after trying for about a year or so, I completely gave up on "training" YT's recommendation algorithm...

On the other hand, Spotify has always been really good for me, the trends of what is recommended doesn't change so often if I don't change my listening habits, when I do change it picks it up as a quiet signal and doesn't overwhelm me with recommendations for a genre I listened to one day out of the year. Also the Spotify's Radio feature for a song/track/album/artist works really well for the genres I listen to, when I'm tired of my playlists, or when I just want to discover new music, a starting a radio from a track, album or artist I like usually gives me very relevant content.

Nice "hack" with incognito! I also notice that Youtube will catch on to a random video I watch and recommend me similar things, even if I am not interested in those topics. I'll think about incognito in the future! "Right Click -> Open Link in Incognito Window"

I always start with a fresh session and never log in. Then if I'm researching a particular topic, I always get recommendations on that topic. If something is worth saving, I bookmark it like a regular page to access it in future sessions.

My biggest annoyance is that it suggests videos I've already watched. Old and new. I totally understand if a recently uploaded video gets suggested several times a day or over a few days, but it suggests videos I've watched years ago, and also months ago. Over and over again.

Their "don't suggest this video" or channel doesn't work for shit either.

inject those two styles into YT website:

    a #video-title { color: red !important;}
    a:visited #video-title { color: black !important;}

That doesn’t help across multiple devices

Are you sure the video was registered as seen ? Sometimes on my phone I watch a video and I don't see it in my latest seen videos. It's an issue with Play Service I think.

I constantly have the same issue. Youtube even shows the video as already watched. I click not interested, then already seen, but another one pops up in it's place. Quite annoying.

I think youtube recommendations are a sneaky and opaque thing. Whatever they're doing, "it works" well enough for their ulterior motivations rooted in surveillance capitalism.

The OP is absolutely right in wanting to regain some control over what's presented to his eyeballs. Right now the only way you can do that is to right-click on an offending video and select "don't recommend this channel" or "not interested". That only partially works.

I wish there were some way to specify actual words in a "black list" such that videos whose titles or descriptions contain a black list word would NEVER be presented as a recommendation. This is sort of like Twitter's muted words list. It really is the only way to block content that you really don't want to enter your headspace. Not perfect, of course, but better than being left to the whims of pavlovian algorithms coordinated by ever-improving AI.

I once made the mistake of viewing a Jordan Peterson video. It was mildly interesting, I found him somewhat provocative but a bit paternalistic, not my cup of tea, no big deal. But then... I got a ridiculous number of men's right's or "red-pill" videos recommended to me which were totally disgusting. It's easy to see how people can get radicalized or worse all because somebody is paying Google money for clicks and Google is, in spite of whatever they say, disinterested in our well-being.

I have two YouTube accounts I use, for watching in different languages.

In one language, the suggestions are pretty good. I've found plenty of new stuff that I otherwise wouldn't have.

In the other, it basically recommends a mix that is 80% videos from my subscriptions that I've already watched 10+ times and 20% new stuff that I just don't have any interest in. I wonder if there is some sort of failure mode I'm hitting.

So, I went to youtube.com right now to check what my recommendations are. Lately I've been watching a few videos about music theory, and some old jazz stuff. And now YT recommends pages and pages of music theory.. hundreds of videos. All the recommendations (useful ones) I saw in the past, about history, old computers, natural sciences.. all gone. I would prefer a slightly more balanced approach from their algorithms.

Ja I'm with you.. I'm super amazed at the channels, doccies, channels and cool videos I did discover ! Now a "real F1 score" is not really possible with the size and nature of YouTube ( Recall VS Precision) but still I find 6/7 days I get recommended cool stuff 3/5 times :) My gf(ok she is now my ex since 2 weeks ago) use to also complain at how bad YouTube algo was for her, but she did enjoy my videos and channels I got recommended. The only difference is I think, was that I spend a lot more time on YouTube then her ?

This sounds like from 10+ years ago. It did that discovery thing for me, but only for a while. Today my youtube is an isolated cell that I have to forcefully break out of to discover something new. As if they were not interested in me consuming new content at all.

Seconded. I find YouTube’s algorithm to be fantastic (maybe I’ve given it too much training data). It does a great job suggesting videos about my various hobbies, movie clips and obscure music recommendations. Surprised it works so poorly for others.

The surprise is mutual :)

(Edit: I'm talking about related videos here. Maybe that's not the same thing.)

5 to 10 years ago I would frequently get lost in Youtube's music recommendations, just going down one rabbit hole after the other with dozens of tabs of untrodden paths. Nowadays everything seems to loop you back to the same tightly knitted clusters of popular music. I suspect that if you were to graph the video recommendations, the resulting graph would be of a devastatingly low complexity.

Yeah same, I guess it depends on how much data they have on you. I've been using youtube and google products in general for a long time and only recently have been breaking away, but I'm sure that that has somewhat helped. Especially considering I created a new account and the reccomendations are not so great, and youtubes in general. Still its pretty good so ¯\ (ツ) /¯

What are your interests, both entertainment and music wise?

If you could drop some examples it might be easier to understand why it’s working for you.

(Please see the reply I left to Krasnol). So, if you just gave me a search-bar and have me play music, it would be post-rock or metal. And what I get on spotify is a bunch of post rock and black/death metal. Which is fine, but predictable and kinda boring.

But youtube manages to recommend to me things that aren't in my alley, but I somehow end up loving.

(just a random sample from music) "Heilung - Lifa" - https://www.youtube.com/watch?v=h1BsKIP4uYM

"The caretaker at the end of time" - https://www.youtube.com/watch?v=wJWksPWDKOc&t=10573s

"Babe Rainbow - Secret Enchanted Broccoli Forest" - https://youtu.be/lh2qHxUDt6o?t=26

A whole new genre called DungeonSynth - https://youtu.be/E9R-vzIe8x0

And one called Witchhouse - https://youtu.be/6XIlNP-c4Ds

And Sovietwave goth - https://www.youtube.com/watch?v=9ZZSyO28RUg

Uhh, drum n bass n sax?? - https://www.youtube.com/watch?v=G55GspnNkBo

And the latest weird recommendation a San Jose band called Sunami.

Oh And This! - https://youtu.be/cIMKJ43TFLs?t=76 ---- I wouldn't be searching for any of that on my own. But I love all of it (ended buying merch from quite a few "youtube recommended" artists). To me that is as successful a recommendation system as it can be. It shows me things I didn't even know I 'd love! From Jazz, to Funk, to Ambient, to SovietWave goth, to psychedelic rock, the list is always expanding.... I was never that open minded, but now I am. Maybe this is more on me and how my personality evolved through the years than on youtube's recommendations...

---

But you can see that many people had the same happen to them by how popular comments like this are -> "You didnt find this video. It found you." Or "I would love to know what I did to make this come up in my suggested videos. Holy shit this is awesome!"

Curious, ain't it?

I reckon part of why it works well in this case is, because they are relatively popular (tens of thousands to millions of views) songs in somewhat niche genre. So maybe more similarity between users? In more mainstream genres, say Hip Hop, Rock, Metal, EDM i feel it is much harder to break out of the loop of sameishness although the genres have much more to offer.

Also in either case, I'm pretty sure, the vast majority of songs out there have sub 10k views and basically never get recommended. Just top of mind a few songs, that based on the list above you might like, but I think I would never have discovered via algorithms, unless I already happend to start quite close.

Roel Funcken - Balaklavskiy prospex https://www.youtube.com/watch?v=Z7WVGwb-9x8

Subheim - Foray https://www.youtube.com/watch?v=MQ7IHYSrquc

Klez.E - Nachtfahrt https://www.youtube.com/watch?v=hi-8wkccjsw

Altın Gün - Cemalim https://www.youtube.com/watch?v=j_J3-yr9GNI

Ok, the last one is fairly large, but it came to mind :) I do discover quite a bit of new music on YouTube, but only when I deliberately select a starting song and go from there, on the main page there is rarely something interesting. If there is music at all. I'm sure it would get a lot better if I'd listen music primarily via YouTube.

I have found that Facebook videos always has had much better video recommendations.

It really does sound like a fake review.

Especially considering the fact that you did not even attempt outlining what the reason might be why so many others have the opposite experience.

Just because someone has an opinion counter to the Hacker News hivemind doesn't mean they are faking it.

In this case, I'd imagine this is a case where the people who have a bad experience with the YouTube suggestion algorithm have a reason to speak up but the people who are having a fine experience do not.

I don't know... honestly. I was extremely surprised. I was actually wondering if they switched to some level of human in the loop, because it used to suck in the past, but now is great. I expected people to have good experiences too. I 'm actually shocked to see how disappointed people are here...

Here's an example of some stuff that just showed up recently.

[1] Some guy camping an exploring an island that used to be a federal prison - https://www.youtube.com/watch?v=2xnI1hZdltw

[2] A documentary about homeless people racing carts - https://www.youtube.com/watch?v=zi-f_J6hV-g

[3] Fantastic funk music (and I don't even listen to funk!) - https://youtu.be/m1oLjnKeUiY?t=2

[4] Fantastic old school theatrical hard rock from Japan - https://www.youtube.com/watch?v=CbI79e5iZKs

[5] Sapolsky's must-see lecture on stress - https://www.youtube.com/watch?v=D9H9qTdserM&t=112s (which I got recommended and was watching this morning in the background while I worked)

Just a random sample from stuff that popped up in the recent past. All the above captured my curiosity 100%. Maybe it's a personality thing, maybe I am eager to just discover novel stuff, that the quality level might not be the highest but the novel factor covers for it.

I have the same experience as the above user. I primarily watch guitar and exercise videos, and have never had any complaints about the youtube algorithm. The one algorithm I do feel underperforms is Spotify's. For some reason, their discover weekly is always filled with covers of songs I already have in my playlists.

If you want a non-coding approach to build a filter, take a look at the Firefox Extension "Distraction Free Youtube".

It allows you to hide the Feed, disable Autoplay, hide the Sidebar and related Videos etc., and thus completely filter out every recommendation, now you would just have to subscribe to all the Channels that interest you - tada personalized feed. Of course this has some short comings: Likewise you won´t discover new interesting channels and other novel videos etc., so to speak you´d have to subscribe to every channel that could contain a video of your interest

That´s my system for not wasting so much time on Youtube; thank you Chris for sharing the post, it inspired me to undertake the coding approach, if i have time in the near future, to build a good filter.

Even better, don't login and clear cookies frequently (Cookie AutoDelete addon, or just incognito mode). I just hid recommendations div with adblock. Still, in the end of every video there are 9 top recommendations displayed, so I'm not losing on anything.

I have maybe 4 channels I check out from time to time, and all of them post new videos pretty irregularly, so there is not much point in managing subscriptions. That said, I recently started using NewPipe on mobile and have subscriptions there.

I found YouTube suggestions to be generally useless, so I have just disabled the suggestion sidebar completely[0]. Leaves more screen space for the video.

The reason I found them useless is that the suggestions were too specialised. I usually do not want to watch a dozen videos on the same topic. I usually want to watch one high quality video about it, then move on. Same with channels. I do not want to follow ten channels about each of my hobbies. One channel or two high quality ones is enough.

So, even though the linked project is super cool from a technical perspective, I am not sure how the algorithm could be improved to attend to my taste.

[0]: https://hakon.gylterud.net/tutorials/youtube.html (YouTube self-defence)

I like the blog!

Yeah, that's pretty much my exact gripe - that the algorithm just keeps wanting to show you more of the same. It's like a friend who gets obsessed about one thing after you said you liked it once :')

I'm not sure how best to solve this though

I like when YouTubers talk about other channels they like, and I often go check those out – especially if they are on some completely different topic. Because if someone is making content I like, they might also have good taste when it comes to what they watch. But I think in general the art of "taste-making" remains just that: an art.

Maybe a "this is cool; but enough for now" button? It would remember the category that you like but turn down its recommendation frequency for a duration (session?).

I suspect that it is the difference from other viewers who are reporting yt algo as good. I also want diversity/exploration on unknown topics rather than it repeating the same shit to me again and again. Once a channel/topic is found, I can explore it by myself, no more suggestions required.

I had no idea uBlock could do that!

I just removed my Twitter treding topics, awesome.

Bingo, me too. Did the same for the FB feed.

I find youtube recommendations on homepage really interesting. So much that I wish that article suggestions by Google in Android left side window were as good.

I mostly watch tech, games, dev, toys etc. including rocket jump, Siggraph, viva LA dirt league, corridor, Tom Scott, Vertassium, Steve mould, numberphile etc.

I almost always end up finding something interesting to watch. When it's not, I either mark it as 'not interested' or leave a dislike and it becomes good again

The thing I hate about recommendations on the homepage is clicking one 30 second funny video from a friend results in clicking "not interested" in 30 videos on the homepage to stop them from being recommended. Similarly, even if you use a different account, if you watch something you like and the algorithm decides it's more related to some other topic you don't like or thing you have no interest in it also floods the recommendations with this "fresh" topic you're "interested" in simply because you liked a video it misclassified.

Much like the author I have no problems finding "decent" content from traditional wide audience channels like Tom Scott, Veritasium, Numberphile, Siggraph, etc it's trying to find that 1 video out of the full homepage of recommendations that's not mainstream but actually "very good" and in depth that ends up eating more time than watching the videos - followed by a refresh to apply all of the "not interested"s I added to try to steer away from certain content. Definitely going to try this repo when I get home.

Do reduce the distraction of YouTube, I stripped off all the suggestions and comments from it. This makes it so YouTube just shows a single video, and doesn't send me down a rabbit hole.

I used stylebot chrome extension with this css:

  #masthead-container, .videowall-endscreen, #secondary, .ytd-comments, #upload-info, ytd-video-owner-renderer, .super-title,.ytp-ce-element, #items {
    display:none
  }

and I used CJS to add this javascript to the page (I did this first, so possibly the css made some of this not needed). Be sure to turn jQuery on.

  const $ = jQuery;
  function hideThem(){
    if(jQuery('.ytd-compact-autoplay-renderer#toggle')[0].checked){
      jQuery('.ytd-compact-autoplay-renderer').click();
    }
  }
    hideThem();

  $(document).ready(()=>{
    hideThem();
  });

Instead of the lambda workflow, you could just stick it on a server in a cron job. I pretty much do that with a bunch of things on DigitalOcean, although there's a ton of comparable services.

ah okay, thanks. Yeah I think I may have over-complicated things by trying to use lambda. Will look into doing this on DO

Since your code is already hosted on GitHub you could also try using GitHub Actions with a scheduled trigger. https://docs.github.com/en/free-pro-team@latest/actions/refe...

If you already made an AWS account, the t2.micro EC2 will work as well, free tier!

Dumb quantitative approach may work for some. Views and subscribers counts are close to irrelevant IMO when it comes to niche or expert subjects. Freshness is irrelevant for woodworking.

I would prefer saved searches. But YouTube doesn’t seem to index the auto generated subtitles.

I want something like AdBlocker extensions but for replacing algorithms of these social media platforms. It can be community driven so I can select “give me scientifically proven posts” algorithm or “non political stuff” algorithm.

As image recognition of objects seems pretty far on, I'd like to block all images of pets ... sorry, I don't care about your pets.

Seems doable.

What other features would you want in that?

This !! I want to be programming bubble. nothing else. I wish google had a way to choose/switch what bubble I want be in. Conspiracy, tech, flat earth etc.

This is an awesome idea. Imagine if the YouTube platform itself let you swap out recommendation algorithm in your settings with alternatives like this one.

This is awesome. So it errs on the side of hiding good content that you would want to see, while YouTube's algorithm errs on the side of showing you clickbait.

I would absolutely love this as a browser add-on. My current approach to avoiding distraction on YouTube is an add-on called "Remove YouTube Recommended Videos, Comments" which does exactly what its title says it does, but leaves the actual YouTube page quite bare

Why doesn’t one pick the videos they would like to watch; and put them in a queue...

so you’re fully in control? Just my two cents.

Too much work.

YouTube or Google search business need algorithm change. They need to go back to what they had 7-8 years back.

Tiktok had the best recommendations system so far. They are designed to keep you hooked. YT they are designed and give feel you are wasting your time on this so go back to your life.

Not sure if search engines can provide user defined algorithm plugins for searching or day use search algorithm 1 or 2.

Most of the time feel like I am restricting myself from watching new videos in youtube because it might think I like this starts to recommend that will be a problem. So I don't even anyone my mobile to watch youtube videos. They should have incognito button so that I can watch new stuff.

Youtube app has an incognito mode.

Note that a supplemental option is to also delete your YouTube account.

I was worried I'd accidentally nuke my gmail account, but turns out you can delete your YouTube account + history without affecting any of your other Google services (I did it last night).

Youtube has some quality content that is not available elsewhere. For example it would be illogical to shred your library card just because you don't like the books that the librarian shows you.

I'm not suggesting nullrouting the site (if you don't need to), I'm suggesting deactivating your account so you're never shown the all-too-enticing personalized recommendations.

You can always watch specific videos without being logged in.

this is what I do, too.

but lately, with ublock origin ON, I cannot see it anymore

This is great. I feel the same way, but have been trying to train the existing YT algorithm. I started using my work email youtube profile to actively subscribe/like/save videos that I want to see more of (about hiring, management, culture, Kubernetes, devtools etc). At the same time I aggressively choose "not interested" and "do not suggest this channel" when the algorithm isn't suggesting what I want (more detail: https://twitter.com/GrantM/status/1325471071265558532).

Don't bother: I tried literally for years to teach the algorithm that I don't eat fish but it stills offer fish cooking videos to this day.

Wouldn’t it just be easier to start eating fish?

I'd rather skip a meal than eat fish. In fact the last time I ate fish was 22 years ago during my compulsory military service which I still have nightmares of (but not because of the fish).

So I'll resist the algorithm until the end! ;)

Sounds similar to how I trained my TikTok feed while I was playing around with it for a week - repeatedly clicking on not interested for dancing, cooking, and general youth stuff left me with a feed composed of nature videos - pretty cool.

On a side note, tiktok has to have the best recommendation algorithm possible - not good for us, but good as in addicting.

interesting - sounds like you've had good results so far?

In my experience the YouTube algorithm is very easily manipulated. Just scroll through your feed and select "not interested" for what you dislike. The type of content changes very quickly.

Over the years I’ve probably „blocked“ every single term from the Star Wars universe. Yet as soon as the next iteration of the franchise approaches box offices I find myself flooded with the most absurd cross-marketing videos you can imagine. Not talking about (official) ads. I have the strong impression the algorithm either takes ad campaigns into account — or is being abused by advertisers very successfully.

Makes sense, ads are payed per impression, so if a company soaks away more impressions that don't lead to actions then that means more revenue. I guess the trick is to make sure your still - only just - the best cpm around.

For me, the YT suggestions are so bad I consider it to be a feature. If they were any good, I'd waste time on Youtube. I like this submission, but I honestly don't want to improve my Youtube experience.

This inspired me to think: would it be possible to employ an opposite approach - negative selection. Develop a browser extension that hides bad results (based on smart filters) instead of trying to guess the best?

really interesting suggestion, I like it

what do you think would be good markers of a bad video? other than the obvious (clickbait terms, repetitive/generic titles), I'm not sure what could work. any suggestions?

Personally GRAPE content (Guns / Religion / Abortion / Politics / environment) especially from opinion news sources.

I notice that when I consume these content I never end up feeling better than when I started regardless of whether I agree / disagree with them. Also a huge time sink.

Also, entertainment content about celebrities like the kardashians.

I'm thinking maybe an extension where when you sign up, it asks: "what do you NOT want to see", and then for all topics you pick it will block videos on those.

Kind of like when you sign up for Twitter or Medium, it asks you for topics of interest - here it would do the inverse

How does a non-coder use what you've created?

I want something that will allow me to add interesting videos to my "watch later" playlist. That way I can then watch the videos on the normal YouTube app on my apple tv

Yeah, it's not that easy to at the moment. I was planning to build a front-end site, where you could just type your search terms and parameters and it would show you the videos, but to honest I ran out of steam after I couldn't get the code to run on Lambda.

Ok, fair enough

One feature I wish Youtube had was a preview thumbnail that was made of 10 screen shots from key moments in the video, side by side. Would make it easy to filter out all the talking head. And right away see if its worth watching.

Does the video teach you something worth knowing? Feel like this could be answered by taking the captions and ranking the difficulty of the language used, or making a word map. Then have AI deciding is this about math, science, politics, and so on. Even simple as rating the words on a grade reading level would help to filter out useless videos.

You might find AWS SAM CLI to be useful for packaging your code artifacts. I'm not sure if you've found this yet, but even so, it can be finnicky sometimes.

Particularly, the "sam build --use-container" command might be useful for simulating the Lambda environment

https://docs.aws.amazon.com/serverless-application-model/lat...

I wish the author would also do this for PeerTube.

Ah, good suggestion! Let me look at what I can get from their API

https://sepiasearch.org/ might provide a better API for you.

yes. as the other commenter said, sepiasearch is the index for around 500 peertube instances. also, can this be implemented in the browser as a website? a recommendation engine better than what we have right now would be nice

I think it's better just to hide the suggestions by default. I use the Improve YouTube! extension

https://chrome.google.com/webstore/detail/improve-youtube-op...

Is there anything to help one stop wasting time by reading about how to stop wasting time?

People must waste more time reading about how to not to waste time than anything. Just like all these people being unproductive while reading about how to be more productive.

Of course the solution to all this is self-discipline but that doesn't sell.

I've done a lot with Lambda, but only with node and go. I've never hit the problem you found -- perhaps translating to another language would work? I've heard that a lot of pythonistas like go, and it doesn't look like you're doing anything too difficult, so it should be possible.

Interestingly, the ranking approaches used here are quite comparable to how those big giants do their recsys algorithms. The algorithms are never perfect and there will always be edge cases that even machines learning models cannot handle decently.

Can this be reimplemented to support peertube? The algorithm would need to be changed to not work on views and subscriptions because those figures are too low. Maybe some other rules based on engagement? Time? Comments?

Yeah, so someone else suggested this too - I'm going to look into the API and see what information I can get from it

great idea! thanks for sharing this.

Recently I noticed I get better suggestions when I am logged out [0]. I am still convinced someone could implement a recommendation system that can suggest more interesting videos based on my history. And it would make me spend more time on the platform so it would be actually win-win if that could have been done by youtube but given their current priorities on monetization I don't know if there is hope. I will give that a try.

[0] https://news.ycombinator.com/item?id=25177807

Ah, interesting. Yeah I get the impression the algorithm really tries to feed you more of what you've watched, whereas sometimes the more 'generally' good videos are what you would prefer.

I think you're right re: monetisation and incentives tbh, I can't envisage it changing any time soon

It looks like I have to pay for Medium premium in order to read this story... can anyone give me a tl;dr? Or it looks like I can use Google cache text only to get around the paywall but that seems sub-par.

Didn't realise Medium was like this - I've just uploaded the article to my personal blog: https://chrislovejoy.me/youtube-algorithm/

Am going to see if I can update the link above aswell.

Here is what it looked like for me after I signed in to Medium[1]. As far as I can tell there is no way for me to read the article without subscribing to Medium. Incognito allowed me to bypass this requirement as suggested by another user.

[1]: https://ibb.co/rQ0kVxq

Ping dang or send them an email through the contact link at the bottom of HN.

Just for another anecdata I must have clicked two other articles hosted by Medium at some point today so I also can't see it - cheers for the alternate link!

Thank you - I was getting the same from Medium. Just slightly less pernicious than Facebook. Slightly.

Try it in incognito tab in Chrome, or equivalent in another browser; that (Chrome Incognito) normally sorts it for me on Medium.

I usually do that but was presented with this text from medium after I signed in:

> Not every story on Medium is free, like this one. Become a member to get unlimited access and support the voices you want to hear more from.

Which makes it sounds like this is a paid article regardless. Incognito worked though thank you.

I'm pretty sure the intended reading of "Not every story on Medium is free, like this one." is "This one is free, but not every story on Medium is."

I think you're right however the only action it gave me was an "Upgrade" button. As far as I could tell there was nothing I could do to continue to read the article without subscribing to Medium. Here's the only thing I saw[1].

[1]: https://ibb.co/cLxMDMV

To avoid this type of spam or paywall:

  1. Open it with Incognito
  2. Setup and forget: use https://github.com/iamadamdev/bypass-paywalls-chrome (also for firefox)
  3. using a website like outline: https://outline.com/7eq6fK

(this one has the article)

The results don't seem very interesting. The highlighted results are just generic vlogs. The highlighted GPT-3 video was just someone sitting ina chair reading text.

I find the most useful recommendations in the list called "Viewers of this channel also watch..." or "Viewers who liked this vide also liked...".

Great project. Seriously though, Be better at search YouTube. Get out of the recommendation business early.

I really wish youtube allowed me to block channels, so over time the quality of videos I do see improves.

It does allow you to block channels. It's the option "Don't recommend channel".

This seems to be only available on the home page, but thank you I didnt know this exists. Do you know what the option on the video page itself does "Not interested"?

You can block channels. It's in the same three-dot menu that lets you remove individual videos.

Quality of channels can go up and down though? Surprised if disliking a couple of vids from a channel isn't enough to prevent getting suggestions for that channel?

YouTube recommendations are generally not good. After I activated my pi-hole it went completely useless. The other day I was suggested Japanese motorcycle drifting.

I wrote my masters around the YouTube algorithm and the implications on politics. Some of my findings: - The algorithm works differently in English or other languages; - The number of views is not a very important metric on the recommendations - The number of interactions is a much more important metric for the algorithm - The algorithm seems to be fairly simple: the more a video has the searched keyword, the better chance you have for it to rank higher - YouTube doesn't necessarily put you on a "bubble" as defended by some authors on other social media platforms. It suggests videos that are related but if no other videos are relevant, it has no issues with suggesting content that is contrary to the initial. An example would be: "Donald Trump speaks about the elections". If no other Donald Trump content is available, it will suggest content about the elections, and this might mean a video about an opponent

very good!

I guess there is no way to have the same result without the need of the API key (I do my best to stay away from google)

You could perhaps use Invidious's API?

Invidious is "an alternative front-end to YouTube" which can be self-hosted or you can use one of the popular hosted instances.

https://github.com/iv-org/invidious/ https://github.com/iv-org/invidious/wiki/API

Really love the intent there. Great stuff.

thanks John :)

Could this become smarttube.com?

Reply on news.ycombinator.com