https://blog.youtube/news-and-events/youtube-now-why-we-focu...
https://www.eugenewei.com/blog/2020/9/18/seeing-like-an-algo...
There’s a more technical recent paper from bytedance as well: https://arxiv.org/pdf/2007.07203.pdf
And another recent one on bytedance user profile system, this paper gives the deepest understanding of their recommendation system: https://www.cs.princeton.edu/courses/archive/spring21/cos598...
There’s a good breakdown of the recent nytimes article on TikTok’s internals here: https://read.deeplearning.ai/the-batch/issue-122/
If anyone has found anything better let me know.
If you try searching yourself you will want to try switching between the keywords “douyin” “toutiao” “bytedance” and “tiktok.”
I know that they probably optimize for ads etc. but if they actually showed me videos I would like to see, then I would spend more time on the platform.
Not directly, most people believe they optimize for session time. It tries to serve you a result that will keep you on the platform watching videos as opposed to needing to keep scrolling or leaving the platform. They truely do want to serve the best videos that they can to you that they think you will be interested in watching. Thankfully for YouTube you watching videos and YouTube getting ad rev is correlated.
>where will you go and watch medium to long videos created by "normal" people?
No one forces you to watch that format of video. You can go on TikTok, Twitch, Netflix, etc and be entertained.
For example, the HN front page is a recommendation system if you literally mean system-that-recommends-web-pages-to-look-at. But it's not personalized; every visitor sees the same front page. This fundamentally makes it a different sort of thing.
I don't think they're really paying much attention to the dimension you're splitting it along, i.e. whether the recommendations are personalised for each user. The huge important idea they have in their head is that recommendations can apply to user-generated social media content too.
* HN and classic Reddit sort their items on a single dimension ("hotness"), calculated using a few input variables and producing a single output variable. This is about as cheap to calculate as recommendation systems get. The XKCD comment recommender is a bit more complex, but still in the same complexity class. Since the whole point of an algorithm like this is to be timely, the naive approach is to compute it on-the-fly, which it's perfectly simple enough to manage.
* At a somewhat more complex level, you get stuff like a basic, uncustomized Similar Items list. If YouTube has no data on you, this is what you get from their sidebar recommender (and their front page would be analogous to Reddit and HN, but sharded by region and language). It's also pretty close to what AdWords used to be, before they started doing user profiling. The thing with this method is, even though it involves some level of AI, it's presenting the same thing to everyone and it's expensive, so the natural solution is to precompute it.
* Personalized recommenders are the worst of both worlds. You can't naively compute it on-the-fly, because it's too slow, but you also can't naively precompute it, because there's a combinatorial explosion of users and items. You actually have to be clever about it.