ByteDance's Recommendation System
github.com
github.com
1. It doesn't even claim to be that.
2. It's over a year old - so even if it was, this is no longer it.
3. There's zero incentive for them to release it, but every incentive to release a fake one.
4. TikTok is too much of a national asset, I doubt the Chinese government would not use it to their advantage (and transparency would be counter to that).
Edit: there's also https://news.ycombinator.com/item?id=42471278
They clearly built something superior. And it can't seem to be matched by the biggest tech companies.
Tiktok was allowed to establish its own brand and develop a community while shorts and reels are intrinsically tied to their past. They may be able to escape that history but I don't think it's helping them be fast movers or win "cool" points.
My intuition would work the other way around. I'd expect offerings from more established companies to have a big leg up in terms of usable data. Youtube should be able to use a viewer's entire watch/subscription history to inform itself about what shorts a user might like, even before they've interacted with their first short. Bytedance, on the other hand, has to start from scratch with each truly new user.
The coolness or stodginess of the company would be secondary to its effects. If boring-old-Youtube could promise shorts creators great exposure to an enthusiastic audience, it would win the platform regardless of its brand.
https://www.threads.net/@mikeynerd/post/DB7DS7LzsVU
TikTok gets a definite thumbs up or thumbs down for every video it shows you whereas if you click on one particular sidebar video YouTube can make no conclusion about how you felt about the other videos in the sidebar. The recommendation literature talks about "negative sampling" to overcome this, I never could really believe in it, I think now it doesn't really work.
I built a system like that and found that, paradoxically, you have to make it blend in a good amount of content that it doesn't think you'd like for it to be able to calibrate itself.
Just a guess, as someone who makes their living from YouTube: YouTube creators are driven to create content that earns them money. As compared to long-form content, YouTube shorts earn next-to-nothing, and it’s not clear that they drive significant new traffic to more-valuable content.
Most large creators on YouTube are focused on the bottom line, not exposure.
It’s not just “did you click the like button”. It’s “did you swipe it away? How long did you watch until you swiped it away? Did you come back afterwards? Did you let it loop multiple times before moving on?”.
They’ll capture likes and dislikes you yourself probably didn’t even knew you had, just from tens and hundreds of these micro actions. And they’ll do it in the very first hour of you using the app, whereas YouTube won’t know too much about you even after months of you using it.
I haven't worked on sites as big as YouTube but on sites with 100,000 members who are very much engaged with one "game" you usually find they are mainly indifferent when you offer them another "game" to play.
I like YouTube for what it is. I have interacted very little with shorts but Google has scarily seen into my imagination. I don't want to go into that rabbit hole.
Two populations can be similar in terms of conventional demographics such as age, gender, race, what kind of clothes they wear, etc. but be different in their behavior. IG users are "players of the Instagram game" and TikTok are "players of the TikTok game" and a whole system of values and behaviors are involved.
To take an example playing the "engagement farming" game on Bluesky I can follow people and know some fraction of people will follow me back, but who do I want to follow?
I postulated that the people I want are people who will repost my photos so I can try following people who repost photos but I find that reposters are not "followers" whereas I get a much better response rate if I follow people who follow another social media photographer since those people are "followers". People have an online behavior signature like that which for me matters more than the color of your skin.
It has nothing to do with the quality of the algorithm. In fact the YT algorithm has gotten worse since they introduced shorts because they shove shorts into people's faces.
A better question would be why is the regular YouTube algorithm so bad. And the answer is because it doesn't optimize at all for the consumers, but for the producers (producers of ads, that is). TT has figured out it doesn't matter what people consume as long as they consume, whereas YT is bullish into controlling what people consume.
Their algorithm is really built around their features. Specifically, temporal representations of user interest:
https://ieeexplore.ieee.org/document/9458799/
The features used by their algorithm tells you what a user is interested, historically.
Contrast this to Meta, which uses the social graph as their features. Imagine features like the number of times a user likes another author's / cluster's content.
Tiktok will serve you $TOPIC because you have $INTERACTED with $TOPIC historically.
Meta will serve you $TOPIC because you have $INTERACTED with $PEOPLE who post $TOPIC, historically.
Meta only coincidentally gives you what you like.
Tiktok knows what you like.
This is the difference. This is why IG is losing.
Don't forget Friendster!
Mainly bad-luck in being too early or too late, though MySpace had some self-inflicted problems with performance and overcomplicating the interface (by allowing customization).
Then again, perhaps this is the algorithm's way of scaring me away to save precious bandwidth, knowing full well that I will never buy products from online ads anyway?
will be far more interesting to know say what is the difference, what got changed/removed to make them feel comfortable that such an open source variant won't get them into troubles with some 3-letters-acronym agencies back home.
Releasing the recommender on Github is a way to try to diffuse that criticism. But it's just one part of the puzzle that is Tiktok's content distribution.
Of course, when the conversation is about TikTok, this often becomes accusations of propaganda.
But YouTube, Facebook, and Twitter all exert significant control over their algorithms and things like their Homepage, Trending Topics, etc. The conservative right often labels such curation as liberal propaganda.
But at the massive scale of Meta or ByteDance, there is a difference between removing problematic content and actively promoting content. They’re two sides of the same coin, but the first is applied based on reactive guidelines (“we’ve previously decided this kind of content shouldn’t be here”) while the second is ultimately an in-the-moment opinion on whether more people should be seeing the content. The line is blurry, but these are not the same thing, and vibes-based content promotion is easier to manipulate.
Are there CCP agents working at ByteDance? Of course there are because it’s practically mandatory — just like American telecom companies have NSA wiretap rooms. Do those CCP agents get consulted on which foreign political candidate should get the viral boost? Perhaps not. But it appears they’ve built a system where this kind of thing is possible and leaves little paper trail because the curated boosting is so integral to the platform.
I explicitly did not mention HackerNews, as the homepage feed is primarily based on user voting - neither algorithms nor chronology. Dang’s moderation is not comparable to other social media platform’s feed curation.
> there is a difference between removing problematic content and actively promoting content
Again, there is sufficient evidence that all major social media platforms do exactly this, not just TikTok. Hence why I said:
>> The conservative right often labels such curation as liberal propaganda.
> where this kind of thing is possible and leaves little paper trail
Could you point to the paper trail that Meta, Google, Twitter provide on their curation actions? Otherwise, this just proves my point that people blindly want to accuse Chinese platforms of shady activities, and Western ones as paragons of virtue.
Purely chronological sorting of posts works the best and is fully automated. It's just really bad at keeping users eyeballs on ads.
Is this just your belief or is there evidence you can point to.
How would you differentiate manual intervention from algorithmic intervention?
Instead, it is likely a component that powers ByteDance's commercial recommender system solution, which they market to e-commerce companies: https://www.byteplus.com/en/product/recommend
This was mentioned in past discussions of the paper on HN.
And even if aspects of this are used for TikTok: (a) it would be just one of many components of their recommendation system, and (b) the TikTok recommendation system has changed a lot during the 2+ years since this has been published.
So take what you see here with a grain of salt. After reading the paper and the code, you will NOT know how TikTok's recommendations work.