Twitter showed us its algorithm – what does it tell us? | Hacker News Reader

Twitter showed us its algorithm – what does it tell us?

knightcolumbia.org

225 pointsrandomwalker163 comments

I would assume that if "probability that other users will positively engage with a tweet" is the primary determiner of reach, then the more you can help Twitter accurately predict that probability, the better, because otherwise the default probability is likely no higher than middle-of-the-pack.

If that assumption holds, then I would guess this type of algorithm favors consistency of content. In other words, someone who picks a certain topic and consistently tweets only about that topic is going to be easier to form predictions around versus someone whose tweets show much more variety, in topics, styles, etc.

What that might mean, from a "gaming the system" point of view, is that if you're a person who intends to primarily tweet about two or three disparate things, you might be better off creating a separate account for each, rather than a single account where engagement is harder to predict.

Incidentally, those same conditions seems to apply to other sites. Lots of Youtubers have multiple channels (a main channel, a livestream channel, a shorts channel).

I've thought many times over the years that I would love to be able to subscribe to a particular playlist or "show" from a channel. There is several channels that I want to see their main stuff, but not their side content. Or a particular game from a lets-play-er, but not their other games.

Surprisingly Youtube did have that functionality, though it was removed quite a while ago. I specifically remembering being able to subscribe to "Is It A Good Idea To Microwave This?" (in the late '2000s time range), without also subscribing to the other videos on the channel.

It doesn't work exactly the same as a subscription, but you can add a playlist from a channel to your library, if the channel offers one that fits what you want.

For example the channel Accursed Farms puts out a Half-Life based show called Freeman's Mind which I love, but I'm not huge on the rest of their content, so I follow their playlist and can easily get just the content I want.

Pitch meetings and Ars Technicas interviews about old games come to mind. Fortunately, pitch meetings got his own channel last year.

Same here, but cynic in me says that they want to increase your engagement by spamming you with more content that you MIGHT also want to watch.

There's nothing cynical about that take, it's a fact this is how YouTube (and modern social media) operate. Longer eyeball time == more ad revenue.

I don’t think it’s only helpful to the algorithm. It’s also helpful for me as a subscriber.

If I want to watch episodes of Breaking Bad, I don’t want you to randomly throw in episodes or M*A*S*H (even though both are good).

I wrote about this - https://www.atomic14.com/2023/03/06/the-curse-of-youtube.htm...

To be honest it's pretty horrible. Why can't I just post about what interests me right now?

When you say:

> Why can't I just post about what interests me right now?

You mean you can’t post without probably messing with some bizarre algorithm that is running YouTube discovery?

Because I’m quite sure you can post about whatever you want.

The problem I see is that you’re mixing two aspects of the creation process.

There’s your desire and curiosity driving you towards creating videos on one end there’s whatever YouTube wants on the other which usually is keeping people glued on their phones to consume as much content as they can.

It’s up to you to decide what is more valuable here. If you want to play the YouTube game you have to accept their idiotic rules which includes posting daily, making stupid shorts, making reactionary content and all that.

OR you can decide that it’s more important for you to make videos you find interesting and stop caring about vanity metrics.

Side note: those three images in your post are completely unnecessary and add nothing to the overall quality of the post. I’d personally remove them and stick with just the content.

But that’s just my 2c

This is very true - I try and keep YouTube fun for me - if it gets to the point where it’s the same as work then there’s not much point.

My wife is a amateur painter but would like to make more of it. I always tell he that she should paint what she wants and make pictures she enjoys.

Her art club is currently running an exhibition and none of her paintings have sold so far.

You can, but it's the equivalent of completely changing the topic of a conversation. Do that enough, and people will be less keen on listening to you.

That doesn't really make sense because most big accounts tweet about a range of topics. There's pretty well established ways for estimating the probability based on how the range of topics you might tweet about would match with the range of topics a user likes. That means you have to try and figure out what your base likes to see, and be like that. Tweeting about only a single topic means that you're only targeting people who are likely to like tweets from accounts that tweet about that one topic.

> That doesn't really make sense because most big twitter accounts tweet about a range of topics.

People engage with celebrities on everything. If an A list celeb announces they enjoy a slice of lemon in hot water, twenty news articles will be published around the world.

I enjoy a lemon in hot water with a lot of hot sauce and I breathe it in through my nose to clear sinus issues.

It's probably less this and more "if you only talk about one topic, then when we show your posts to similar users, they are more likely to like it"

As the article calls out, the code is right there. Post your results of tests, not knee jerk conjecture. Wrong opinions are a dime a dozen.

Uh, the article also points out that the algorithm itself can't tell you anything definitive about how a given tweet will be ranked without the training data.

In all the infinite wisdom of developers and tech gurus, we get dictatorial systems that rigidly force dumb and obtuse rules and stupid "success scripts" on creative people that only end up rewarding trust fund babies, scammers, and/or hackers... Leaving the most talented and original creators to wither in obscurity.

Social media sites like Twitter (Now Titter) have brought out the worst in us, and now that they're collapsing under their own arrogant leaders, no one is really considering the lessons learned... They can't exist without creators, but creators can exist and function on any platform. Screw Twitter's dictatorial algorithm, mastering it does not help anyone to be creative and successful unless they're trust fund babies, scammers, and/or hackers.

Let's be honest here: vast majority of "talented and original creators" on social media are the ones seeking out, sharing, and geeking about "success scripts". And, of course, social media isn't unique here. This is a story as old as time: any activity that has a competitive element to it gets dominated by those "playing to win". It's true in every aspect of life. I feel the only way to avoid it is to suppress the entire competitive angle - which means removing the rewards altogether, or making them purely random, as any correlation between activity and reward will attract people willing to game the activity.

You can also throw the ball, aka gameification of parts of the activity, so the success nerds can be busy romping and playing, while the other grown ups get on with the actual fun part.

Unfortunately, "success nerds" end up owning their social status, and they also generate buzz. Frantic activity is a catnip from entrepreneurs, so sooner or later, those "success nerds" end up being the ones making real money off their "success", and as money tends to corrupt things, your activity becomes controlled by the "success nerds", for their own benefit, and the benefit of their sponsors.

See also the Iron Law of Bureaucracy[0], which is closely related[1].

EDIT:

The "success nerds" are usually the grown-ups, encouraged by non-participating grown-ups who see the entire activity as just a money printer.

[0] - https://en.wikipedia.org/wiki/Iron_law_of_oligarchy

[1] - I don't think it's an accident. Those are the same group dynamics at play.

Man, all this effort when all most of us want is to see tweets from the people we follow in chronological order. Absolutely wild.

Wouldn’t this be unusable? Just a huge flood of tweets that you have no hope of keeping on top of?

Check out Mastodons federated feed, it’s a mess but fun place.

This is what Twitter used to be - I would log in, scroll down until I had seen all updates since I last checked and then left the app. This was, however, before you got tweets in your feed from people you don’t follow (outside of retweets)

is that so different from the current system? Besides, its highly dependant on how many people you follow, and whether you care about seeing every tweet from every single one of them

Only if you follow way too many accounts.

You could/can get this with Twitter lists, and it's actually way better than anything else.

That's why you only follow specific people, and there's also a couple of tools left to tweak the timeline further (e.g. very useful: disable retweets from specific 'noisy' peeps).

If you follow thousands of people then maybe. But in that case you're going to miss tweets anyway, so if you've got way too many tweets buffered and you want to see more recent stuff you can opt to skip them.

Works just fine on Mastodon for me - I've got the people who I follow and I see all their posts in chronolgical order. There's also the federated which flows past swiftly and is quite fun to dip a net into when something interesting drifts past.

Wow, you must be young. This is how the internet used to be, and people liked it.

Is that the point?

Then again if that is indeed the point, just follow fewer accounts.

RIP third party clients.

This actually made me stop Facebook. I so often saw something interesting, and was never able to find it back later. It's so incredibly frustrating. You leave the site in one state, you come back and there is not relation between what you left and what you find.

Thanks for breaking my habit I guess.

I use lists for that. I also get no promoted tweets or anything like that. Just chronological order tweets and retweets by the people that are in it.

That's a good idea actually. If I get bored on Mastodon and feel like I wanna give Twitter another whirl I'll try that. I went opened it up a few days back and saw a promoted post for every ~8 real tweets or so. Unreal.

How is Mastodon? Does it rival Twitter in terms of usefulness?

I like it, I happened upon a good collection of accounts to follow by chance and it worked out well. I really only switched because the company that made the Twitter client I used (Tapbots) made a Mastodon client (Ivory) shortly after Elon yanked API access for most third party clients. My acct is in my bio if you wanna take a look, it’s pretty chill but def less busy than Twitter

Aka the reason I stopped using facebook. I don't want you select who you think are my best friends, I don't want you to randomly sprinkle in group posts, I don't want to be reminded of random events from years in the past, I don't want notifications that a random friend I've not spoken to in years has posted a picture, I don't want you to exclude some friends from the list based on gut feeling.

I just want a chronological list of all of my friends updates.

You can already do this right now

This is what people say but revealed preferences say otherwise.

Anyone who is interested should also read https://twitter.com/aakashg0/status/1641976869460275201 for a rather different take.

It particularly interested me that Twitter under Musk is trying NOT to discuss Ukraine, and PENALIZES people who attempt to interact with those outside of their general political circle. I can give arguments for why they should do both, but I think both are ultimately bad ideas.

> It particularly interested me that Twitter under Musk is trying NOT to discuss Ukraine, and PENALIZES people who attempt to interact with those outside of their general political circle.

While Musk's Twitter explicitly censors references to Russia's genocide of Ukraine, Musk himself feigns ignorance and false indignation accusing the "western press" of insisting "on pushing such a lopsided view of the conflict".

https://twitter.com/VsimPohuy/status/1645699649003569152?t=v...

The two ideas aren’t necessarily in conflict with one another. In fact, they make sense to go hand-in-hand. Take for instance affirmative action or reparations. They’re instances of trying to correct for something that’s the opposite of how the group desires it, by doing the opposite of what supporters claim is being done to the marginalized group.

The "marginalized group" in this case are the ones that invaded a country and are killing innocent civilians for natural resources.

From your link (thanks!):

> 9. Making up words or misspelling hurts

> Words that are identified as “unknown language” are given 0.01, which is a huge penalty.

Does that mean if I tweet about coding and use identifiers like "setUserName", which is not an English word, the tweet gets a huge penalty? If so, that's disappointing.

That jumped out at me as a possible misreading of the code. Is it detecting the language of the whole tweet, or just a word as the author claims?

Demoting a tweet that's entirely unidentifiable as any human language seems fair enough.

Man if someone asked me to build a system to merely identify whether a unicode string is human language or not I would flatly refuse. There are thousands of spoken languages, many of them with no standard written form, some that are transcribed into multiple different writing systems, some with no writing tradition at all and with only ad-hoc transliteration unique to each user and use.

Even being 90% confident would be a massive undertaking, and "speakers of this language may/may not use the internet" feels like high stakes for getting it wrong.

It seems a little niche but I'm sure a few times a year some far out town gets connected and suddenly there are speakers of a previously unknown-to-the-internet language newly online.

Note that the metric here is "is the tweet in one of the languages spoken by the user". This hypothetically allows more nuanced implementations than you contemplate.

For example, they could have a language "unrecognized" and assume everyone speaks it.

I broadly find this useful: I see tweets in other languages when they're retweeted by people I follow, and about half the time I machine translate them. But I don't want my whole feed to be that.

Well if someone asked me to do that, I would suggest that it’d be based off their recent tweet history and not just one tweet. And I would make my case in the meeting.

Second, it’s already been done so my next suggestion would be to look what at all the computational linguistic majors have been up to.

I think it would be pretty easy with the language models we have available these days.

And there’s always the options of having unsupported languages or inferring it from user settings or user location.

From a product perspective you will need this feature though if you want worldwide coverage, because very few people are polyglots and most people don’t speak English as a first language.

Yeah that struck me too. I can see the reasons why you'd want it but the collateral damage on that must be huge.

For example do they check for the common but nonstandard transliteration systems arabic speakers use? There have to be similar systems in other languages that don't use the roman or cyrillic alphabets too right?

Or for that matter what about languages twitter simply isn't aware of? There are thousands with native speakers after all, does this make it basically impossible for them to organically use twitter together?

Also RIP the Conlang community on Twitter...

I speak a language that is highly colloquialized, and in its casual written format includes a rather inscrutable system of abbreviation (one of the features of this system is to basically omit nearly all vowels). I had always figured that this language would be impossible for machines to translate, but I just tested it and both google translate and ChatGPT can accurately identify the language and translate the slang into English (Google didn’t pick up some of the subtleties between similar dialects, but still provided a correct translation). So I’m somewhat optimistic that they could be potentially managing these problems quite well.

What's the language?

Indonesian. ChatGPT could quite reliably tell the difference between Indonesian and Malaysian. Google translate seemed to have a bias towards thinking it was Malaysian. But if I tried Indonesian mixed with Javanese slang (which is a common way of talking), they would both just say it was Indonesian. I only tested a few phrases though, so maybe it breaks down at some point.

Why do we assume that Twitter, a global communications company which has had offices in Dubai, might not consider the nonstandard transliteration requirements of Arabic, the 4th largest language in the world, which would mean that they would now be only showing content that is explicitly non-Arabic to Arabic speakers?

We aren’t giving them enough credit here IMO!

The actual code comment doesn’t mention “words” but rather if the “tweet language” isn’t in one of the user’s “understandable” languages. As such, I assume your example is perfectly fine (would be extremely surprising if it wasn’t).

Whether the user implies the reader or author, I don’t know, I assume the reader as that would make most sense.

https://twitter.com/aakashg0/status/1641976943141699584?s=61...

I think the correct reading of the code is that if English is your only language, non English tweets would be weighted 0.01x on your timeline.

OP here. Unfortunately this thread is mostly misinformation. There were a bunch of viral threads from the growth hacker / influencer crowd, including this one, within hours of the code release with a very superficial understanding of the code (and how recsys work in general). That's partly what motivated me to write this article.

See here for a rebuttal of the main tweet in that thread (near the bottom of the article). https://solomonmg.github.io/post/twitter-the-algorithm/

If this is for their Crisis Misinformation Policy why only one specific callout and specifically directed to Ukraine? Seems like a generous assumptions to make on your part that it's a nothing burger. The takeaway we should go with is that we now know that internally they are willing to programatically segment out Ukraine related topics. The question to me that this new knowledge should lead to is why a policy to segmenting this? (not to call immediately jump to 'nothing burger' or as you put it in the above post 'misinformation').

I was not at all impressed with the analysis in that thread. It makes a bunch of assumptions that don't feel very thorough to me, but announces them as if they are unimpeachable facts.

Biggest example is this one:

"9. Making up words or misspelling hurts - Words that are identified as “unknown language” are given 0.01, which is a huge penalty."

The code in the screenshot for that looks like this:

    // Boost (demotion) if the tweet language is not one of user's
    // understandable languages, nor interface language.
    optional double unknownLanguageBoost = 0.01

That doesn't match the description of "Making up words or misspelling hurts" at all!

input youtube thumbnail of cat in the hat enraged "DR SUESS CANCELLED?! TWITTER WON'T COMMENT!" ragebait youtuber.

The systemic racism people are going to go wild about this

I agree with you (although I think one should be upset about this, and you seemed to imply that you disagree with the "systemic racism people").

This would directly penalize endangered languages and dialects, which is a tragic loss for linguistic diversity. I think a lot of people hoped the WWW would connect these small and dwindling communities, and even help revitalize them, but policies like this sound actively harmful.

If we're looking at it through the lens of systemic racism, it seems pretty straightforward as well – anyone who speaks a non-standard variant or dialect of their language will have less power to share their thoughts and ideas. They will also see less other people doing so, creating a feedback-loop discouraging the use of non-standard language.

> anyone who speaks a non-standard variant or dialect of their language will have less power to share their thoughts and ideas

If a recipient doesn't speak the language, the speaker doesn't have much power in the first place. This is noise reduction.

It’s systematic Monolingualism. Which even though this is something that has a big negative impact on me and my particular life, I am willing to first attribute to ignorance before discrimination. It’s just easier to make a product that follows a one country one language pattern. And the numbers of multilingual people are in the minority.

> I am willing to first attribute to ignorance before discrimination

I think ignorance is a fair answer for an individual like you and me.

However when you are a giant organisation, the organisation can be evil without anyone individually being so. In organisation ignorance and evilness is really hard to distinguish, and easy to conflate on purpose.

It is the job of the leadership to not be ignorant, or at least to hire someone who isn't.

Thats why AI ethocs teams where the correct move.

Most of the world is multilingual:

https://www.linguisticsociety.org/resource/multilingualism

Of course in the US this is not the case, and Twitter is a US based company that creates software used globally.

Headline can be: twitter shadowbans cultural appropriation - that should trigger everyone.

I agree that seems presumptive. But of course twitter conveniently didn’t share the source code for how that factor is calculated. Turns out it’s hard to evaluate a big complex distributed system by looking at just one slice and no data.

interesting though, that comment directly says that if I tweet in english and my interface is in german then I'll have a serious problem. Which I wouldn't think is so rare. How many people have english as their second language and use it online? It feels like this deboost would hit a lot of people. Or actually, I have my interface in english and I often tweet in german. So that would hit me from both sides.

"One of the user's understandable language" presumably refers to the list of languages in the browser's Accept-Language header. If you are tweeting in English, you'd likely have English in that header.

It may or may not use the Accept-Language header as well, but this is a user setting accessible via the Twitter web UI-- you specify your primary interface language and can specify any number of additional languages that you understand.

There's also a place in Twitter's settings to show languages that Twitter has inferred that you know (e.g. following a German-language account and interacting with it almost certainly means that you know some German).

Yeah on the surface this looks more like "This user doesn't know German we should make German tweets much less likely to appear on their timeline" kind of thing. Just a complete misinterpretation, presented as fact without any supporting evidence.

Ironically, sounds like a flaw of using Twitter as a medium. I'm sure he has more to say, or at least a more polite way to say it, but there's a character limit.

It’s unclear if it penalizes discussion of Ukraine equally though.

There have been many stories that have come to light in the last few months. Merkel and Macron admitting the Minsk agreements were used to buy time for CIA and British to arm rebels since 2014 was big story. Large amounts of money the US has supplied Ukraine and lack of oversight to where this is going (the total US aid now surpasses Russia’s entire military budget per year). But this same poster (aakashg0) claims these stories have been suppressed, even though they would be counter to dominant narrative in western media.

I think algorithmic moderation on a particular topic is hard; you still need someone in there boosting the stories you want people to read and downplaying the stories you don’t.

Tell us more, who are the "rebels" in this story and what arms did Merkel send?

(Is this what news in the PRC feel like?)

The rebels are right-wing paramilitary groups. And Germany didn’t send any weapons during 2014-2022, but she said in a Der Spiegel interview from Oct 2022 that during the Minsk negotiations, it became clear that the US’ objective was to buy time to secretly arm Ukraine (which is newsworthy because this would imply a violation of the Minsk agreement).

So in a year the US has sent more than Russia's yearly defence budget, yet Minsk (which one, even?) was needed to secretly (what was secret?) arm the Ukraine over 8 years? Who are the "right-wing paramilitary groups" and if they are Ukraine, since this is who you are alleging is being armed, why are they rebels if they are government-aligned?

This is all very easy stuff for you to verify for yourself and wasn’t the original point of my top comment (which was that these stories are hard to suppress without manual effort— although apparently many Americans are unaware).

But to be clear, the US was funding Ukrainian rebel groups (right-wing paramilitary organizations) 2014-2022 but through clandestine means. This is much more difficult to do without the support of congress because the support has to be indirect — the funding has to be off-the-books — because this was a violation of the Minsk agreement.

Since 2022, the floodgates have opened and the US is now openly sending money and weapons systems, now totaling over $100B since the Russian invasion. The Russian Defence budget is estimated to be $70-80B per year.

The gigantic key factor in all this that you're leaving out is that Ukraine is defending itself against a full-on invasion by a hostile neighbor.

The "CIA and the British" clandestine funding of "right-wing paramilitary Ukrainian rebel organizations" with diplomatic support from "Merkel and Macron via Minsk" is "all very easy stuff" for me to verify?

"many Americans are unaware" of this story because it's nonsense that puts together various tidbits to spin a yarn long enough to envelop the world.

Which side in the conflict are you talking about? It is very unlikely that the US was funding the Russian-backed rebel groups. It is likely that they were supporting the Ukraine-supported paramilitary groups. Who are not rebels cause they were government supported and supporting.

I think maybe you misunderstood; nobody in this thread is claiming the US has backed Russian rebel groups.

You literally did. The rebel paramilitaries are the Russian-backed separatists.

The Ukrainian-backed paramilitaries aren't rebels because they support the government.

The paramilitaries I was referring to are these: https://en.wikipedia.org/wiki/Category:Paramilitary_forces_o...

Many of these are right-wing militias that have been fighting in the Donbas since 2014. Some have been absorbed into the UA national guard.

I have never mentioned pro-Russia paramilitary groups; I think you and the other commenter got confused because I didn’t explicitly say Ukrainian paramilitary groups. (Thought it was obvious from context which side I was referring; obviously the CIA would not be supporting pro-Russian militias)

The problem is you kept saying "rebel groups". "Rebel" means that they are opposed to the government. Like the separatists.

Plus, you didn't answer the multiple times when asked to clarify.

Consider my answer [1]

Also I have vouched the top comment of this tree because I consider this point as valid.

[1] https://news.ycombinator.com/item?id=35537017

> The paramilitaries I was referring to are these

None of those are “rebels”, which is the term you used.

> Some have been absorbed into the UA national guard.

The National Guard of Ukraine is one of the “paramilitaries” on the list you linked, so, yes, it is fair to say it has been absorbed into…itself.

> I have never mentioned pro-Russia paramilitary groups

You said “rebels”. The only rebels in Ukraine are those forces consisting of Ukrainians (as non-Ukrainians would be invaders, rather than rebels) fighting against the government of Ukraine, i.e., Russian-backed and Russian-aligned armed groups.

He also said 2014, maybe you need to reread that comment once more. Those were times when Ukraine was not clearly separated from Russia, for example most of big boys in the Government were openly pro-Russian. Ukraine have been a rebel since 1991 and the process is yet to be finished.

Boy, Ukraine have never had a proper borderline with Russian, I mean most of that border was nothing more than just demarcation sticks.

> But to be clear, the US was funding Ukrainian rebel groups (right-wing paramilitary organizations) 2014-2022 but through clandestine means.

You're still being UN-clear. And this far down the thread it's becoming harder and harder to believe that it is unintentional.

"Ukrainian right-wing paramilitary pro-Russian 'rebel' organizations", or "Ukrainian right-wing paramilitary pro-Ukraine 'rebel' organizations"?

I mean, the fact that the Ukraine re-armed itself after Russia invaded their territory isn’t news, is it? I think it was reported on pretty substantially. And a good thing too since they were invaded a second time, this time with a strike towards their capital. I sort of assumed that was obvious public knowledge and don’t understand why people are making it into a “story.”

> Merkel and Macron admitting the Minsk agreements were used to buy time for CIA and British to arm rebels since 2014 was big story.

Wtf are you talking about? What rebels in Ukraine did CIA and British arm? Or is one of those “it’s complicated” comments?

>Merkel and Macron admitting the Minsk agreements were used to buy time for CIA and British to arm rebels since 2014 was big story.

You don't mean "rebels" (who were/are in fact simply Russia proxies), you mean the other side, Ukraine.

Not sure what percentage of all arms delivered to Ukraine has come after February 2022, but it must be well north of 90%. So apparently all these plans to secretly and slowly arm Ukraine amounted to nothing, and it was the invasion itself that triggered the flow of arms. Some dastardly plan.

BTW always love the not-at-all-loaded use of "admitted".

>Large amounts of money the US has supplied Ukraine and lack of oversight to where this is going

These are the kind of fact-free notions that seem to start their life in the TuckerCarlsonVerse and spread outward from there. Just saying there's "lack of oversight" does not make it so, particularly when there's plenty of oversight by the US gov't. In fact, oversight is one reason why the buildup of weapons supplies (in terms of quantity and weapons systems types) has been so gradual. The US wanted to make sure Ukraine's army knows what to do with the stuff and won't leave on the field of battle so that it ends up arming the enemy, like the Afghans did.

>(the total US aid now surpasses Russia’s entire military budget per year).

That's completely meaningless. Russia's military budget pays for many times more personnel and weapons systems than the equivalent number as part of the US military budget, because the purchasing power of, say, $1 million is vastly different when it's spent by the Pentagon in the US, paying US prices and employing Americans, or by the Russian MOD, paying Russian prices and employing Russians.

>>(the total US aid now surpasses Russia’s entire military budget per year).

> That's completely meaningless. Russia's military budget pays for many times more personnel and weapons systems than the equivalent number as part of the US military budget, because the purchasing power of, say, $1 million is vastly different when it's spent by the Pentagon in the US, paying US prices and employing Americans, or by the Russian MOD, paying Russian prices and employing Russians.

It also seems misleading in the sense that, for example, we have a ton of Abrams tanks that the US military didn’t want, but that Congress has over time decided to buy. So if we send them to Ukraine, how should that be accounted for financially? I guess the cost of a gently used Abrams is pretty high but we already bought it and the value to us is pretty low.

This is an example of "a lie can get half way around the world before the truth gets its pants on".

I'm no fan of Musk these days, but there is plainly no evidence in the repo that Ukraine is being suppressed - the linked code is very obviously a model dispatch from an initial classification system, and it makes perfect sense Ukraine would need a call out there since a major war most likely would normally run afoul of profanity, violence, and calls to violence filters without actually violating them.

> It particularly interested me that Twitter under Musk is trying NOT to discuss Ukraine

Because of stuff like the NAFO trolls I can understand why that discussion is not brought to the top anymore.

Anyway, whoever is interested in the war going down there can most definitely have access to both sides' views, I personally find Twitter one of the few media/online outlets that still makes that possible (and props to them for that).

That's a great way to eliminate misinformation, I don't really see the problem.

> It particularly interested me that Twitter under Musk is trying NOT to discuss Ukraine

What makes you say this? I’m aware tweets about Ukraine are down ranked. This seems analogous to subreddits (such as world news, geopolitics, etc) stickying a live thread for the war. This is to prevent it from taking over the subreddit and it happens to slow down discussion.

Seems like the Ukraine war topic is similarly tuned on Twitter. Ukraine can still trend it just takes more activity than other topics.

This is exactly what "trying not to discuss Ukraine" is. Without outright banning it.

When subreddits create a mandatory megathread, it is always with the purpose of kneecapping that specific topic. Putting a large topic into a single thread makes sharing information about the topic extremely difficult.

I'd like to add that there's no reason to believe that such topics that get buried into megathreads would have taken over a subreddit otherwise. The news cycle turns. For example, worldnews today would look no different if the Ukraine war topic had been allowed to breathe. At most it would have caused the minor inconvenience of going to page two for the fraction of people not interested at all in the Ukraine war at that time.

I was disappointed by this article and how it omitted the fact that Twitter hardcoded how references to Russia's invasion of Ukraine should be downranked.

https://news.ycombinator.com/item?id=35410841

I don't know when this dataset was published from, but there was def a time when Ukraine news was beyond saturation.

Because if he's going to claim to be the internet's town square, he can't pick and choose topics that he's personally 'tired of'.

It's the sort of thing I would expect on a highly opinionated Hacker News (iirc like how posts about Apple have a penalty applied to them to counteract massive usual interest in them), but less so on something more general audience like Twitter.

I'm not really looking to twitter to say "Actually, we've all heard a bit too much about Tennis". I don't want Twitter's timeline to have an editorial voice.

All social media algo timelines have an editorial voice. Otherwise, you would exclusively be seeing engagement bait.

> All social media algo timelines have an editorial voice.

Sorry, that's a bullshit excuse. Russia's invasion of Ukraine was hardcoded to be downranked like DMCA violations and high toxicity content.

This has zero do to with "editorial voice" or other bullshit excuse. This was a blatant attempt to smuther any reference on what Russia is doing to Ukraine.

I was simply responding to the comment that was made.

You have basically all traditional and social media pushing pointless and rushed news about Ukraine, a good part of which turn out to be incorrect later in time.

Can we at least have one "safe space" where we don't have to constantly hear about death and glorification of weapons? Especially since we've never talked about Yemen or Syria with the same intensity.

I heard plenty about what was happening in Syria, much of which also involved untargeted Russian brutality waged against civilians. Happy to link you to some of the articles about “barrel bombs” if you don’t remember them.

I followed the Syrian war heavily and the suggestion that its coverage was comparable to the coverage Ukraine got is not true and I think we all know it is not true.

I think we can both agree that the heavy degree of reporting in Ukraine is a good thing.

This must be the peak passive aggressive. Dare to explain why that is a good thing?

You have to actively go look for Syria (or Yemen) news, while Ukraine news come looking for you.

I don't get why you feel the need to get on the offensive about Russia. The only case I'm making is that being constantly subjected to rushed war news pieces isn't of any use except the instigation of primal/emotional reactions.

The most trustworthy information will be disclosed after the war is over anyway, when the need for propaganda diminishes.

I thought the OP was complaining that the horrors in Syria weren't reported enough. From your post I gather that the objection is there is too much reporting on the horrors of what is happening in Ukraine, and you'd prefer to use the example of Syria to justify less reporting on things like, for example, the thousands of Ukrainian children that Russia is holding in camps and not returning to their parents [1]. That's pretty horrifying.

[1] https://apnews.com/article/ukrainian-children-russia-7493cb2...

Thanks for proving my point by spamming me with more unrequested news.

FYI I'll ignore it as I've been bombarded with this children in camps news piece for days already on all the media I have access to.

Since I'm not a hypocrite like you, I can tell you that I don't care at all about what's happening in Ukraine as I prefer living my day without being forcefed terrifying/incorrect news or even worse propaganda. The same way as I ignored what was happening in Syria and Yemen or whatever other war/disaster happening right now in the world.

> Thanks for proving my point by spamming me with more unrequested news.

You made no point. Just because you don't want to follow a topic that does not mean it's ok to censor it and stop everyone from liking things you don't like. That would be a stupid thing to defend, specially considering your content is already served by recommendation systems.

> You made no point.

Oh I've made it very clearly. You just don't seem to be interested in understanding it nor having an honest discussion on it.

> Just because you don't want to follow a topic that does not mean it's ok to censor it and stop everyone from liking things you don't like.

Which I never did or asked for.

On the other hand, you clearly show an intent to force feed everyone else with the kind of news you deem more relevant than others. Don't like the "editorial line" of Twitter? Go read the Kiev Independent.

Why should twitter follow your editorial line of preference, or why should I be subjected to it?

Last week I made deviled eggs with hollandaise sauce. I posted it on a few websites that you'd consider social media. What connection does the war have with my deviled eggs?

If you can answer that and not consider yourself a lunatic demanding your singular topic to be at the forefront for all people at all times, then consider yourself the most important person in the world.

I click away from conversations dealing with AI because they tend to be filled with people scared about something that does not exist or a blatant misunderstanding of what a tool does. I close websites that have enormous spins on topics I am intimately familiar with either lies through omission or generally ignorant statements that demonstrate no understanding of what is the main topic.

Today, after posting this comment, I will click off Hackernews, because your comment is an example of the weird behavior that infests discussions across the internet, demanding only their position be the correct one even in the face of normal, good faith explanations for why your position may be not be correct. I know I am being a major hypocrite because I've insulted you in the previous section and in this one as well, but that doesn't matter to me right now.

Hackernews will lose my engagement today, and if I am ever asked why I left, that blame is going to be you. I may not/won't return for a while. At best, I'll be here tomorrow, at worst I will never return. The effect that may have on Hackernews is to be determined, but what is known to whoever at all cares about engagement is they've lost a user.

If I recall correctly, Twitter has explicitly said before that they do rank topics very differently because otherwise Justin Bieber would be trending all day, every day.

Twitter's source code does not ban Justin Bieber content as if it was DMCA violations.

That's not it.

Or maybe they try to avoid the kind of discussion that more often than not degenerates into low quality/bot content or outright flamewars. On the other end of the spectrum, FB allowed war posts urging violence against Russian.

Thanks but I prefer to not be subjected to constant hate speech and the militarization of any discourse. Try to visit reddit these days, it's become a cesspool of war propaganda and military glorification.

Whatever they did, they quickly removed that piece of code when it was pointed out. Similarly as they removed the piece code that was categorizing tweets posted by Elon or one of both US political parties.

If I had to guess it's because it gets poor engagement and would crowd out other topics due to the popularity of the topic.

If that was remotely true then why wouldn't the topic be handled at the training level? Instead, Twitter is censoring references to Russia's invasion of Ukraine through the same mechanism used to kill DMCA violations. This is not an approach motivated by "engagement".

Training is a waste of time when you know exactly what you want to block.

HN did the same thing with Bitcoin when it was soaring in value. I remember the front page felt like it was 90% Bitcoin. I was grateful that HN added an exception, bringing the discussion back into balance.

However, I am unaware of a public position HN has taken in relation to "free speech absolutism" or denounced the moderation of content as "proof that the woke mind virus has infected peoples brains"

That HN artcile is "[flagged]". Reading the comments I think it's because there is no analysis of what "UkraineCrisisTopic" actually means or does? Author seems to just grep code base for "Ukraine" then draw what conclusion fits narrative.

Versus the push that we should draw the conclusion that it's a nothing burger? It definitely highlights that Twitter is happy to categorize and treat Ukrainian content differently which is insightful to know.

> Versus the push that we should draw the conclusion that it's a nothing burger?

What push? Conclude what you want, preferably based off available evidence.

> It definitely highlights that Twitter is happy to categorize and treat Ukrainian content differently which is insightful to know.

Not really. That particular analysis AFAICT, highlights the term UkraineCrisisTopic occurs in the twitter algo code base somewhere.

It's not trying to detect all Ukrainian content, it's in a section of the codebase related to manual flag penalties, so it's probably just one category of public misinformation warnings that twitter can apply to accounts promoting misleading or propagandistic information. see e.g. https://www.reuters.com/technology/twitter-tackle-ukraine-co...

Why is it one or the other? I'd argue we shouldn't conclude anything until we know for something a fact.

This is quite nuanced.

Damned if you do: people will question your political stance.

Damned if you don't: you'll lose a lot of users who uses twitter for various reasons other than politics and warfare. People that just want to relax and unwind, be entertained, and plug into the matrix.

Seriously though I think a better UI for muting topics would be a better user interface than hardcoding these things. In twitter you can mute topics but you gotta dig through the settings first.

I think if you don't (boost or demote) is generally perceived better than if you actively interfere.

They could simply provide option to block tweets with specific hashtags (or maybe they already do, I don't use twitter), and I think everyone would like that.

yes they allow muting tweets with certain words that you provide, but it is burried deep in settings.

Tweets with hashtags and links do worse.

It would seem anti-woke tweets do very well. I see such tweets a lot when logged out.

Replying to an account with an unverified account is automatically collapsed. Only twitter blue accounts get to post replies in comments and not have those comments be collapsed. They can also post replies to their own tweets without the replies being collapsed.

>It would seem anti-woke tweets do very well. I see such tweets a lot when logged out.

I have no idea if this is true. However... I have a coworker that was just banned from Reddit, her daughter is a high school swimmer, and there is a trans issue going on. I saw the comment she was banned for, while I don't necessarily agree with all of her points, it was absolutely not ban-worthy. She is waiting on an appeal that won't come. I've now seen with my own eyes that moderators and at least some admins at Reddit are shutting down anything that takes "the wrong position" on trans topics as "hate", when it's definitely not. Let's say over-woke to promote a consensus.

So, my question to you is this... How could we tell if anti-woke tweets do well on Twitter, or if they are overly censored and penalized on other sites?

How can you be sure it's not your reference and the systems you are comparing to are fair or not?

There’s a lot of anti-trans stuff on Reddit. Here’s one with 800+ comments.

https://www.reddit.com/r/Conservative/comments/1241rxj/a_tra...

I’d have to see your coworker’s post to really know if it was hate speech.

All I can do is tell what she wrote wasn’t hateful by any stretch.

But, did you post the right link? I read a post that said a trans person shot up a school which happened and says “Reddit pushes trans propoganda”, is that hate? The top comment says they didn’t use pronouns. Was the hate in the rest of the comments?

> I’d have to see your coworker’s post to really know if it was hate speech.

You sound like the guy from the shoe factory: https://www.youtube.com/watch?v=idb_qsAAe1c

> Tweets with hashtags and links do worse

I don't understand the rationale... My feeds are mostly comments with a link to the article, and following a hashtag is the way to form a community. What does Twitter want to be ? Some glorified public chatroom ?

Why are there no social media platforms that give the users more power over what they want the system to show them?

For example, this suggestion I posted on the Twitter algo

https://github.com/twitter/the-algorithm/issues/1363

Because they generate less money. People don’t get hooked to content they did not know about.

Mastodon??

Because, and I know it's a tired saying, but you aren't the user you're the product.

Advertisers, the actual users, get pretty precise tooling on who to show their ads to.

I really dislike that replying or commenting on things is seen as a positive indicator. This comment would be seen as positive engagement, I think the "algorithm" shown is fairly useless.

Especially since the weights are tied entirely to how well calibrated the individual predictive models are.

The article has a side-note saying:

> “two models trained on the same data but with different algorithms will behave much more similarly than two models trained on different data with the same algorithm. That’s because the training data contains the patterns in user behavior that algorithms mine and use when recommending new content.”

Do we know how ‘true’ this statement is? Or how much % an algorithm must deviate to cause a % change in results? What about difference in data? If someone can point to somewhere to read up on this, I'd like to know!

Yes, it is 'true'. The topic you want is fundamentals of machine learning.

Thanks, I'll check it out

Given what has been open sourced so far, it makes sense that content that is likely to be controversial, or content that generates neutral to negative engagement would have a smaller probability of being displayed.

I suspected this and told my far-right/left wing acquaintances, that no, Twitter (and Facebook too) isn't suppressing you, your content is just a net-negative from the platform's perspective. The platform is in the business of keeping the bulk of its users and advertisers happy.

When people talk about suppression do they mean that their own tweets are suppressed, or the tweets of people they follow are? Or tweets from news organizations are, depending on their content?

The evidence is abundant for active suppression of certain political views beyond just predicted engagement. No clue about Twitter's algo, but Facebook certainly does this.

Can you, uh, share that abundant evidence?

Clear-cut example #1, which you might consider "flipping switches" more than part of the algo is the suppression of posting about Hunter Biden/laptop on Twitter.

Instagram also downweights posts about Biden passing 1994 crime bill. [0]

[0]: https://twitter.com/perma___ben/status/1339293381625864195

Do you have something substantive? IE, evidence that there is a systemic, large-scale policy being applied? Anyway, it looks like they outsource their fact checking and applied a fact check. The fact check is extensively documented here: https://www.usatoday.com/story/news/factcheck/2020/07/03/fac...

I'm a bit tired of the "aggrieved people claiming censorship" over small potatoes that often turns out to be "the company applied their policy uniformly"

The posts with the biggest reach are consistently from right-wing sources on Facebook (or were when they released this data). Is the suppression just highly targeted?

I’d love to hear what former Twitter Timeline engineers can say about this.

I think it shows us that I got the most thumbs-down for suggesting we rewrite everything in Python lol.

it tells us, musk bought twitter for reasons other than money...

Reply on news.ycombinator.com