All metrics will be gamed at some point. I don't know exactly how you could even fight this.
- Only accounts that have a decent amount of activity (pushing code, commenting, etc)
- Has set up SSH
- Older than 2 years
- Account active consistently for at least a year
- Must have 2-factor enabled
- Filled out profile
etc
And while some boy accounts may have them, I doubt many have most.
Also, you argue on semantics but the general idea of setting up a legitimacy test that factors in various things is very easily doable, the factors can be kept private, and you definitely can find ones that are generally hard to game.
Then you have people complaining about being "shadowbanned" (because there's no recourse if you're a person and the algorithm thinks you're not active enough), or that github is being anti-privacy (by requiring phone number). It's hard to win here.
It's not a matter of "here is a list of requirements that no one knows about, and here is slight randomness/delay to obfuscate".
How much do you think it takes to pay an actual human from a poor country to come to work each day at 8am, create one github account after another, enter them in a database, and leave at 5pm?
If you want to "study" how github handles stars because there is legitimate financial incentive for you in it, for $100 a day you can pay 10 or 20 of those people to create few thousands accounts a day. Do it few times a month, and throw these accounts in an automated system that creates random repos, pushes a few commits here and there, etc. Also "introduce some slight randomness or delay to obfuscate these events". Do some A/B testing to figure how the 300k accounts under your control affect a repo star system, then advertise a "GitHub stars service" "$0.50 per guaranteed star on Github". Your average VC funded startup could get 10k stars for $5k.They probably give AWS 10 times that a month.
Once github changes their requirements, do more testing, figure out what the requirements now are, then you're back in the game. If people do it all the time to Spotify, YouTube, Google, Amazon, Reddit, and Twitter, why do you think GitHub would somehow crack that nut?
people do it all the time to Spotify, YouTube, Google, Amazon, Reddit, and Twitter, why do you think GitHub would somehow crack that nut?
Because the listed projects do basically nothing, a bare minimum. They don’t even care as long as bots don’t play against their direct interest. Who cares at a media company, or a sales company, who exactly is at their top, as long as they are both not bad enough? Profits come either way. They all are shittiest examples of it who created, incorporated and are themselves part of this problem.
It’s akin to immune system. Its goal is not to protect you from every hiv and cancer, but to avoid constant infections from stupid low-effort attacks. You don’t have to make it prefect, but it must be there. The more cryptic it is, the less welcoming it is to game it through basic means, the better.
Well-connected people will get the tip off. And your PR team will have to keep batting down conspiracy theories, since if there's one thing the nutters love it's black boxes.
In GitHub organization settings you can require to only use secure 2FA which kicks anyone who use SMS 2FA out.
How might a 'connection' look on GH? Will people freely connect, or will they appraise requests more closely?
But of course this is too complex and “no one will use it” (tm). So we’ll better have a screwed up recommendation system that doesn’t work at all, cause that’s simpler!
There are a handful of people that I know IRL that I follow on GitHub. And a few hundred that I follow in total. Out of the handful of people I know IRL, and who I follow on GitHub, only two or three of them are active there any given week. All of the other people I follow I have very little idea who they are. Usually I follow people I don’t know if I come across their profile and either the profile itself or their projects make me follow them. But I star way more different repos than the number of people I click follow on.
For me, the main way of discovering new repos are:
- Frontpage of HN, and comments in posts on HN.
- Specific search results on Google when I have searched for libraries or programs that do specific things.
- Libraries on crates.io that I think might be interesting to look into in the future.
Maybe once or twice a month I happen to click on the main page of GitHub itself and see mentions of repos that have been committed to or starred or created by people I follow.
So for me I don’t think “friends of friends” is a particularly great signal for things to look at. Most of the people I follow, I don’t know much about them.
Likewise, for anyone that follows me it’s not necessarily any strong signal that I follow someone else in order to determine if activity from that someone else should be shown or weighted as more significant to my follower just because I happen to follow that other person.
If you do want a strong signal for who to boost for my followers based on my own activity, go and look at the dependencies that I am using in my own projects. That’s a pretty good indicator that I put some amount of effort and interest into looking at something. This could be done by GitHub itself, parsing the Cargo.toml files of my projects and extracting the dependencies section and looking up which of those dependencies are hosted on GitHub.
The wrinkle is that measures that don't easily quantify are more resistant. For example, showing provable use by other reputable or trusted projects, or a significant amount of resources allocated to maintenance, or ...
Really just anything that can't be reduced to a single number in a canonical way will in the long run prove far more useful for longer.
This of course shifts some of the burden onto potential users to assess things more critically, and forecloses direct numerical comparison. But the idea that you could just look at a number and make such comparisons was faulty from the get go.
Set any law you want, our nature will push us to circumvent it even legally.
Only if we let that minority keep manipulating the system without consequences, it becomes the driving market force that the rest of the population also feels they have to comply to, to go along, as it already has happened in finance, academia, etc.
For varying and self-serving definitions of fair. (Almost everyone in the rich world is in an unfairly-advantaged minority.)
Actually one of the keys is repeated contact. People who have to interact again and again will try and game the system less. Not sure how to build that into a star system but why give up so easily? Do programmers give up when you say "this algorithm can't be made any faster?"
The other is hierarchy. You can't automate reputation scoring.
It doesn't matter if people have to interact frequently if there is no real consequences to that interaction. The punishment in those collectivist cultures involves social shunning, shaming, etc. Individualistic cultures almost pride themselves on how much they can disregard social shunning and shaming. Shameless people are celebrities and elected officials. They are admired as opposed to shunned and ignored. A bad actor in an Amish community is expelled and loses access to what that community offers. That would be illegal in the general society unless their "bad act" was actually illegal. Discriminating against someone for being a dickhead who exploits loopholes and unregulated corner cases (without explicitly breaking the law) would be illegal in many contexts.
> Not sure how to build that into a star system but why give up so easily? Do programmers give up when you say "this algorithm can't be made any faster?"
I don't think people have given up. Online fraud detection is a massive industry as is. Spotify plays, YouTube views, Google search, Amazon reviews, reddit upvotes, twitter's retweets, facebook likes/shares, etc all fall exactly into the same bucket. There is even a significant dollar amount attached to many of those more so that GitHub stars. All are frequently gamed/faked and it's a battle between the platforms and the adversary
Large "Collectivist" communities have body count in the hundreds of millions.
https://en.wikipedia.org/wiki/High-trust_and_low-trust_socie...
For example, Tokyo has a lot of people and they actively dislike interacting with strangers but if you leave your laptop unattended while peeing at a coffee shop, it's very unlikely to have been stolen.
But people who have chops for that probably have high enough paying jobs not to care. As most likely no one would pay for reviews of libraries.
Like, imagine a group of professionals of decent sized, all specializing in a similar field, and having lots of strong connections between each other where they have ample opportunities to share information. It would be hard for an outsider to come in and astroturf their product without immense effort (like hiring shills to attend conferences). In-person networks also obviously solve the problem stars as reputation: reputation spreads naturally in these sorts of networks.
I think the problem comes with algorithmic scale. Maybe a solution would be to have more community building activities (maybe preferably offline).
I believe the only thing anyone can do is take metrics of how the metrics are gamed, as this particular paper has done.
Yeah, people would love that for sure.
> showing _regional_ stars like Apple/Google would be a start.
What does that mean? I thought regions only impact ranking not the net amount of stars (assuming we're talking about Apple/Google Maps). Which as far as I know, github doesn't do ranking.
> What does that mean? I thought regions only impact ranking not the net amount of stars (assuming we're talking about Apple/Google Maps). Which as far as I know, github doesn't do ranking.
At least on IOS reviews and ratings are by country, I dont actually know about google play though. (I dont have an android to check since I am not poor)
Sir, this is an HN.