I assume they're trying to keep ai bots from strip mining the whole place.
Or maybe your IP/browser is questionable.
Or maybe your IP/browser is questionable.
The future is a .txt file of John Carmack pointing out how efficient software used to be, locked behind a repeating WAF captcha, forever.
Could you help me understand what the difference is between your point and the arguments MPAA and RIAA used to ruin the torrent users' lives they concluded were "thieves"?
As a rule of thumb, do you think people who are happy with the services they contribute content to being open access and wish them to remain so should be the ones who are forced to constantly migrate to new services to keep their content free?
When AI can perfectly replicate the browsing behavior of a human being, should Github restrict viewing a git repository to those who have verified blood biometrics or had their eyes scanned by an Orb? If they make that change, will you still place blame on "jackasses"?
Well the main difference is that this is being used to justify blocking and not demanding thousands of dollars.
> When AI can perfectly replicate the browsing behavior of a human being
They're still being jackasses because I'm willing to pay to give free service to X humans but not 20X bots pretending to be humans.
I’m not arguing you shouldn’t be annoyed by these changes, I’m arguing you should be mad at the right people. The scrapers violated the implicit contract of the open internet, and now that’s being made more explicit. GitHub’s not actually a charity, but they’ve been able to provide a free service in exchange for the good will and community that comes along with it driving enough business to cover their costs of providing that service. The scrapers have changed that math, as they did with every other site on the internet in a similar fashion. You can’t loot a store and expect them not to upgrade the locks - as the saying goes, the enemy gets a vote on your strategy, too.
I'd like to focus on your strongest point, which is the cost to the companies. I would love to know what that increase in cost looks like. You can install nginx on a tiny server and serve 10k rps of static content, or like 50 (not 50k) rps of a random web framework that generates the same content. So this increase in cost must be weighed against how efficient the software serving that content is.
If this Github post included a bunch of numbers and details demonstrating how they have reached the end of the line on optimizing their web frontend, they have ran out of things to cache, and the increase in costs is a real cause for concern to the company (not just a quick shave to the bottom line, not a bigger net/compute check written from Github to their owners), I'd throw my hands up with them and start rallying against the (unquestionably inefficient and on the line of hostile) AI agent scrapers causing the increase in traffic.
Because they did not provide that information, I have to assume that Github and Microsoft are doing this out of pure profit motivations and have abandoned any sense of commitment to open access of software. In fact, they have much to gain from building the walls of their garden up as high as they can get away with, and I'm skeptical their increase in costs is very material at all.
I would rather support services that don't camouflage as open and free software proponents one day and victims of a robbery on the next. I still think this question is important and valid: There is tons of software on Github written by users who wish for their work to remain open access. Is that the class of software and people you believe should be shuffled around into smaller and smaller services that haven't yet abandoned the commitments that allowed them to become popular?
I don't think many people were particularly sympathetic to people making money off piracy - by and large, people were upset because people committing piracy for personal use were getting hit with the kinds of fines and legal charges usually reserved for, well, people who make money off piracy.
> Am I wrong in assuming most of this scraping comes from people utilizing AI agents for things like AI-assisted coding?
Yes. The huge increases in traffic aren't from, say, Claude going and querying Github when you ask it to, it's from the scraping to drive the initial training process. Claude and the others know the first thing about code because Github and StackOverflow were part of their training corpus, because the companies which made them scraped the whole damn site and used it as part of their training data for making a ~competing product. That's what Github's reacting to, that's what Reddit reacted to, that's what everyone's been reacting to - it's the scraping of the data for training that's leading to these reactions.
To be clear, because I think this is maybe a core of our disagreement: The problem that's leading to this isn't LLM agents acting on behalf of a user - it's not that Cursor googled python code for you - it's that the various companies training the models are aggressively scraping everything they can get their hands on. It's not one request for one repo on behalf of one user, it's the wholesale scraping of everything on the site by a rival company to make a rival product, most likely in violation of terms of service and certainly in violation of anything that anyone could reasonably assume another corporate entity would stand for. Github's not mad at you, they're mad at OpenAI.
> There is tons of software on Github written by users who wish for their work to remain open access. Is that the class of software and people you believe should be shuffled around into smaller and smaller services that haven't yet abandoned the commitments that allowed them to become popular?
You store your money in a bank. The bank gets robbed repeatedly by an organized group of serial bank robbers, and increases security at the branch. You move your money to another bank, because the increased security annoys you. You understand the problem here may repeat itself elsewhere as well, right?
I do, and how is this to cap off our discussion:
>You move your money to another bank, because the increased security annoys you.
On my way out, I would quote Benjamin Franklin: "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
I go out of my way to help my community, and I expect those I support to do the same. That bank could put more resources into investigating/catching the robbers who are attacking not just the bank but the security and liberty of their community, or it could treat every customer with more suspicion than they did yesterday. I know where the latter option leads, and I won't stand for it.
You're right that it repeats itself elsewhere. Often, I find.
Since Microsoft is struggling to make ends meet, maybe they could throw a captcha or proof of work like Anubis by xe iaso.
They already disabled code search for unauthenticated users. Its totally plausible they will disable code browsing as well.
And seriously if they keep this up, with limits on their web interface but leave unauthenticated cloning allowed, I'd rather clone the repo than log in.
GitHub code browsing went south since microsoft bought them anyway. Having a simple proxy that clones a repo and serves it would solve problems with rate limits and their awful UX.
I'm using Firefox and Brave on Linux from a residential internet provider in Europe and the 429 error triggers consistantly on both browsers. Not sure I would consider my setup questionable considering their target audience.
Personally, I like sourcehut (sr.ht)
(Joke's on Reddit, though, because Reddit content became pretty worthless since they did this, and everything before they did this was already publicly archived)