Hacker News Reader

Comment by mikert89 | Hacker News Reader

alot of distributed systems in big companies grow at this rate, its not exponential, its mundane

When a service that was already the primary git hosting provider for most of the world for 20 years grows at that rate its not mundane and its not comparable to any example.

AWS saw this growth every year for two decades, hyper growth tech sees it all the time

The absolute number doesn’t matter nearly as much as the change in rate of growth. The number of commits had not been doubling every six months at GitHub for a long time.

At Amazon if traffic volume consistently doubled every six months that is actually quite a lot easier to plan for, it just becomes part of everything they do from very early on.

im not buying that github is a unique engineering problem, harder than the rest of hyperscalars.

No one said it is unique. But if you take an infrastructure and engineering org that had been growing at 10% a year for a decade, you are going to have a different set of capabilities and practices in place. Adapting to a new reality of doubling every few months will predictably produce failures anywhere. GitHub is not unique in that regard.

They have what amounts to an unlimited budget for AI spend. If it’s so fantastic, why can’t they let it crawl over every piece of their codebase, every metric, every log, and spot these problems before they occur?

“We misconfigured a sidecar” is something I would think AI could quite easily find and fix.

I don't know who you are responding to. I haven't made any statements about what AI can do for them. I would expect AI is making the situation worse, like it is in many dysfunctional tech organizations.

ive spent 15 years in big tech companies, this problem is common and its why they pay engineers 500k-1M. this happens at meta literally all the time.

half of engineering in big tech is just rewriting a system to scale

microsoft is incompetent, they havent changed windows/excel/outlook in 30 years

Tell me about the time that an app at Meta grew at 10% a year for a decade and then began doubling every few months.

its probably happened on half the core teams, idk what you want me to say, it sounds like you dont have alot of experience

Don't dodge the question and attack the asker, that's rude.

Instagram, Facebook and even threads all had much more mundane growth rates and definitely no unexpected jumps like GitHub is experiencing. I'm sure if suddenly the solar system had 10 more earths with each about 10 billion people and they would all start using Instagram tomorrow we would have exactly the same growing pains and outages that GitHub has today.

Luckily for Meta agents are not yet as much into doomscrolling as humans are.

instagram growth rate != core service team in instagram growth rate

Can you give an example?

> AWS saw this growth every year for two decades

That does not seem to be true - which two-decade period are you talking about? AWS has only been around for ~20 years, and I just reviewed a 10 year period, and not a single one of those years saw doubling in the whole year, let alone doubling in a few months. Which 20 year period are you referring to, and are you referring to doubling every few months over that 20 year period?

AWS saw doubling in usage every month for 240 months? So it now exceeds the number of atoms in the universe?

Eh, I would be more empathic in this situation[0].

Github isn’t small startup, where other 10x threshold is as cheap as buy bigger box in your IaaS.

When you are already biggest player in the ecosystem and you suddenly get 10x persisted traffic, with at least 30x+ forecast “soon” - I am not surprised they have issues.

[0] even at current MS owned github

cloud providers and hyper growth tech deal with these growth rates all the time

Did you read the article? It is growing exponential.

this is common at AWS/cloud providers

Reply on news.ycombinator.com