Hacker News Reader

Comment by s_dev | Hacker News Reader

I recall reading years ago that cloud services were expected to run with a reliability of 3 or 4 '9's and that if they didn't competing services would quickly overtake them in adoption. The industry was supposed to be that cut throat.

Has big tech reached a similar status like banks in that they are "too big to fail" i.e. when they do fail we all just look the other way and say: "well everyone else is out too". Didn't someone recently calculate that GitHub is running at 95%? For comparison the Irish Rail service which is not reliable has 80% of it's trains run on time.

This seems absurd and really challenges a lot of ideas I had about big tech and cloud infrastructure. GitHub seems to have remained the dominant player relative to GitLab etc.

View on HN

People overestimate how much they care about stuff. Moving off GitHub would be more costly for us than having 5% downtime. Obviously there’s a tipping point, but it shows people are tolerant given the price tags.

For a reasonably common class of deployment and organization (E.g. an online service where the cost of a short outage is significant) it becomes a pain point. For instance if there's a sudden security flap and you need to re-spin your system and redeploy, but that process is gated on GitHub working, now you're screwed if that security flap happens when GitHub is in its 5% down time. Basically you've coupled your uptime to GitHub's uptime in some measure.

I agree with you in spirit, as we have this problem from time to time as well, but number of businesses migrating off-GH shows this isn't the trigger point yet.

More specifically you've coupled the likelihood of your downtime coinciding with githubs downtime. Which as its a product is likely a very small % chance.

If I'm running, say, an e-commerce business, an hour of downtime on my serving-path might cost me $100k of revenue. An hour of downtime for my development stack might cost me 30 person-hours = a few grand. As a result, I'm much more willing to have unreliable github than unreliable CDN/Compute/etc.

https://vimeo.com/355556831

We don't care. We don't have to. We're the phone company.

That canot happen if the market becomes monopolistic, with every bigtech out there buying every startup worth a penny. The erosion of anti trust in the US created this monstrosity.

A lot of it is inertia I think. I hear people saying that they "just use so-and-so for version control" but really we all have tons of CI/CD build and test pipelines, config, business processes etc in Github (rightly or wrongly).

That number of 9s might be a thing for the lower layer cloud services like on AWS or Azure or GCP. Even then, that us-east-1 outage was an accidental power move. It established that many are best off putting all their stuff on us-east-1 cause then their outages only happen at the best possible times, whereas multi-region complexity might cause its own kind of outage when nobody else is down.

Except you cannot run a business in Asia with datacenters in the US. The latency is unacceptable.

Competing services can't afford the data storage to compete with Big tech

This is the important part, we've moved away from monopolies on software to creating a vertical stack that allows a few companies to corner all cloud hardware, which let's them prevent anyone from competing on both price AND scale.

I think AI is going to change this. Once you've built around github you feel locked in, since moving all your CI and other actions out of there, and your workflows out of there, is labor intensive. Now you could migrate from github to another service in a week, maybe less depending on how many workflows you need to move.

I feel you are correct. Throwing AI at this problem works reasonably well. “Rewrite this Actions workflow to run on $PROVIDER” is a sensible task.

It takes a while by hand, but I recently saw a migration of old Jenkins projects to Actions with AI (yea), and it was smooth.

Github is a social network, not a cloud service.

(only 5% /s)

Reply on news.ycombinator.com