GitHub was down | Hacker News Reader

GitHub was down

status.github.com

39 pointspegler26 comments

Guys, please don't forget the advantage of using DVCS to begin with. I like Github too, but if Github goes down it shouldn't put the brakes on your work. Push to multiple remotes. Solved.

A lot of people rely on other features of GitHub for their workflow: issues, pull requests, comments.

I've been telling people for a fair while that I wish Github issues were stored as part of the git repos themselves, not so much for availability reasons as that it'd be awesome if the state of bugs got branched and merged along with everything else, so that a bug could be fixed in a branch, but not yet in master, and would get fixed in master when the branch got merged. But I guess availability would be a nice additional perk.

You're welcome: http://bugseverywhere.org/

Wow, slick. Thanks for sharing.

What I would like to see is an analogue to offlineimap for GitHub's project management services.

The protocol could even be email based.

e.g. a pull request is an email with the git formatted patch attached. Issues are emails with discussions as threaded mail/news, etc.

That sounds a lot like Linux kernel development.

If you use the "fixes #7" syntax in a branch commit, the issues will be closed only when that branch is merged to your default branch (usually master). I realize that this is only part of what you desire, but it's a useful recent addition: https://github.com/blog/1386-closing-issues-via-commit-messa...

Genuine question – who DDoSes Github? From what I've seen, DDoS attacks often seem to be a response to a site or company entering the political sphere in a controversial way.

In previous threads about Github being DDoS'd, I asked the same question.

Apparently it's some scheme to ask ransom from companies/startups that have enough money to pay and/or not enough infrastructure to sustain the attack without giving up.

Yep. There's a well-written article about an early DDoS extortion attempt where the victim fought instead of paying, developing some novel protection technology in the proces: http://www.csoonline.com/article/print/220336

Yea, all of us successful web companies face these all the time. We get them multiple times a day. Most of us end up giving up and pay for Arbor gear and ridiculously over provisioned pipes as well as good relations with our hosting facilities.

Additionally, some group may be targeting a specific project on Github

Yeah, there was a brief major service outage according to the status page.

https://status.github.com/messages

Today

1:45 UTC Major service outage.

2:01 UTC We're still experiencing a large DDoS attack. The site is experiencing major packet loss and is mostly unavailable. We're working to further mitigate the attack.

Unfortunately this occurred at the exact moment I was deploying some changes. Makes me think that being so dependent on GitHub for a production code repo is a little big dangerous.

Redundancy would certainly solve this issue, though.

There's already built-in redundancy. Wherever you pushed to Github from has a complete copy of the current state of your repository. Github is effectively just a publicly accessible mirror, but you can always push to a different remote elsewhere, push directly to your production instance, or even host your repo in an s3 bucket, all while Github is down.

I've been in some development environments where the GitHub master branch is what the build system pulls from in order to deploy to dev, staging and production. With that setup, it's more challenging since the build process probably involves more steps and potentially ssh keys that are only allow the build server to deploy.

Depending on how many repos you need, I would say set up an alternative remote on bitbucket.org.

Right, I get that...I meant a separate remote when I said redundancy. GitHub has just been so reliable I haven't even needed other remotes for the past 8 or so months we've been using it like joeblau mentioned.

Crazy how many people are affected when github goes down for less than 2 minutes.

There goes our plans for a release tonight. This DDoS seems to be a fundamental flaw of popular SaaS services.

It's already back up (for me at least).

I happily created a new Github repo, imported some code, made some changes and pushed stuff, all OK. And just now when I want to post announcements of my new library, bam! Github is down.

Mostly I don't care if they are periodically unvailable, but the timing on this was amusing.

Does GitHub use a CDN? It seems like that might help them with the DDoSing.

CDNs are only useful for things that can actually be cached, and which have been accessed recently. Static content is easy, and I'm sure Github does it, but dynamic content doesn't benefit as much for a CDN.

As another note, anyone doing a DDoS is going to target parts of the system that are more intensive. They'd just skip the cached things and go after the ones that have to be regenerated.

That makes sense, but can't a CDN also help identify the DDoS and block communications from certain servers. I remember seeing a captcha on Adria Richards website after she got those two devs kicked/fired. Maybe GitHub just needs better tools to identify and block the DDoSes quicker.

Reply on news.ycombinator.com