A Note on the Recent GitHub Outages | Hacker News Reader

A Note on the Recent GitHub Outages

github.com

30 pointsmbrubeck10 comments

As much as it sucks when Github goes down, there's a sense in which I look forward to it happening. It's yet another chance for me to get a peek at their architecture. As a high school student it's pretty disappointing to me how hard it is to find fully laid out architectural plans for large websites. So I appreciate sites like Github and Ravelry being open with that information.

highscalability.com == architecture/infrastructure porn.

Google architecture: http://highscalability.com/google-architecture

Google infrastructure: http://en.wikipedia.org/wiki/Google_platform

Amazon architecture: http://highscalability.com/amazon-architecture

No one will layout the blueprint for you, but most general information regarding infrastructure are out there.

Wow, badass comment. I had no idea, thanks!!

Wow, thanks for bringing highscalability.com to my attention. Awesome breakdown of sites.

Another Yahoo! bot behaving badly? Between them and MSN/Bing's bot, what the hell is going on with search engine spiders lately?

Imagine you have to write a computer program to visit every page of every website on the entire Internet.

Sometimes you are going to make a mistake, and someone on the Internet is going to notice.

Except this has been going on for months, has happened a number of times, and occurs in violation of standards, like robots.txt, that were created to prevent it from happening. I'd expect it from an individual or startup, but not a large company that's been doing spidering for a long time.

From the explanation it looks like github isn't using '5 whys' when things go wrong. Has anyone tried this? Is it overkill or does it work?

http://en.wikipedia.org/wiki/5_Whys

If I were GitHub I think I would have mentioned the cause of outage number 3 after I put the rate limiting fixes in place, which is not the impression I got from the article. It is not hard to write a basic HTTP request spoofer and it sounds like they're still vulnerable.

Reply on news.ycombinator.com