Hacker News Reader

You can watch a very entertaining talk from one of the Google engineers who went to fix the website at the time: https://www.youtube.com/watch?v=GLQyj-kBRdo

View on HN

The most ridiculous part, they didn't have a way to run distributed commands over ssh. When a filesystem mounted in read-only mode, somebody had to ssh into each of the 175 servers manually and run commands to fix it.

What do you mean by "distributed commands over ssh"?

Say you want to run a command on a group of machines using SSH. Rather than manually running the command on each remote machine you can automate their execution over the group using what is known as distributed commands.

http://www.tecmint.com/using-dsh-distributed-shell-to-run-li...

http://www.ibm.com/developerworks/aix/library/au-satdistadmi...

Reply on news.ycombinator.com