Can anyone throw some light on what google is doing with tens of thousands of machines in these clusters? Are they hosting crawlers on them?
[1] http://www.wired.com/2013/03/google-borg-twitter-mesos/
[2] https://www.youtube.com/watch?v=0ZFMlO98Jkc
- websearch
- google brain
- video transcoding for youtube
- appengine (think snapchat)
- gmail
Almost everything Google does runs on Borg.
So whatever else was going on, protein folding and drug docking and telescope simulation was soaking up whatever leftover cycles there were.