Very cool, I came several years following and we too had a grid for similar tuning work on LHC. My fondest memories were the bizarre hangups that could occur in custom stuff like this, and the way you'd find out that you done fucked up. My favorite time was when I broke a database by sending an accidental "query all the stuff", but it was really stupid, like from the console all you had to do was type a partial "ls" and it would assume a wildcard. So when the authority in my group attached me in the email and said I broke the main monitoring database I told them they had stupid defaults and never heard anything again. My second favorite memory was when I found a way to gain production access to grid resources just by naming your output datasets in a particular way. I was sat down in a hush ceremony with the Pakistani post-doc from my group and the Russian? programmer of the system and asked to keep it quiet. I kept it quiet, and am pretty sure I failed some sort of spy test that day. It wouldn't be the last time I felt like I failed a spy test, I guess that is my third fondest memory from my time in grad school.