How removing shell calls sped up GitHub
github.com
github.com
When I am hacking something together and can't find a library or don't have time to write one to do something that UNIX does really well already, I use the shell too.
When performance becomes an issue, you rip it out and do it better.
I appreciate the visibility into the service issues and the related fix. That was a good move on GitHub's part.
GitHub is a stellar example of how to start a software company.
So, yes, a hypothetical reimplentation of git in ruby as a C FFI interface would probably be faster than ruby reading the .git files itself; however exactly as demostrated in the blog entry shelling out to perl/sh code shelling out to C is much slower than the ruby program reading the .git files itself.
I'm surprised that Github was sloppy enough to be dropping to the shell to get things done.
I don't know Ruby, but wouldn't the better solution be some variant of C's exec call that executes the instruction directly without interpreting the shell string? This would reduce some parsing overhead and still allow the site to use the official version of Git.
The real solution to calling C code from within Ruby is to call into the C code, either using DL/Win32API (tricky but doable) or by writing Ruby bindings.
There was a Google Summer of Code project that tried to address this 'libification' problem, but it did not get very far - at least, not far enough to do what something like GitHub needs to do.
http://code.google.com/soc/2007/git/appinfo.html?csaid=5DB94...
Actually, the Ruby implementation for most of these operations is pretty fast (though not as fast as C) and the object specs are incredibly slow changing. The commands change around a lot, but the objects themselves are pretty simple and have not changed much in years. Doing this is far less prone to git changing than depending on command line (or possibly even binding) calls.