In case anyone else made this mistake, Github's help section has an article describing the steps needed to remove sensitive data from a git repo:

http://help.github.com/removing-sensitive-data/