I presume this is not strictly a Rails problem.

You can check in things that shouldn't be checked in with any language/framework.

If you have done this, here's how to fix it: http://help.github.com/remove-sensitive-data/