Configuration files probably include database passwords, AWS secrets, API keys, maybe even SSH keys. Fairly clearly:
1. They're really important and change periodically in sync with the rest of the app.
2. They shouldn't be in a public repo under any circumstance, EVER.
3. You probably don't want every single developer having access to your production database passwords.
Point 1 argues strongly in favour of putting them into version control for all the same reasons you'd put anything into version control.
Point 2 and 3 argues for putting them in your .gitignore (or equivalent) and copying files around by hand (or via a fabric script or what have you) during deploys.
Common wisdom seems to be that point 2 and 3 outweighs point 1, and for open source projects which live in public repos it probably should. Ditto for large enterprisey projects. But for a startup, working on a non-open source project, I think point 1 massively outweighs everything else.
As an example, the most common advice for configuration for django apps is to have a settings.py which imports a settings_local.py; you then put settings_local.py into .gitignore, and in every environment you create a new settings_local.py and add the local settings (database connection strings, template paths, whatever) into it. I started out doing it that way too, but I've recently reversed it: Now I have a settings_dev.py, settings_staging.py, etc, each with the specific environment settings, and each of which imports the common settings from settings.py, and all of which are in source control. Now I can checkout any branch or version of my app, and have some confidence that I've got the right settings to run it in whatever environment I want.
(I see philjackson has listed another reason for being careful with config files, but I'm not sure I agree. I'd say he's basically giving an argument for using different config files in different environments, and generally being a bit organised. And if there is any risk of developers fat fingering the config files, isn't that an argument to have them in version control so you can fix it easily?)