Sounds like a better idea to just change the secret.
Edit: I had similar objections to "why not rework databases as an immutable diff history?" https://news.ycombinator.com/item?id=13581096
Removing the file, or the password and adding a comment, as well as changing the password where it's used is much less likely to end up with a re-added password later.
Of course, removing the file, adding it to .gitignore and changing the password makes it even harder as a contributor would have to work to add the password back, which is even less likely to happen.