CLI tool that finds secrets accidentally committed to a Git repo (2017) | Hacker News Reader

CLI tool that finds secrets accidentally committed to a Git repo (2017)

github.com

103 pointskiyanwang31 comments

I read that as "CLI tool ... accidentally committed to a Git repo," as in the CLI tool itself committed to a Git repo by accident, and that the CLI tool finds secrets.

Title as of my reading was "CLI tool that finds secrets accidentally committed to a Git repo (2017)".

I read it the same way

I made a similar tool https://github.com/zricethezav/gitleaks Reasons why you might want to use gitleaks over this:

* performance (powered by go-git)

* scan github orgs/users

* ref targeting

I'd like to have some way to prevent certain code from being committed in the first place.

E.g. when I have a line saying

    printf("debug: now at: %d\n", i); /* DONTCOMMIT */

then when I type "git commit ...", a script will be invoked that will recognize the "DONTCOMMIT" string, and it will abort the commit.

Should be possible if you create a pre-commit hook, like this one https://gist.github.com/guilherme/9604324

Hmm, it uses "git diff" but in my case, this shows a few lines of context. So if the context contains "console.log" (and not the change itself), the script would still complain.

change it to look for DONTCOMMIT instead

You could use 'git diff -U0' which shows no lines of context. Or you could just search for "DONTCOMMIT" (which is what your example used).

Literally what client side pre-commit and on server end post-commit scripts are for

https://appventure.me/2016/04/04/prevent-accidental-test-cod...

Nitpick: this line

   if test $(git diff --cached -z $against | grep $marker | wc -c) != 0

may be changed to

   if git diff --cached -z $against | grep -q $marker

As mentioned, should be doable with local or remote hooks.

Additionally, GitLab has push rules[0] that help preventing pushing secrets (based on filenames).

[0]: https://gitlab.adhoc-gti.com/help/push_rules/push_rules#prev...

https://github.com/awslabs/git-secrets

TIL the UK Home Office has over 700 projects on GitHub.

Is there a similar tool that can run as a pre-commit hook and prevent it from happening in the first place / after this scan?

git-secrets from the AWS team: https://github.com/awslabs/git-secrets

I don’t know of a tool in particular, but it’s not too hard to scan for patterns and high entropy, blocking the commit

I wrote one using the same difference library a few months ago: https://github.com/burdzwastaken/git-rid-of-keys.

Look at pre-commit.com, this had lots of supported hooks including preventing secrets being added

Looks nifty, but I'd ask: why use this over Trufflehog?

https://github.com/dxa4481/truffleHog

As well as the CLI tool the repository contains code for a server to listen for pushes to repositories in github via webhooks and write any violations found into elasticsearch.

Maybe I'm missing something, but doesn't a whitelist (the `.secignore` mentioned in the README) defeat much of the point of a secret-scanning tool? If I whitelist a file now for containing the (benign) string "password," I'm likely to miss a future problem in that file.

There are backdoor for SMTP credentials in django projects: https://searchcode.com/?q=EMAIL_HOST_PASSWORD%3D&p=1&loc=0&l...

Now if only there was a CLI tool to convince my coworkers that storing secrets in git is a bad idea.

I'm guessing such people have little interest in new CLI tools, since CLI is for old tools, etc.

And how to really remove a file from your git history....

https://help.github.com/articles/removing-sensitive-data-fro...

> And how to really remove a file from your git history....

That's misleading. Once you commit and push it to a remote, you should consider it compromised. You could remove it from the repo and force-push to re-write history, but anyone could have pulled from the same remote since your original push and have the file locally on their system.

A better option is to push a new commit to remove the file from the repo (add to .gitignore), then set a new, strong passphrase/whatever and don't ever use the one you pushed again.

How is the article misleading? It said the same thing you did in bold....

Warning: Once you have pushed a commit to GitHub, you should consider any data it contains to be compromised. If you committed a password, change it! If you committed a key, generate a new one.

Your comment was misleading, not the article, which is why I quoted you comment and not the article...

Fair Enough....

This reminds me of gitguardian.

Reply on news.ycombinator.com