Comment by Hovertruck | Hacker News Reader

Comment by Hovertruck | Hacker News Reader

We received an email from Github yesterday informing us that one of our repositories had been accessed by a third party due to this issue. While it's not a fun notification to receive, it definitely made our general security paranoia feel justified – we're lucky that from the get-go we've held best practices around keeping secrets out of the codebase. Obviously we still dedicated time as a team to prune through our repository history with a fine-toothed comb for anything that could potentially be a vulnerability, as we take this very seriously.

One of our engineers came up with a useful script to grab all unique lines from the history of the repository and sort them according to entropy. This helps to lift any access keys or passwords which may have been committed at any point to the top.

I think this is a great example to illustrate the tough edges of security to less experienced engineers. Github will most likely never let something like this happen to you, but on the off-chance that they do it's great to be prepared. Additionally, the response from Github was very well received. No excuses, just a thorough explanation of what happened.

I also can't help but mention that we're hiring, if you'd like to work at an organization that values security and data privacy very highly. :) usebutton.com/join-us

I'm curious, how did they calculate entropy? My first thought was to do something with Huffman encoding.

I wrote the script in question and actually used a simple shannon entropy value. (http://codereview.stackexchange.com/questions/868/calculatin...). It worked well enough help rule out several problem spaces.

Would you mind posting the script? I'd love to run it against our codebase and see what it comes up with.

It might be a fun thing to open source as part of a "I've inherited a project, what now?" toolkit that helps you decide what to fix.

Sure. It's a simple tool but the concept could be augmented toward something like the scenario you described.

https://gist.github.com/jasonmoo/06691c8fea09b62aa35235fc93e...

IIRC Instagram released a plugin for Bandit (the OpenStack static analyzer for Python) that does this.

I thought the idea was interesting so here is a little PoC in Ruby:

    require 'facets'
    lines = Dir['**/*.rb', '**/*.py', '**/*.cpp'].map { |f| File.read(f).lines.map(&:chars) }.inject(&:+)
    puts lines.sort_by(&:entropy).map(&:join).last(10).reverse

Using: http://www.rubydoc.info/github/rubyworks/facets/Array%3Aentr...

Is it easy to modify this script to run over all lines that have ever existed in the repository history?

For example, could you pipe the output of `git log -p --all` through this and filter out all the commit hashes somehow?

Yup, just use:

    lines = `git log -p --all`.lines.map(&:chars)

So I found that `git grep /.+/ $(git rev-list --all)` is a better way to get the content of all the files: https://gist.github.com/Dorian/e1514535c3c5036cf327ce61eb34a...

But actually an hex number regexp might me far more accurate than the entropy (e.g.: secrets are often long hex numbers).

I tried it and it yields interesting results: https://gist.github.com/28110f0b8105db11e8973d1d0be85259

I got the other end of that email today, saying that an account in my organization had inadvertently downloaded private repos from another customer when fetching from one of our own. Fortunately for GitHub/that user, it was almost certainly our automated provisioning system so we never had any idea and whatever it was never made it anywhere interesting.

The email was kind of funny though, part of it was effectively "if you have this data pretty please delete it without looking at it". I'm sure that's the best they can do, but it still made me chuckle.

Of the very small number of repos affected, there are now two of us reporting that it affected us :). And I had the same approximate response: well, I don't keep secrets in the repository, so not that big of deal. I'd rather the source not get shared with the world, but shit happens and they owned up to it right away. If that source were valuable enough, I'd be hosting it on-prem with encrypted off-site backups.

Reply on news.ycombinator.com