Hacker News Reader

Comment by matan-h | Hacker News Reader

Gitleaks regexes are fairly accurate. For example, the regex to find a GitHub PAT is "ghp_[0-9a-zA-Z]{36}" which mean it has a specific number (36+4) of characters from specific group (alphabet+number). And I try to filter out the obvious non-secrets (like 'abcd','xxxx' and '1234'). However, as I stated in the article, most of the data is not actionable: most people just revoke the token, use an old one, change some random letters, etc.

View on HN

I think the real take-away is that StackOverflow does not have key detection like GitHub does.

Has GitHub open sourced their key detection?

No, Microsoft is keeping all of that stuff under the wraps. They have a "secret scanning partner program" where they allow companies to have a endpoint GitHub can use for figuring out if something is a secret or not, so it's not just a library with a bunch of regex, seems like a service in itself and Microsoft doesn't really open source stuff like that.

You are correct. Though, speaking of regex, they work with partners to create the most accurate regexes possible using non-public information like expected entropy or checksums.

That's surprising (to me), because the enterprise custom scanning feature only supports hyperscan-flavoured regex.

Sorry, I should clarify that some of those things are _in addition_ to regex. You are correct that it uses Hyperscan to find initial matches, then their first-party patterns go through some additional local processing magic.

(This is my understanding based on conversations with people working on the secret scanning feature at GitHub, I don't have firsthand knowledge.)

But... But... I thought Microsoft ♥ open source?

You mean https://docs.github.com/en/enterprise-cloud@latest/code-secu... ?

I see real-looking keys posted to SO at least a couple of times per week (stuff like Twilio and Stripe keys are the most obvious as they’re tagged-strings; followed by GMail SMTP creds; I edit them out and flag the posts for the mods, as one does). Granted, most of the time it’s just some kid who doesn’t appreciate what secrets are worth keeping, or wasn’t paying attention when copying+pasting into their post, but every so-often I see secrets in a post from what looks like an outsourced worker assigned to a “real” business, with very real things to lose - and I get depressed from wondering how modern society even holds itself together given the scale of incompetence I witness first-hand…

(Fun-fact: the next SMS text-message you get from a major chain informing you on an upcoming appointment was likely sent to you via Twilio from a desktop client with a hardcoded AccountSID and AuthSecret strings shared by all 20,000 (multitenant) users; Don’t ask how I know, but it’s depressing; I do report these things (anonymously) to the vendors but then receive a reply from a non-technical manager accusing me of “hacking”. I haven’t yet reported them to e.g. Twilio directly because I don’t want Twilio to revoke their creds and cause potentially hundreds of thousands of people to not-receive essential comms from those tenants. Le sigh…

There was another thread a few years ago where someone suggested reporting to US-CERT or another CERT. It has some advantages like "they know what a credential leak is", "they know that people reporting security issues aren't necessarily malicious", and "they sound official when they try to get it fixed". And "your name will no longer be on the report".

I haven't had occasion to try this myself, but it sounded like good advice!

Reply on news.ycombinator.com