Hacker News Reader

Comment by bombcar | Hacker News Reader

The API keys I’ve used (admittedly not many) all seem to be long random text strings - how does GitHub detect them? By then being used (ie in api code) or do they actually have a known format?

View on HN

GitHub documents the process over at https://docs.github.com/en/developers/overview/secret-scanni.... You specify a regex, and you check if the secret is valid on your end.

There must be an astounding number of false positives for common patterns like N-length string of base64 chars. Could someone upload a malicious file with millions of matching strings and watch Github DDoS a company's verification endpoint?

I imagine the scanning would be rate-limited on per-repo basis.

Yeah, that would be reasonable.

Probably also a max false positive rate; this isn't a guarantee, just a service, so if it detects X false positives it could just exclude the repo entirely as problematic.

"Now you have 2 problems."

This is a difficult problem indeed, but thankfully it is just as difficult for the malicious actors as it is for the "good guys". Since various bad guys have presumably been scanning public repos for years already, Github and PyPa adding this feature is leveling the playing field, even if it is not a 100% accurate search algorithm.

Not sure how these particular scanners do it, but during security assessments you sometimes use tools that will find all strings in an application package with high entropy.

Usually its junk, but occasionally you do get lucky and find tokens.

PyPI API keys have a known format, they start with "pypi-".

Reply on news.ycombinator.com