We handle that with secret scanning - code scanning focusses on static analysis of your code to find vulnerabilities in your code, rather than committed secrets.
We have a partnership with AWS (and many other token issuers) that handles this really nicely. If anything that looks like an AWS credential is committed to a public repo we send it over to AWS - if it's a real token they notify the token's owner (and in some cases automatically revoke the key).
There's full details at https://help.github.com/en/github/administering-a-repository....