My adventure in designing API keys
vjay15.github.io
vjay15.github.io
If you want aspects of the token to be inspectable by intermediaries, then you want json web tokens or a similar technology. You do not want to conflate these ideas. JWTs would solve the stated database concern. All you need to store in a JWT scheme are the private/public keys. Explicit tracking of the session is not required.
I suppose it’s there to avoid round-trip to the DB. Most of us just need to host the DB on the same machine instead, but given sharding is involved, I assume the product is big enough this is undesirable.
If you use a cryptographically secure hashing algorithm, mix in a secret salt and use a long enough checksum, attackers would find it nearly impossible to synthesise a correct checksum.
Others have pointed out that the checksum is for offline secret scanning, which makes a lot more sense to me than ddos mitigation.
But it's trivial to make a secret checksum. Just take the key, concatenate it with a secret 256-bit key that only the servers know and hash it with sha256. External users might know the length of the checksum and that it was generated with sha256. But if they don't know the 256-bit key, then it's impossible for them to generate it short of running a brute force attack against your servers.
But it does make the checksum pretty useless for other usecases, as nobody can verify the checksum without the secret.
Experience tells otherwise
That assumption is false. The article states that the DB is hit either way.
From the article:
> The reason behind having a checksum is that it allows you to verify first whether this API key is even valid before hitting the DB,
This is absurdly redundant. Caching DB calls is cheaper and simpler to implement.
If this was a local validation check, where API key signature would be checked with a secret to avoid a DB roundtrip then that could see the value in it. But that's already well in the territory of an access token, which then would be enough to reject the whole idea.
If I saw a proposal like that in my org I would reject it on the grounds of being technically unsound.
I just was confused regarding the JWT approach, since from the research I did I saw that it's supposed to be a unique string and thats it!
Or are you suggesting that the API requests are signed with a private key stored in an HSM, and the JWT certifies the public key? Is that common?
Very. The thing that certifies the public key is called a JWK.
https://datatracker.ietf.org/doc/html/rfc7517
This is typically hosted at a special URL that enables seamless key rotation and discovery.
https://auth0.com/docs/secure/tokens/json-web-tokens/json-we...
Here is a detailed write-up on how to implement production API keys: https://kerkour.com/api-keys
1. Why do you use the API key ID AND the organization ID, and not just one of them, to prevent the confused deputy problem?
2. Why is not necessary to use something like Argon2id for hashing? You say "our secret is already cryptograhically-secure", but what does this mean exactly? Is it due to the fact that the secret is already very high entropy and cracking it, even if we use much faster hash functions like the ones mentioned in your article, it would practically not be possible even PQ with highly parallelized hardware?
Anyways, very interesting read, thank you!
That lets clients detect leaks, but malicious clients cant generate lots of valid-looking keys to spam your API endpoint and generate database load for just looking up API keys.
Even the random hex with checksum component seems overkill to me, either the API key is correct or it isn't.
Reading “hex” pointing to a clearly base62-ish string was a bit interesting :-)
Also, could we shard based on a short hash of account_id, and store the same hash in the token? This way we can lose the whole api_key → account_id lookup table in the metashard altogether.
But when I mentioned it to my senior he wanted me to default with the random string approach :)
Even a million rounds of hashing only adds 20 bits of security. No need if your secret is already 128 bits.
PS : I too am working on a APIs.Take a look here : https://voiden.md/
Well I would have done that and saved half the blog post.
Plain old API keys are straightforward to implement. Create a long random string and save it in the DB. When someone connects to the API, check if the API key is in your DB and use that to authenticate them. That's it.
But otherwise, yes, for love of everything holy - keep it simple.
This is pretty much just plain-old-api-keys, at least as far as the auth mechanism is concerned.
The prefix slug and the checksum are just there so your vulnerability scanner can find and revoke all the keys folks accidentally commit to github.
An attacker will be able to identify valid keys, but won't be able to sign them.
You can either split the values like aws or join them with a separator.
Good idea with the slug though, makes it easier to report leaked tokens to the issuer.