AWS SIGv4 and SIGv4A – How AWS signs and verifies API requests
shufflesharding.com
shufflesharding.com
I ended up implementing an additional auth API endpoint where you use your long-lived secret to periodically request a short-lived JWT that is only valid for a few hours, this short-lived secret is what is used for the actual APIs to authenticate the users.
https://datatracker.ietf.org/doc/draft-ietf-httpbis-message-...
Library support for it is spotty, depending on your framework of choice but they exists.
Your second approach is covered by oAuth refreshable tokens I believe.
I went through the same process as you in an API project, eventually ended up using Keycloak as the auth server.
As for the second approach, yes exactly it's very similar to OAuth2. In fact I guess this is the method used by GCP APIs by service accounts. The clients use their long-lived secrets to get an oauth2 access token, this access token is JWT I guess that also contains authorization information such as scopes.
But after much googling it appears not! Seems like a fairly simple additional variables aws could pass through and condition to support, so you can say “only allow this request if the signing time is < 1min ago”, closest you can get is that the policy is only valid after X date! Is a shame!
[0]: https://github.com/psanford/awsv4signer/tree/main/examples/t...
However, that secret is no longer used for the signing. Instead it's combined with a date and service-description to generate a public-private keypair, which is itself not secret (both AWS and you can derive this if you know the algorithm). Your requests are now signed with this private key (client side) and can be verified by the services with the public key (server side).
AWS still knows your secret, but it keeps it in a more secured location verses out on the farms. The farms would need a way to request your public key for the day from the secured location, so they can verify the sender. But, the farms no longer have access to your secret key, so cannot sign other requests masquerading as you.
Once the key is loaded into the TPM as an HMAC key, you can no longer get the plaintext key back out. You have to use the TPM to perform any HMAC operations using the key.
It's not as good as having something that supports a hardware token, of course, but it's better than the default awscli suggestion to keep the secrets around in plaintext either on disk or in env vars.
Singular it does, per IAM user.
I'm not claiming it's ideal, but it works, with one Yubikey per IAM user:
https://aws.amazon.com/premiumsupport/knowledge-center/authe...
Thanks, but you cannot[0] use hardware keys on the terminal.
We haven’t released the code yet but are in the process. If you think this could work for you or you’d just like to see how we did it DM me on Twitter @timmattison and I’ll give you the code ASAP.
The TPM is a good option because every non-Mac ships with one already (and there are similar facilities available on macs).
[0]: https://github.com/tpm2-software/tpm2-tools/issues/1597#issu...
On the auth side, the major change since then is that you can use the IoT credentials provider to provide certificate based auth to all services (https://docs.aws.amazon.com/iot/latest/developerguide/author...). You don’t need to be using any of the other IoT services. It was created to make it easier for devices to use AWS services but can be used by anyone/anything.
What we did was combine the AWS CLI feature to source credentials from an external process (https://docs.aws.amazon.com/cli/latest/userguide/cli-configu...) with a script to do the certificate based auth. This allows you to obtain STS credentials using a certificate and pass them to the CLI (access key, secret key, session token). Your secure hardware just needs to do the normal work of assisting in the mutual TLS auth which in our case was done with curl and Zymbit’s OpenSSL engine. We are releasing that code along with a SoftHSM2 setup so people can see how it works in a test environment.
At least on MacOS it uses keychain. There are other storage backends for other platforms.
With the secret stored in a tpm it cannot be extracted. Instead you ask the tpm execute the hmac() function on your behalf with the secret only it can read.
Seems like a solid protocol (though a lot to implement in a client, that can be encapsulated in an SDK)
One downside is that it would make widespread web service API abuse even easier than it already is. One would need to understand even less about the mechanics and workings of same-scheme services across companies. Yes, this is security through obscurity, but why leave more surface area open than necessary?
Malicious actors will certainly appreciate it!
Well, there's some things to consider.
First, Amazon AWS is a very rich target. It hosts a very large quantity of services that would be ripe for exploitation. I'm sure attackers would love to find holes in the AWS mechanism.
Second, because of number 1, I'm sure the AWS services are not only a prime target for attack, but also actively being attacked. By both nefarious black hat ne'er do wells, to state level agencies. I would fully expect US, Russian, Chinese and other state intelligence agencies to be very interested in an exploit of something as ripe as the AWS system.
Third, Amazon has the motivation, due to number 1, to keep on top of and ensure that its technique is sound and robust. Not only that, it has the resources to do it.
Four, the technique is open, and documented, and available to all. No skullduggery is required at an algorithm level to analyze it and understand it. Thesis seeking white hat PhD students can have at it and advance the field.
So, if there were to be "one algorithm", Amazon and AWS have the experience and know how to hold theirs up high on a pedestal labeled "You could do a lot worse".
I also wonder whether those subkeys are copied over to the frontend server (that might be a lot of data to copy) every day or requested and cached (possibly high latency for first call)
> I’m Colm MacCárthaigh, a VP and Distinguished Engineer at Amazon Web Services, and this website is my blog.
Any further translation on what VP and "Distinguished Engineer" means here? It sounds like both of them are honorary and doesn't really mean much, but then "Distinguished" makes it seem they received some sort of Nobal price of computing as well.
And if neither of them really mean anything, then this guy is really "just" a developer right?
Probably yeah, he went to a coding boodcamp and was hired after that.
This is a fairly fluffy piece of writing. His internal writings were much deeper.
Comp is $950K++++ per year range.
It’s hard to get familiar with engineers at Amazon. They have spokes persons who release things written by others and their external contribution process is difficult at best.
It isn’t a surprise people think he is a nobody. I still remember him after five years.
The further/higher you get you usually do less "programming" and more design/architectural, plus line management.
Sign the bytes, not the meaning.
Canonicalization is treacherous and generalist developers shouldn't freelance their own formats. Canonicalization problems are one of the reasons we're at SIGV4 and not just, like, SIG. But Amazon has at this point thought about this as carefully as any organization in the Internet.
Which is why it's good advice to, if you're going for a signed-request API, just shoplift SIGV4 from them.
I can't really imagine what the latter is, other than perhaps signing byte values within a structure rather than the structure as a whole, and if that's it I don't understand how it's a non trivial difference, or what you mean by 'byte sequences mapping to a given meaning'.
Not looking to roll my own, just curious!
What they'd like instead is a signature format that simply signed the raw request, the "bytes".