Authentication Techniques for APIs | Hacker News Reader

Authentication Techniques for APIs

docs.google.com

257 pointsSujan57 comments

Not impressed, particularly with the basic-auth description. Basic auth is purely a well understood vehicle for sending a tuple (aka the credentials) for authenticating a HTTP request, most of the concerns highlighted are with regards to how the credentials are acquired and potentially reused across requests - that has nothing to do with the HTTP protocol. For example, my API product scanii.com has used basic auth for 7+ years and I firmly believe it strikes the right balance between security and easy of use. Besides fairly complex key/secret tuples for server side usage, we also provide one-time auth tokens for when you want to make API calls directly from a web browser (or another insecure device).

Author here. The doc does acknowledge basic auth as a valid approach for server side usage, but the language was a bit too pessimistic about basic auth.

I just modified it to say the following

    If you use HTTPS, then basic authentication is a good choice for server side only applications. It is the easiest to get started, and is well supported by clients and server frameworks.

It prevents delegation, though, which is an important use case for some APIs. As in "I want this third-party service to read my calendar from another server, but I don't want it to hijack my account, and I want to be able to revoke its access later on."

That's an implementation detail of how the credentials are created, managed, interpreted by the server, and their use reported on to the user, none of which is specific to the credentials transport or encoding, which is all basic auth is. The thing to be aware of is how different HTTP clients, specifically user-interactive browsers, use (apply and remember) the credentials.

And if you dig deeper in this direction, you will find yourself Greenspunned into Kerberos.

If you reinterpret Basic auth as "send a token that's not the user's password in the Authorization header", you're just doing OAuth 2 but writing "Basic" instead of "Bearer".

Basic auth is only OK if you don't have :80 open. Clients will send the creds over the clear to :80. It doesn't matter if you reply with a 400, the creds are already compromised at that point.

Even if you don't have :80 open, that doesn't mean there isn't a MITM that would accept the connection instead of you.

As long as https:// prefix is used, this is not true, MITM cannot downgrade that.

plus a HSTS header for any type-in traffic.

As a rule, HTTP basic auth is inappropriate without SSL, even for intranet apps. An office could easily have an insecure Wifi point, and someone sitting outside running Wireshark.

Also important to note that even on secure Wi-Fi with WPA2, if the attacker knew the password to the network they can just as easily sniff such plaintext ocontent.

I feel that this table leads readers into thinking that the top axis is a set of things you need to choose between, but many of them are orthogonal. JWTs, for example, can be used as a particular (standardized) "stateless session cookie" if you store them as a cookie. They do not, contrary to what the table says, need to be stored in local/session storage, though that is an option. Similarly with "Random Token" and "Stateful Session Cookie".

OAuth, similarly, is orthogonal to the rest of the table. You can easily use JWT as your token format in OAuth.

The key decisions you have to make are something like,

* Do I have any sort of "authentication token", or does the client simply auth each request?

If you go w/ authing each request, you get things like HTTP Basic Auth, or AWS's style.

If you go with some sort of token, then you get both of the following questions:

* If token, do you store the token in a cookie, or local/sessionStorage? (The former is vulnerable to CSRF, the latter can be read by JS in an XSS.)

* If token, what kind of token? A reference to a row in a DB, or do you just give the client a signed and possibly encrypted blob detailing the authentication?

If you go w/ a signed and possibly authenticated blob, then JWT provides a standard format for representing that data and many of the concerns around authentication tokens, such as expiration.

I'm probably over-simplifying a bit. The table just doesn't — for me — do a good job of showing the various choices along the spectrum, and what consequences arise from which choices. (Aside from also conflating JWTs with where you store an auth token, and Oauth with auth tokens)

("Stateless Session Cookie" is a bit of an oxymoron.)

Author here. The spreadsheet started off with a brain dump, and then was revised into this blog post - https://hashedin.com/2016/07/05/choosing-right-authenticatio.... Perhaps this is more in line with what you have laid down?

The overall structure, yes, I think is much clearer than a spreadsheet. The specifics leave me with a lot of questions though:

What do you define "session" and "token" based authentication as? My best guess is that "session" means an opaque identifier that identifies the authentication details stored server side, and "token" means a complete structure that details the authentication, but to me, those are odd words to associate with those things. Nonetheless, Googling that seems to indicate that you're not the first to use these terms this way, and that this might be accepted terminology for these concepts that I've just missed entirely (hopefully I've gotten the right impression from resources like [1]). I'm not sure I'm fond of those words for those concepts, if that is indeed what you mean, since they don't seem to provide real clues as to the thing they describe. (Why is an opaque DB ID not a token?) That said, I've not got a discrete word for these concepts.

> If no, or if you don’t know what CSRF is – prefer token based authentication.

But token based authentication — if you do mean a signed blob of authentication details, such as JWT — is no more inherently immune to CSRF than an opaque "session" ID into a database somewhere. Either can be stored in a cookie, and vulnerable to CSRF, and either can be stored in localStorage and thus immune to it. How you structure your authentication credential (JWT or a opaque DB ID) is orthogonal to how you store/transmit it on the client.

> You should use JWT as a short, one-time token, and not as something that is reused multiple times.

Why? Does that not completely defeat the point of a JWT? I suppose you could do this, and I suppose it would work, but that would require the client to reauthenticate each individual request, and that's more overhead than most people are willing to bear, and I don't feel that's really how most people use a JWT. (I would say most people have an authentication server or endpoint issue a JWT for a reasonable set of audiences for a reasonable length of time, and then use that JWT over multiple API calls.)

> Instead of JWT, create a random token, store it in redis/memcached, and validate it on every request.

Again, one of the great points of a JWT is that you can validate it with little to no network traffic.

> Session based mechanisms are painful because a mobile app doesn’t automatically maintain and send the session cookie. For a mobile app developer, it is far easier to set an authentication token as opposed to setting a session cookie.

Again, here, I feel like we're mixing up whether we're using an signed-authentication-structure "token" or an opaque "session" ID w/ how it's transmitted (via the Cookie header, or via the Authentication header). While I'd personally prefer Authentication simply because the structure and function of that header is simpler. But, why is it easier for the mobile developer to set one header over the other, really?

> Signature based mechanisms are generally not useful because you cannot embed secret keys in a mobile application.

We're talking about presumably authenticating the user of the mobile app — why would his or her credentials be embedded in the application at all? (If you mean a signature based auth like what AWS uses, the AWS id/secret would presumably be stored very similarly to however a mobile app would store my username/pass, or a JWT, or a session token; the issue is the device's storage mechanisms, not the authentication protocol. I'm not saying you should store an AWS key in an app, I'm merely detailing that type of flow, e.g., an HMAC of the actual request details.)

> If yes, prefer token based authentication, because signature based auth requires clients to store secrets.

A JWT token is a secret. A username and password is a secret. A DB identifier for a session is a secret. Any of those, if divulged, result in the ability to make authenticated requests.

> OAuth 2 uses token based authentication.

This doesn't jive with the definitions of session/token auth that I've been using at all, so I'm lost. OAuth2 works with either.

> Server-to-server API calls, where the client can store a shared secret and generate a new JWT for each API call.

This statement is so unusual to me that I want to know how you think JWT works. If the client can create JWTs, why use it at all, as opposed to the much simpler basic auth, or AWS-like signature-based authentication?

For reference, to me, a JWT is a standardized structure that describes the assertion that a particular subject (typically a user) is authenticated, potentially for a limited set of things, typically for a limited amount of time (it expires). It's proof that the user authenticated in some manner. Typically,

1. a client would authenticate w/ an authentication server (potentially, the same server it'll later make API calls to, particularly for small services, but not necessarily); if successful, the server returns a JWT, good for some period of time. (This step could be OAuth, the access token could be a JWT, but does not need to be.)

2. a client proceeds to make API calls using that JWT, supplying it in some manner, perhaps as an Authentication header, perhaps as a cookie. (of course, the server/client must agree on the method)

3. the receiving server can prove that the client is authenticated from the information in the JWT alone, since it's a signed message from the authentication server claiming that. (Excepting some things about revocation of JWTs.)

(I'm not saying this is the only way to use JWTs, just the way I would expect most web applications to make use of them.)

[1]: https://security.stackexchange.com/questions/81756/session-a...

Who is the author? It's just a bare Google Doc AFAICT.

Some quick notes: random tokens do rely, broadly speaking, on cryptography, as the token must be generated by a cryptographically strong (P)RNG.

Signing and encryption are conceptually different ideas, don't mix them as in the description of the stateless session cookie. There's a great old Matasano blog post in the style of a play about this...

A more complete consideration of authn techniques would include mutual certificate ("SSL") authn, as well as signed-request approaches not using a shared secret.

This one.

https://news.ycombinator.com/item?id=14290234

Author here, though not the one who submitted it.

>> random tokens do rely on cryptography

True. Random tokens do need a cryptographically strong PRNG. But when I wrote no cryptography, I meant there is no encryption or signatures. The token is used as a key to a server side data structure.

>> Signing and encryption are conceptually different ideas, don't mix them

Not sure where I mixed them? A stateless session cookie can be signed or it can be encrypted - both are valid. A signed cookie would let clients see the contents, but wouldn't let them modify it. An encrypted cookie would make it opaque to the clients.

If you're passing sensitive data to the client (the cookie) it's a hazard to encrypt only without a signature. _Always_ sign and verify, sometimes encrypt before signing / decrypt after verifying signature.

What's the hazard?

The problem is the client can modify and post the ciphertext all it likes until, eventually, the server does something undesirable, since the server is merely decrypting and acting on the data without verifying it hasn't been tampered with.

Why certificate based authentication wasn't included?

The server stores the public cert from all clients and uses it to authenticate at the transport layer. No shared secrets either.

Yes, that's a miss. I'll add it as another option. Thanks!

Every time I see "API" and "authentication" I go look to see whether they have incorporated the best practice of always be willing to accept two different authentication credentials/passwords/etc. So that upgrades can happen where first one side will accept a new password, then the other side switches what it sent, then the old password is discontinued.

Rather than the common "big bang" upgrade where everything has to update at once. (A process which is so scary that in fact it turns into passwords NEVER getting updated.)

Yeah, sadly this idea is never mentioned. :-(

I seriously don't understand the hype around JWT. They're useless to me since they can't be revoked. It's so much easier to just use a secure random token with an expiry. I could understand JWT at-scale to avoid hits to the DB, but none of us are likely at that scale.

They can be revoked. Our JWTs have an ID in them. We have a revoked table. If the ID is in the revoked table, the JWT is invalid. It requires a DB hit but it is worth it for the ability to revoke them.

What advantage does this have over a session token/table? If you have to hit the db every time the JWT is referred to for an operation in order to prevent post-revocation activity, what advantage does the JWT provide?

We keep our ID blacklist in something like Redis. For how rarely we need to add clients to it, the switch to simple-state JWTs has made noticeable improvements to the speed of our API, especially during higher usage times where our Postgres DB bogs down a bit.

So I'd say the advantage really varies by where you're coming from.

You could even switch what you have in redis to a bloom filter if the space of revocation tokens gets large (which it shouldn't).

You don't need to support a database online if you're just checking for membership in a small finite set.

Exactly, the database solution is overkill for tokens that are revoked prior to their expiration.

You also just do it in Varnish/Nginx/etc and stick that in front of the application server entirely.

That's my whole point. If you're doing that, you might as well not use JWTs seeing as that defeats the entire purpose of using a stateless token that doesn't require DB hits.

You can cache the table — which is usually empty — in the places where you need to validate a token to not require the DB lookup. You can also push changes in the revocation list to those places as well. Or, you can just have the DB lookup, but then cache it for 5 or 10 minutes: revocation will take that long to take effect, but most authentication verifications will still be very cheap and not require a lookup.

JWTs allow it to be stateless, or some tradeoff along it appropriate to your needs. Having the client pass an ID to a database row means you can never get that behavior, as you always need to look up that row from the DB.

In addition to single token revocation, you can use the account epoch approach (at least that's what I call it):

Store an "epoch" date in the database for each account. Tokens issued for that subject are considered invalid if they were issued before the epoch. This allows you to do mass token revocation (e.g. sign out all active sessions) without knowing the IDs of the tokens that have been issued.

This is the approach I take. The subject should always be queried, so just use a field on that subject for the purpose of killing current sessions.

Think of JWT as a lightweight SAML, to enable Authority A to vouch for User P's identity, roles, or other claims, in order to access a service (the "relying party") that trusts Authority A to do this, but does not itself maintain a way to authenticate or describe User P.

If you are both the security authority and the relying party then maybe JWT will not be useful.

A simple approach to revocation is using an ID + some kind of state maintenance (maybe redis) in conjunction with the token's "exp" attribute for easy TTL'ing of that extra state. This is probably no worse than CRL/OCSP features of PKI.

Calling the last column "OAuth" over-simplifies quite a bit. Specifically, there are ways to do OAuth like client credentials that don't involve 3 parties. OAuth provides a standard way of solving authentication, and it makes life easier for developers. Saying it's "overkill" might turn people away from a solution that could help them get and retain API customers. More about client credentials here: https://tools.ietf.org/html/draft-ietf-oauth-v2-31#section-4...

JWT docs aren't accurate. Also why not just store JWT in the cookie? It's pretty trivial to use JWT in a cookie and do a sliding refresh on server side when its close to expiring. You can use JWT this way with zero code on the client.

Same with the "Random Token". You can easily just shove a secure random ID in a session cookie and use it. Doesn't pretty much every framework support this?

This docs is... not accurate for most of these.

Wouldn't you need client code to cover csrf cases?

you just cover them with a CSRF token embedded in page like ye olde days :)

> JWT docs aren't accurate?

I'm curious - which portion of the document isn't accurate?

> Why not store JWT in the cookie?

At that point, it becomes a "stateless session cookie". It's a valid pattern, just called a different thing in the document.

> Same with "Random Token"

Yes, and the document calls it "Stateful session cookie", and even highlights that every framework supports it.

Also, JWTs can be used with OAUTH2.

What about JWT with Refresh Tokens? How does this fit in the list and does it help alleviate any concerns covered by the other methods?

Keeping this information, which I think is very useful, on a github repo here: https://github.com/teamsecure/authentication. Submit PR or create issue and collaborate if you feel like to do it :)

Question about this from your blog:

> Does your web framework protect against CSRF? If no, or if you don’t know what CSRF is – prefer token based authentication.

Can you elaborate on this? If I implement CORS and also make sure my API is well-designed (for example, GET requests are truly idempotent and do not mutate anything), am I not protected from CSRF?

Thanks!

CSRF is an attack where an attacker can forge a request that appears valid from the server's point of view, and convince the browser to execute that request.

Typically, it arises if an application stores its authentication credentials in a cookie, and the server checks only that cookie. Because cookies are always send along w/ requests, a malicious website can construct and submit a POST request to execute an action, and the browser will attach the cookie to that request, thereby erroneously authorizing it.

A simple separation between GET/POST is not necessarily enough. Third-party websites can make POST requests by creating <form> tag in a hidden <iframe> and submitting that form w/ JavaScript upon being loaded in a user's browser. (You can't submit arbitrary POSTs like this; the browser is limited in what mimetypes it can construct, for example. For example, it isn't possible to construct a Content-Type: application/json request in this manner, so not all POST endpoints are vulnerable if they discriminate on Content-Type.)

Also, some CORS requests do not get preflighted even if they're POSTs, so a CORS-unaware server can end up executing those even if the browser won't give the result back to the requesting JS (the result doesn't matter; the action does).

https://en.wikipedia.org/wiki/Cross-site_request_forgery ; https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(...

After reading through this it seems no clear winner, most of them are "painful" in implementation?

Author here. I wrote a simpler blog that summarizes the spreadsheet - https://hashedin.com/2016/07/05/choosing-right-authenticatio.... It may help you to find an approach that meets your needs

Thanks for making and sharing this document. I need to choose an authentication method for our distributed application.

I'm confused by the type of usage covered. Are these for web sites accessed by browsers (users), or clients (e.g. swagger) accessing micro service REST API servers ?

Support of cookies is built in browsers, but what about authentication with clients ? It looks like JWT is the most versatile and supported in many languages. But it's still unclear how I'm supposed to use them.

Where could I find more about this ?

For clients outside of the browser, you have a few choices. JWT token passed in authorization header, or a secure random token passed in authorization header, or a signature that verifies the client has access to a secret.

The easiest way is to create a secure random token and store it in database or in-memory cache like Redis. When a request comes in, you extract the token and check your database/cache if the token is valid. The drawback is an additional network call to the database/cache every API call. If your application can afford this extra call, this is a pretty good approach.

With JWT, you don't have to make an extra network call, because the token itself has everything you need to verify if it's valid. But, if you want the ability to revoke the token - then you will need to maintain a blacklist of tokens somewhere, and check against this blacklist on every request. Arguably, at that point, you lose the original benefit of JWT - the ability to bypass the database/cache.

And finally, if your clients are server side applications only, then you can opt in for signature based approaches.

Is there a book to learn more indepth about all these?

Its called the internet

OWASP [0] is a good resource for most things Web Security related

[0]: https://www.owasp.org/index.php/JSON_Web_Token_(JWT)_Cheat_S...

Not specifically about the web, but Cryptography Engineering: Design Principles and Practical Applications is my favorite book on the subject. It's not a thorough treatment of the subject of security, but it shows how much care needs to be given to such systems and why security is hard in general. Throughout the book you will find plenty of advice on how to approach such problems, even for the web.

For traditional sessions:

1. All You Ever Wanted to Know About Sessions In Node.js (Stormpath) - https://stormpath.com/blog/everything-you-ever-wanted-to-kno...

2. Express.js Production Best Practices - https://expressjs.com/en/advanced/best-practice-security.htm...

3. express-session (npm module) - https://ewiggin.gitbooks.io/expressjs-middleware/content/exp...

4. Node.js authentication strategy using Passport.js and Redis - https://blog.risingstack.com/node-hero-node-js-authenticatio...

For JSON Web Tokens:

1. JWT Handbook: https://auth0.com/e-books/jwt-handbook

2. IETF Specification: https://tools.ietf.org/html/rfc7519

3. https://jwt.io/

4. Authentication in React Applications, Part 2: JSON Web Tokens - https://vladimirponomarev.com/blog/authentication-in-react-a...

6. Token Authentication Implementation - https://docs.docker.com/registry/spec/auth/jwt/

7. JWT Validation and Authorization in ASP.NET Core - https://blogs.msdn.microsoft.com/webdev/2017/04/06/jwt-valid...

I have learned a lot about OAuth from: Mastering OAuth 2.0 by Charles Bihis.

I'd add a section/row on `browser vulnerability: XSS`.

Reply on news.ycombinator.com