Ten years of JSON Web Token and preparing for the future

FWIW, I built a system previously that got around this "having to check the DB on every access to check for revocations" issue that worked quite well. Two important things to realize:

1. Revocations (or what is usually basically "explicit logout") is actually quite rare in a lot of user application patterns. E.g. for many web apps users very rarely explicitly logout. It's even rarer for mobile apps.

2. You only need to keep around a list of revocations for as long as your token expiry is. For example, if your token expiration is 30 mins, and you expire a user's tokens at noon, by 12:30 PM you can drop that revocation statement, because any tokens affected by that revocation would have expired anyway.

Thus, if you have a relatively short token expiration (say, a half hour), the size of your token expiration list can almost always fit in memory. So what I built:

1. The interface to see if a token has expired is basically "getEarliestTokenIssuedAt(userId: string): Date" - essentially, what is the earliest possible issuance timestamp for a token for a particular user to be considered valid. So, revoking a user's previously issued tokens means just setting this date to Now(), then any token issued before that will be considered invalid.

2. I had a table in postgres that just stored the user ID and earliest valid token date. However, I used postgres' NOTIFY functionality to send a broadcast to all my servers whenever a row was added to this table.

3. My servers then just had what was a local copy of this table, but stored in memory. Again, remember that I could just drop entries that were older than the longest token expiration date, so this could fit in memory.

On the off-chance that somehow the current revocation list couldn't fit in memory, I build something in the system that allowed it to essentially say "memory is full" which would cause it to make a call back to postgres', but again, that situation would naturally clear up after a few minutes if revocations went back down and the token expiration window passed.

This sounds more complicated than it actually was. It has the benefits of:

1. Almost no statefulness, which was great for scalability.

2. Verifying a token could still always be done in memory, at least almost. Over a couple years of running the system I actually never hit a state when the in-memory revocation list got too big.

You can keep revocations in a very fast lookup system (eg broadcasts + in-memory store), combined with reasonably short token renewals, like 5-60 minutes.

Massively cuts down the number of token validity checks, and makes the system tolerant to downtimes of the auth system. That's less relevant for basic apps where the auth data is in the same DB as all the other data, but that is rarely the case in larger systems.

For example, a magic link sent via email can have a substantial validity duration.

I fail to see the relevance of your scenarios regarding JWTs. I mean, I get your frustration. However, none of it is related t JWTs. Take a moment to read what you wrote: if your account is compromised, the attacker started abusing credentials the moment he got them. The moment the attacker got a hold of valid credentials is not the moment you discovered the attack, let alone the moment you forced the compromised account to go through a global sign-off. This means that your scenario does not prevent abuse. You are revoking a token when it was already being abused.

Also, as someone who implemented JWT-based access controls in resource servers, checking revocation lists is a basic scenario. It's very often implemented as a very basic and very fast endpoint that provides a list of JWT IDs. The resource server polls this endpoint to check for changes, and checks the list on every call as part of the JWT check. The time window between revoking a token and rejecting said token in a request is dictated by how frequent you poll the endpoint. Do you think, say, 1 second is too long?

> Same with roles; if you downgrade an admin user to a lower 'class' of user then you don't want it to take minutes to take effect.

It's the exact same scenario: you force a client to refresh it's access tokens, and you revoke which tokens were issued. Again, is 1 second too long?

Also, nothing forces you to include roles in a JWT. OAuth2 doesn't. Nothing prevents your resource server from just using the jti to fetch roles from another service. Nevertheless, are you sure that service would be updated as fast or faster than a token revocation?

> So then all you are left with is a unified client id format, which is somewhat useful, but not really the 'promise' of JWTs (I feel?).

OAuth2 is just that. What's wrong with OAuth?

Also, it seems you are completely missing the point of JWTs. Their whole shtick is that they allow resource servers do verify access tokens locally without being forced to consume external services. Token revocation and global sign-offs are often reported as gotchas, but given how infrequent these scenarios take place and how trivial they are to implement (periodically polling an endpoint hardly changes that.

It really depends on the system. In my experience, there are tons of apps that want to be able to revoke access but weigh that against transparent re-authentication. OIDC handles both nicely with:

* short access/id token lifetimes (seconds to minutes)

* regular transparent refreshes of those tokens (using a refresh token that is good for days to months)

This flexibility lets developers use the same technology for banks (with a shorter lifetime for both access/id tokens and refresh tokens) and consumer applications (with a short lifetime for access/id tokens and a longer lifetime for refresh tokens).

You can do it all via individual browser cookies but it will be complicated. However you can dump session cookies to a database and then you can do claims locally on the server and use that cookie to tie it all together.

So I think you can do it either way.

JWTs are mutually authenticated (shared secret) but cookies are not.

I agree that JWTs don't really do anything more than a cookie couldn't already do, but I think the use case is for apps, not web browsers. In particular apps that do raw HTTP API calls and do not implement a cookie jar. And then because most companies do "app first development", we end up having to support JWT in the web browser too, manually putting it into localstorage or the application state, instead of just leveraging the cookie jar that was already there.

We just recently had to implement an SSO solution using JWT because the platform only gave out JWTs, so we ended up putting the JWT inside an encrypted HttpOnly cookie. Seemed a bit like a hat-on-a-hat, but eh.

Now that JWT exists, there is a standard way to do it so you don’t have to write the same boring code a bunch of times in different languages. You just have one string you pass in one field and if you tell someone else that it’s a JWT, they know how to parse it. You don’t have to document your own special way anymore.

At the end of the day, it’s just a standard for that specific problem that didn’t have a standard solution before. If passing data like that is not a problem for your use case, then you don’t need the tool.

To use your Protobuf example, there was a time before Protobuf or tools like it existed. I can tell you that writing the exact same protocol code by hand in Java, PHP, and Python is absolute tedious work. But if it never came up that you had to write your own protocol, you neither know the pain of doing it manually nor the pleasure of using Protobuf, and that’s fine.

The use-case I always remember people presenting for JWTs was mostly part of the "serverless" fad/hype.

The theory was presented like this: If you use a JWT your application logic can be stateless when it comes to authentication; So you don't need to load user info from a session database/kv store, it's right there in the request...

The only way that makes any sense to me, is if your application has zero storage of its own: it's all remote APIs/services, including your authentication source. I'm sure there are some applications like that, but I find it hard to believe that's how/why it's used most of the time.

Never underestimate this industry's ability to get obsessed with the new shiny.

I had an eye opening experience many years ago with a junior dev (I was significantly more experienced than he was then, but wouldn't have called myself "senior" at the time).

He had written an internal tool for the agency we both worked for/through. I don't recall the exact specifics, but I remember the accountant was involved somewhat, and it was a fairly basic CRUD-y PHP/MySQL app. Nothing to write home about, but it worked fine.

At some point he had an issue getting his php/mysql environment configured (I guess on a new laptop?) - this was before the time of Docker; Vagrant was likely already a thing but he wasn't using it.

From what he explained afterwards I believe it was just the extremely common issue that connecting to "localhost" causes the mysql client to attempt a socket connection, and the default socket location provided in php isn't always correct.

As I said, I heard about this after he'd decided that connection issue (with an app that had already been working well enough to show off to powers-that-be and get approval to spend more paid time on it) was enough to warrant re-writing the entire thing to use MongoDB.

As I said: never underestimate this industry's ability to get obsessed with the new shiny.

Cracking 256bit by brute force is unrealistically unlikely as you said, and there are many systems that could be compromised by that compute, an isolated jwt sig seems like just a very specific example.

A nice benefit of JWT for me is that it can be asymm signed and verified (ID tokens)

JWTs across servers are typically used with signatures, not in HMAC mode (so no globally shared HMAC keys). Then the issuer simply exposes a JWKS endpoint for downstream consumers (so no additional maintenance to distribute public keys).

Not sure if I’m remembering correctly but isn’t it recommended to not store any identifiable information in a JWT precisely because of this?

Unfortunately, it seems like 99% of the industry decides which token to use based on Medium articles, LLM responses or how many unmaintained packages that implement this thing they can find on NPM.

JWT is mostly used as an access token, but for the vast majority of use cases it's a bad fit. If you've got low traffic no strict multi-region deployment requirements, random IDs are the best approach for you. They are extremely lean and easy to revoke. It's pretty secure: the only common vulnerabilities I can think of with this approach are session fixation[1] and timing attacks[2]. Both attacks are preventable if you take just a few simple precautions:

1. Always generate 32-byte session IDs using a cryptographically secure random number generator on authentication. (Never re-use existing session IDs for new logins)

2. Either use a cryptographic hash (e.g. SHA-256 or Blake2b) of the session ID a the database field used when querying sessions or make sure that the Session ID field is indexed with a hash-based index (B-trees are susceptible to timing attacks).

In cases where you really cannot use Session IDs, your service is usually big enough and important enough to use custom Protobuf tokens even a more special-purpose format like Macaroons. These formats give can be far more compact and give you full control on designing for your needs. For instance, if you want flexible claims (with most of them standardized across your services), together with encryption, you can use a combination of Protobuf and a libsodium secret box envelope.

[1] https://owasp.org/www-community/attacks/Session_fixation

[2] e.g. https://github.com/advisories/GHSA-cvw2-xj8r-mjf7

JWTs on the other hand allow to be used across domain, so that you can use JWT issued by your IDP on one domain, to be trusted on another domain. crypto signature helps in verifying integrity of data.

sessions are usually tied to a single backend/application server. Its hard to reuse a session data across different apps.

JWTs on the other hand allow sharing session data across different app servers/microservices.

I have a web app that I'm doing sysops for which ended up with both. The web devs insisted on JWT and cough "forgot" about the auth bearer bit in the header because their API didn't use it. I ended up amending and recompiling an Apache module for that but to be fair, they will support it in the next version so I can revert my changes. A few db lookups in the Apache proxy JWT module I'm using and you have your claims.

On the front of that lot you have Apache session cookies generated when auth succeeds against a PrivacyIDEA instance - ie MFA.

I suppose we have cookies for authN and JWT for authZ. Cookies could do both and apart from session I'm not too familiar but it looks like claims would require multiple cookies where JWT does it all in one.

Attempting to generalize it ends up in pain, suffering, and AWS IAM.

> Biscuit is an authorization token with decentralized verification, offline attenuation and strong security policy enforcement based on a logic language

— https://www.biscuitsec.org

We solved it by simply using bitmasks.

Say, you want to encode an access rule "allows reading from Calendar objects". The typical CRUD actions can be encoded with 4 bits. For example, all bits are zero => no access. The first bit is 1 => can create. The second bit is 1 => can read. Etc.

Then, say, if your system has 32 different types of objects, you can say that, "position 13 encodes for calendars". So you get 32*4 = 128 bits, i.e. just 16 bytes to encode information about CRUD rules for 32 different types of objects.

Sure it sounds complicated but if you move it to a library, you stop thinking about it.

It seems to be a NIH-ed serialization format with hard-coded ciphersuits. It doesn't seem to support use-cases like delegation and claims.

Verify: https://cozejson.com

Spec: https://github.com/Cyphrme/Coze

OAuth doesn't, OIDC does for the ID token[0]. OAuth, at least the inital RFCs, were released 3 years before JWT was defined. But many extensions of OAuth do require or support JWTs.

Either way, I'm just not sure the demand is there.

My employer has had an open issue for Pasteo[1] for years but hasn't seen much community support. Some other interesting comments here[2]. Looks like most of the implementations[3] are libraries rather than standalone auth servers.

0: https://openid.net/specs/openid-connect-core-1_0.html#IDToke...

1: https://github.com/fusionAuth/fusionauth-issues/issues/773

2: https://www.reddit.com/r/KeyCloak/comments/1e2h5w7/is_paseto...

3: https://paseto.io/

I think it is a real good trade-off, as in case of a security breach you have an easy way to mitigate leaked tokens. The downside is, your user will have to re-login all devices. If you do not want to burden your users with the login on all devices, you should ask your self how often you do have security breaches and leaked tokens, might be you have others issues going on.

JWTs are standardized (RFC 7519) and used outside the JS ecosystem. Not a vanity project

Though often overused and poorly misunderstood where simpler and more secure methods would suffice.