JWT vs. Opaque Tokens
zitadel.com
zitadel.com
JWT is a solution to a problem 95% of us don't have: https://apibakery.com/blog/tech/no-jwt/
Stripe has been doing that for their public API since forever, and that's one of the big reasons why it is so easy to work with them.
For stripe, though, I'm surprised they wouldn't want to ensure internal data requests are being made on behalf of customers who should actually have access to the data.
Also, for bigger companies (although, why not smaller ones?), you hit a point where you can't keep blindly trusting every service. Better to have internal security controls.
As for when these tradeoffs are appropriate, hard to say. But a good rule of thumb could be when the product is big enough to need an SOA.
And that is my biggest problem. If you don't know whether you need something, you don't need it.
Or you just don't have enough expertise to understand why you need it, but that means you will implement it the wrong way anyway.
That's a bad approach for security.
Starting with the solution is the wrong way to do it, but you certainly need to actively look if you have problems.
Should you do criminal background checks? Install spyware on laptops? Forbid hiring from other countries? Restrict developer access to production env? Invest all your development time into security features until you run out of those?
If you actively look for problems, you will justify all of those.
Security is not about knowing which problems exist and fixing them all. It's about knowing which risks are acceptable at your current stage. If you don't know which risks are acceptable, applying random security practices is worse than no security at all. At least when there's no security you are aware of it.
If you can't come up with an attack vector where HTTP Basic Auth causes significant problems, you don't need JWT to secure communication. And even if you can come up with an attack vector, is it really easiest one to execute?
Speaking of attack vectors, how many of the folks that use JWT for inter-service communication actually rotate encryption keys when anyone who has access to them leaves the company? My bet is very few.
It's a serious question. As time goes on I more and more land on the side of Linus Torvalds wrt to security.
It kills me the amount of stupid shit I've seen done in the name of security.
If security people were to come up with driving standards, no one would be allowed to turn left because it's more dangerous then turning right.
1. An ingress moat - your ingress makes decisions on whether someone should be allowed in, after which internally anything goes.
2. Two-stage auth - building on top of the above, internal messages need authentication/authorization for communication channels to be established. This might be basic, x509, kerberos, or federation-based protocols. Microservices environments might use SPIFFE for this.
3. Messenger auth - in addition to the above, requests need to show evidence of direct/indirect interaction with the parties involved. For example, an access JWT may be sent around since it would provide indication of a user and tenant.
4. Intermediary-authorization - a variation of the above, evidence is required on each communication channel 'hop' that the initiator(s) are authorized to make the request against a service. This may involve more centralized services to create many opaque/integrity-protected authorization decisions from an external systems view.
5. Transactional auth - another variation of 3, evidence is required specifically that a given action is authorized, including parties and potentially parameters. This might be sent directly from the original initiating party, or may be created for internal systems close to the ingress point.
Note that for a service with a public API, all of these can hypothetically look identical. However, once you get to a certain amount of internal authorization checks, you naturally start to try to represent those with integrity-protected stateless tokens to avoid needing a massive centralized system with low latency consistency and high uptime needs. OAuth access and refresh tokens were designed to help there.
https://web.archive.org/web/20180324054020/https://storify.c...
https://twitter.com/FiloSottile/status/1300946068411121665
https://insomniasec.com/blog/auth0-jwt-validation-bypass
https://paragonie.com/blog/2017/03/jwt-json-web-tokens-is-ba...
1) not having to worry about authentication in their app (and not about security issues, exploits, account recovery, fake accounts, bots, ...)
3) the ability to use it in any framework they want (e.g. the Go standard library "framework")
4) the ability to "upgrade" to full microservice authorization
And yes, these are sort-of kind-of not-quite-but-almost provided in Django, Spring and Ruby On Rails, but if you want to use whatever framework you want ... JWT is definitely the closest thing available, or at least it seems so.
WTF? They just spent the whole article talking about how bearer tokens were stupid (everything they said about JWT applies to all bearer tokens). And then they say "instead of this thing we said was stupid, we use the exact same thing but named differently". WTF?
Remember they stated:
> Major advantage of JWT compared to bearer tokens [...] is that they don't require looking up the token.
Other problems with JWTs vs opaque bearer tokens they listed include:
- increased attack surface due to complexity of JSON parsing/validation
- misconfiguring JWT libraries to allow no signature
- leaking data to users because you thought JWTs were encrypted
Now you are right that some of the complexities involved with JWTs like refresh token can apply with arbitrary bearer tokens, especially if you want to limit the lifespan of the access token, but those are implementation choices. Technically you can just have a single long lived bearer token instead of a long lived refresh token, and a short lived access token.
> You should not use Thing X because the advantage it has over Thing Y is not an advantage. In fact both Thing X and Thing Y have the same disadvantage. Except that Thing X has a workaround and Thing Y doesn't. But that workaround adds (gasp!) complexity! Obviously, we only ever use Thing Y and you're a bad developer if you use Thing X.
It's smoke and mirrors. They never give an advantage of bearer tokens over, well, JWT barer tokens, which are a type of bearer token, the thing they said they use. They just said JWTs don't actually have one specific advantage, except when they do. That's not an argument against JWTs!
> increased attack surface due to complexity of JSON parsing/validation
People who use JWTs are already parsing JSON everywhere. It's not hard. If it's not valid JSON, clearly you don't authenticate.
> misconfiguring JWT libraries to allow no signature
You can misconfigure anything. You can misconfigure your server to not actually check the bearer token. Why is it more likely you'll forget to check the JWT signature than the bearer token? This is something you write one time and never touch; it takes an afternoon at most.
> leaking data to users because you thought JWTs were encrypted
Yes, you should not put sensitive data in a JWT, encrypted or not. This is a fair criticism.
> Technically you can just have a single long lived bearer token instead of a long lived refresh token, and a short lived access token.
Which, as they pointed out in that article, is a disadvantage of long-lived bearer tokens.
I agree with the premise of the article, which is that JWTs aren't the right answer for every solution. Ine pattern we've often seen to mitigate some of the issues of JWTs is to store them serverside, in a session. Now you get all the benefits of session management (revokability, single view of usage) but can still present a JWT to other APIs if needed.
I'd never put sensitive data in a signed JWT. In fact, at FusionAuth, even JWT claims like the user id is a UUID, just so you don't inadvertently leak information like "how many users are there" via an integer user id.
The main reason why JWTs work so well is their statelessness. By that I mean the fact that you can verify their integrity without "calling home" (especially if you've set up asymmetric keys correctly). In this case, you don't have the same availability and scaling requirements for your server that issues JWTs that you would for a server that was responding to opaque token requests.
It's from last year, but I compare and contrast these two approaches in this video I presented at a meetup, starting a few minutes in: https://youtu.be/rArCF7nUcvY?t=451
JWTs are part of building an app that scales. But like any other scaling choice, it has consequences. Just as you wouldn't shard a database before you need to, you shouldn't build an app that uses JWTs to scale unless you need to. Though switching between sessions and JWTs is going to be an easier transition than sharding vs unsharding a database. Data has inertia and all that.
I don't believe there is much functional difference between sessions and opaque tokens. Both have to be stored somewhere, both require a central source of truth, both require that store of truth to be constantly available. There are differences in how they are managed/acquired, but from an architectural perspective they are similar.
…
> The main reason why JWTs work so well is their statelessness.
You realise this is obviously a very weak argument for using them? Stateless in a way that is apparently not useful for most people. Great.
I’ve never seen a serious JWT implementation that didn’t use a database to manage lockout and other state related things that inevitably turn up as product requirements.
Once you do, you now have a stupid pointlessly complicated session implementation that potentially leaks sensitive information.
That’s the whole point of the article.
If you you need jwt for “other services” issue jwts for them when they become a requirement.
Well, it depends. With JWTs in a session, you get a time bound set of credentials that can be verified server side, with rotation built in and a lot of docs, libraries and experts thinking about things. (Regarding that last, there's a lot of brainpower put towards edge cases and security in the OAuth working group!)
Other sets of users use JWTs in their pure stateless form, often with a short expiration time in a secure HTTPOnly cookie, transparently refreshing often. This offers easy horizontal scaling, as long as you only want to talk to servers in a single domain (you can always proxy calls other domains if needed).
I don't think that JWTs are a magic bullet, but they definitely have some strengths.
This seems weird to me. The whole point of the JWT is that you know with confidence that the client has the data you need and cannot change it. I mean, if it works for them cool, but at that point you might as well just have a session with an ID and a table with all the relevant session information in one swoop.
I would change that to "The whole point of the JWT is that you know with confidence that the client can provide tamper proof data, often identity, to other servers and APIs."
Yes, the OIDC id token is for the client to consume, but if it is an access token, it is for the resource servers (to slip into a bit of OAuth jargon). Put whatever you want in the id token, knowing the client will have full access to it. If it is a javascript or native client, you don't know where that will go, but having it shouldn't grant access to anything privileged.
This is obviously why people should adopt OIDC.
I don't really get the hate for OIDC. Nothing says "Junior Developer" like posting on Hacker News how shitty this widely adopted, durable technology is, especially by calling it "JWT."
This token expiration boogeyman is especially dumb. If there's a security misconfiguration so disastrous that the user can exploit it in less than 5 minutes (a typical access token lifetime), you're only going to be booting them out of the system after they've done the damage anyway.
Surely not, because, it’s be obviously stupid using a refresh token right?
Not a long lived refresh token.
A refresh token… that you might, say, need to revoke at some point?
Somehow.
Definitely a boogeyman. …because people don’t understand how it works.
This word, revoke. It appears in OIDC exactly twice. You can look it up yourself. In order to get a new access token, you have to hit an authorization server URL, not an application server URL, with your refresh token. If it was revoked, you'll know then. The authorization server will simply not give you a new access token, and while you have a string of bits still called a refresh token, it is as useless as an old password.
I get that your snark is stylized, and this is Hacker News, and uniformly, the junior developers in my life that I engage with deploy snark, that's all fine. Really the emphasis is on how much the OIDC people have thought about all these things, and while I agree it is a lot to learn, it is in principle the only winning standard for authentication and authorization nowadays, it is what everyone uses, so you might as well learn it.
I'll explain: A UUID should be pretty unique in the universe. Wherever it crops up, as there should always be an assumption of the possibility of leaking data and associated data at the same time, whomever the UUID belongs to has had their information leaked. That could be associated meta data in a request like IP address, to a print out that has a user's UUID in the header. In all cases it ties data to an individual.
Better is to just use a high integer that doesn't simply increment, but changes by an amount. But low enough to blur into noise. Heck, this can be rotated.
I stand to be convinced otherwise, but via the reasoning above I believe UUIDs are not desirable for user privacy.
You're saying that because UUIDs can be assumed to be unique in the universe, when it's leaked alongside other data it can then be correlated with other data leaks to tie data to an individual? How does a user ID being `8646846` make it any safer from leaks or correlation than `123e4567-e89b-12d3-a456-426614174000`?
That being said, always going to the database for connecting an opaque session token to an identity can quickly become slow, and if those features listed above are not desirable, having a blocklist of revoked JWT IDs in an in-memory cache (like Redis) can bring back some performance benefits.
Doing this obviates some of the benefits of JWT's statelessness, but for situations where revocation is really important and you can't have that few seconds of JWT validity after a user logs out, this totally works.
My employer has an article about this topic here: https://fusionauth.io/learn/expert-advice/tokens/revoking-jw...
PHP does this every single request. I’ve never had enough users that this became a significant issue (and you probably don’t either)
with JWTs you have the flexibility with the opaque you don't.
JWTs also allow you to do client-side logic on things like entitlements but then verify against database when the user tries to view something
Still very few companies need something like this.
If select by id or token is slow for you then I worry about the rest of the application quality. Probably much slower elsewhere that the identity lookup is the least of your concern.
A few reasons why you might want the opaque token come to mind:
* You want the OAuth ecosystem (the libraries, the scopes, the user permissioning)
* You are being forced to use an opaque token because your user authenticates somewhere else, and they only provide you an opaque token.
This is where a identity service also can help ;-)
Surely you could trust a signature and lifetime but that makes the cookie no different than a JWT (only storage wise the differ in that case).
Generally speaking it is recommended to use opaque tokens and rely on the userinfo / introspect endpoint call to make for the session management.
Only in specific cases where latency and/or scaling might become an issue you should opt for JWT. At least this is my opinion.
I think that is the point of the op: If you have to check a db to realize revocation, then you can use session cookies because they also need to hit the db. (Because you don't get rid of the db hit for using jwt).
As a rule, there is no difference. But if you are one of the exceptions, those can have differences of orders of magnitude.
You normally do lots of API requests while you are logged in. To services and to download an image for example.
You make sure the current jwt is valid for a few minutes and hit only the database with the refresh token for example.
Only in worst case you really need to block a token and it might be much easier to sync those few tokens in your system into some local cache and let them expire automatically (because you know when they expire as it is contained in the token).
But yes if all of this sounds complicated, use sessions and a distributed redis for your session or just the database.
> In addition, since [opaque] tokens are stored on the server, they can carry more data than JWTs, and you can easily revoke them if necessary.
Opaque tokens don't carry _anything_. THey're just an identifier used to query a database inside your application. The article repeatedly implies there is meaningful content inside an opaque token.
More messiness:
> Opaque tokens cannot be read by people that hold them since they are undecodable strings. A client cannot read the contents of the token
Sure they can be read. It's just that the totality of the content _is_ the token itself.
Opaque tokens are opaque to consumers. There’s no limit on what can be in there, as long as they’re opaque. So yeah, you don’t need that token to just be a PK. It can be a real payload (generally encoded).
> It can be a real payload (generally encoded).
Based on your definitions, a JWT is therefore a specific kind of opaque token. I don't think it is personally.
I think we really should be talking about tokens as falling into two categories, of which JWT is an implementation of the first:
1) stateless - the token contains meaningful content that can be verified by a consumer. JWT is such an implementation with a standardized way of verifying the content without additional database queries (excepting revocation here)
2) pure identifiers - the token is merely an identifier, and all consumers must reach out to some other service (eg redis) to retrieve the details/capabilities that the session should have.
Edit: alternative proposed terms: content-full, content-less
Content-full and content-less break that abstraction.
Anyway, I have always understood it as the token being opaque for the verifier too, that only ever tested it for equality. The token that is opaque only for the holder is normally called "encrypted". It may not be opaque to the issuer, but that doesn't change much.
But anyway, depending on what you read, you will get different meanings.
Which falls flat on its face the second you realize the "verifier" can pull out information based upon that opaque token.
It just means the intent is not for the person receiving it to do anything except reflect it back on subsequent requests. That's it. Typically it's done by handing something that is meaningless without context, said context not being available to the consumer being issued the token. But there is absolutely nothing that says you can't hand them a JWT meant to be used as an opaque token. If you're really worried about it, encrypt that bad boy before giving it to them.
I prefer session cookies for web applications. Sure you have to worry about CSRF, but that is easily solved with CSRF tokens. Furthermore, is CSRF even really an issue when you are using a JSON API and have CORS properly configured.
We recommend HTTPOnly, secure cookies for storage with an SPA. Diagrams here: https://fusionauth.io/learn/expert-advice/authentication/spa...
If you need to access APIs from elsewhere, run an API proxy server side that can validate the JWT and then forward on the requests.
Querying against over 100M on postgres adds one or two millisecond p99 latency to all requests. That's perfectly fine. In fact, that's about the same performance we saw with Redis. Both were over the network, so I presume the network accounts for most of that delay.
Having them in an rdbms is so convenient. It makes customer facing feature like enumerating current sessions very simple. It makes token expiration and remote logout simple. It makes spotting weird bugs much easier.
I can't recommend it enough.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
I've come to like JWT's for mundane, tiny projects because I can use https://www.keycloak.org/ and get a lot of boilerplate account stuff for free -- password reset, registration page, eula gate, et c.
it has all the information the server needs except the signing keys, so the server doesn't need to store this information server-side. This means that users can get a token from your authorization server and use it in another without those servers needing to consult a central service.
Now this may be nitpicking but it comes with an important implication: Your jwt validating party needs to have network access to the SSO service (unless you want to provide keys for token verification yourself, which I do not recommend), which is not always a given, especially in hybrid setups (On-Prem + Cloud).
One thing I just wanted to append to this here:
> Not necessarily in real-time, but at least on a frequent periodic basis.
Periodically fetching the keys is not really a best practice on its own. The consumer (RP) must be able to handle newly published keys at runtime. Since a JWT includes a KID it is recommended to lookup a local cache pre-filled from a scheduler while fetching new KIDs on demand.
Another option would to be prepackage N public keys with the resource server during deployment. This means there is absolutely no network access required, just filesystem access to the public keys.
You want more than 1 so that you can rotate the signing keys if you need to, without redeploying said service.
What is the correct value for N? I dunno, depends, but it's not hard to generate 100 public/private keypairs and then bundle up the 100 public keys into a deployment artifact.
Is this as common as using JWKS? No, not in my experience. Is it possible? Absolutely.
[1]: https://login.microsoftonline.com/common/v2.0/.well-known/op... (jwks_uri element)
and use it in another without those servers needing to consult a central service.
Recommended thought experiment: Every programmer who makes or consumes the tokens in your system needs to attend a meeting about the tokens. Picture that meeting in your head. Was it a huge endeavour, involving several hotels and many months of planning? Then you probably need JWTs. If the meeting could happen in a coffee break then JWTs are wildly inappropriate.
Don’t use it for storing sensitive data
Expire oftenIf you have one you can keep using it without extra network roundtrips as long as it doesn’t expire.
Only the validity of refresh tokens (for generating new access tokens) is checked.
With that they are a better choice than most homebrew/framework-specific solutions.
There are some other established (/"standardized") solutions, which might fit your use-case better though, but most of them lag behind JWT in implementations/dev familiarity.
Not good for security nor speed in the use-case of a auth token!
Unencrypted: maybe a user preference, or first name, or something that adds value but does not overlap with auth. Like if a frontend could serve the same functionality across three departments but the styling is different, the token's unencrypted claims could determine which style set to use.
Encrypted: user uuid, roles, maybe other known settings, things the backend can handle. Someone stealing a token will just try it anyway whether they can see claim or not, but because it's encrypted it will get decrypted so it's also a different logic flow than unencrypted and could trigger a process i.e. did it come from an accepted ip address or range? Does the ip match the previous? Does the ip match the other known ip for a websocket? So also in this sense we wouldn't want anyone to know what else we might check.
How would you do that? At the end of the day the front-end needs to know what you can access somehow.
I guess you could add a compile step that rewrites all your permission checks into validating opaque uuid’s.
The thing is that you cannot change the payload of the token without invalidating it, so you couldn’t set the role to admin and then still have the server accept it.
To answer the question in your other comment, the back end can, in principle, provide a boolean yes/no value for every authorization-related decision the front end needs to make: can_change_order_status, can_view_supervisor_dashboard, can_view_all_conversations, etc. That way you never leak the model (such as roles) underlying those decisions. In practice, I've never seen anybody bother. Instead they have the back end send the front end values that map 1:1 to the back end authorization model: is_supervisor, is_account_admin, etc. That's why I think for most applications it doesn't matter if users roles are encrypted in the JWT.
Story time.
Our company recently switched to JWTs for our SPAs from regular OAuth2. I told them up front - we need a way to invalidate a token if an account is compromised. The lead on this initiative said we can blacklist any token. I told him that's not practical because you would need to know the exact token to be able to blacklist and because we don't store them in the database and don't log them have any real way to figure out which token is being used by an attacker and which are not that it's basically useless.
I suggested a simple fix: cypher the private key we sign the JWT with the hashed password of the user account we're generating the token for. If an account is compromised, we can reset the password and invalidate all tokens for that one user.
I was ignored and lo-and-behold within a month CTO and tech lead are trying to track down JWTs for a hacked account. I told them we can update the app to cypher the key, it'll invalidate all tokens and people will have to log in again but at least they'll be secured. Nothing has been changed to this day, despite banging this drum for literally over a year. We've recently extracted our authentication into a micro-service (for no reason really, i'm still mad about this too) and still nothing has been done about this issue when it would have been the perfect time to fix this.
I love my job but "priorities" and "business cases" as an excuse for this kind of incompetence is rage inducing.
The point I'm trying to make is. You are most likely using some kind of web framework that can just plug in an authentication implementation, just use that. NIH is very real and it is more than a waste of time, it can be dangerous.
You can even maintain statelessness with opaque tokens if you really need it, e.g. issue both a JWT and a session cookie bound together by some shared value, say Fernet tokens. One goes in the JWT in local storage (anti-CSRF), the other goes in an HttpOnly cookie (anti-XSS), requests still require both to decrypt to the same value after the service passes them through.
Generally this is overkill, though. JWTs for the service and your preference for the app works fine decoupled.
You could store JWT in a cookie, or use opaque token in a HTTP header, for example.
That said, the session cookies we loved back in the day were usually opaque tokens, yeah.
As sibling comments point out, an opaque token can be stored elsewhere (though, to be fair, the session identifier which is in that cookie can be placed elsewhere too).
Cookies are limited in where they can be sent (https://developer.mozilla.org/en-US/docs/Web/Security/Same-o...).
Revocation is either offered by the authorization server or managed by the app. If the authorization server manages it, then JWTs vs. opaque tokens are not a concern because the authorization server issues and revokes its own tokens. If the app manages it, then generally it does so based on the token type. If the app revokes based on opaque tokens, it can handle any type of token, including JWTs. If it revokes based on JWTs, then JWTs are required.
Beyond that, the only differences between the two token types are size and data leaks. Size rarely matters (hehe), so just ignore that. Data leaks are only an issue if you app is leaking JWTs, which is usually considered a critical vulnerability. Remember that access tokens are the main units of identity and if I steal an access token, I effectively become that user/client.
For those unaware, JWT has the 2 ways of encoding it: JWS and JWE. Because of these properties, you can use it how people would use it for application level messaging. If you are already on a secure channel, you can sign or encrypt the message so application level checks can happen.
This makes it act like a replacement of PGP for you paranoid people out there.
(Btw, I recommend against this, and think people should use PGP or Tink, as it's easy to footgun yourself with JWS.)
They should be unguessable and unrelated to anything. Read 20 bytes from /dev/urandom (or whatever stdlib your language provides), create an entry in your store for token->user_id, then put it in a httpOnly + secure cookie (so Javascript can't access it), and send it to the client.
The "IP changes" is a common fallacy - even the mobile IP depend on the datacenter(s) process it. There is no grantee there will be a designated aliasing, and effectively there could be multiple IPs for the same client interleaving.