Hacker News Reader

Just as an aside, anything using x509 certificates is a nightmare operationally. I sure don’t have an alternative, but it murders me every time. Now when I architect clouds I consider it a core service, not an afterthought - an absolute minimum set of services and some strong runnooks to generate certs, distribute, and renew/rotate them. All applications these days support them, but don’t support managing them well in my experience. There’s glue needed for each app to request a cert and install it.

I recently ran into problems with not being able to pass client certificates through to a server behind a load balancer/ingress (think Apache/Nginx) for the app on the other side to consume. The information online hints at either it not being possible altogether due to how TLS is designed, whereas other alternatives mess around with parsing/setting headers, but all of those feel hacky.

So i'd like to suggest that not only x509 is problematic, but in certain configurations mutual TLS is also a non starter, though there are certainly other challenges in regards to certs themselves - for example, managing expiry dates properly, sometimes needing a CA with the proper chain of trust, other times needing to mess around with the formats for particular tech stacks (especially problematic in Java, in my experience, with its keystores and trust stores) etc.

That's one of the reasons why at work we recently moved one of our internal services over to JWT for server-server auth. Now, granted, i did ensure 100% coverage with tests for the code that i wrote and largely relied on already established libraries for the low level bits and therefore mostly just wrote glue code, but setting everything up was a breeze. Now, of course, you do need to figure out which approach to JWT is better for your app, from symmetric encryption with something like HMAC512 and a shared secret, to something like RSA512 that once again deals with private/public keys but doesn't force its own network topology upon you.

In combination with something like overlay networks with additional encryption (in the case of containers or a service mesh), you can get a "good enough" level of security, though depending on the circumstances you might also need to look into how loosely/strictly your tokens are also validated and write additional logic for the subject/claim/issuer/audience fields, issuance date and expiry date checks and so on.

When forced to deal with x509 certificates, however, i think the best option is to go for the simplest possible use cases (e.g. server cert for SSL/TLS, instead of mutual TLS) and to manage them at an ingress level, ideally with something like Let's Encrypt. Using them for more than that is just asking for additional challenges and/or trouble.

This is unfortunate but it is not a requirement of OAuth with certificates or with the OAuth MTLS RFC. It may just be lousy policy on part of those government services.

The only time you should need actual certificates (e.g. trust chain with attributes) is when you have multiple authorization servers run by independent organizations. In that case, a central party may do certificate issuance so that each AS can tell that the client has been centrally vetted.

In all other cases, any use of certificates is because they provide a pocket in which to store a public key. A self-signed certificate is just as good for authentication (proving you are the same party as before), because it is the crypto challenge that is important.

For actual certificate issuance, we are unfortunately in a gap between browsers killing off the <keygen> tag and easy adoptability of newer schemes like ACMEv2. Providing public keys via password-protected zipped PDF files has me thinking that their solution for this problem was possibly ill-conceived, and they are issuing private keys for import rather than using CSRs at all.

Note that for multiple AS systems, one common pattern is to use dynamic client registration from OAuth, which was extended to support software statements. In this way the certificate can be used to get local, non-certificate-based client credentials with a particular AS.

The grants being opaque at the API level is unfortunately a common function of authentication and security functions - they don't want to tell you the thing you did wrong, and that is sometimes used as justification for making the errors mostly pointless. A good AS should have some focus on providing more information within transaction logs. However, if the certificates are being shared over MTLS rather than through JWTs the amount of error information is unfortunately more commonly constrained due to infrastructure layering.

> Secondly the whole token exchange thing is quite opaque. If there's something wrong with the certificate or the returned token, it can be quite difficult to figure out what's wrong from our end.

When you say "token exchange" are you specifically referring to the token exchange type grant that was ratified in 2020 or do you mean calling the token endpoint to get the token as part of the PKCE workflow? If it is the prior, I'm curious about your experience with this relatively new feature and why it's so difficult to use.

Sorry, I was being sloppy, I was just referring to the whole OIDC exchange from authentication to authorization.

> Some gov't services our users need to integrate with have moved to M2M OAuth/OIDC, using organization certificates for authentication.

Hmmm. Are they using MTLS: https://datatracker.ietf.org/doc/html/rfc8705

My understanding (from reading, haven't implemented it) is that this is quite a deployment problem, but addresses issues with bearer token security.

They just use https://datatracker.ietf.org/doc/html/rfc7517#section-4.7

Since the full chain has to be included, say hello to one giant JWT...

At my day job we run our own OIDC setup. Even though we own everything it's still a nightmare.