The complete guide to protecting your APIs with OAuth2 | Hacker News Reader

The complete guide to protecting your APIs with OAuth2

stackoverflow.blog

289 pointsmooreds65 comments

For another quick way to get started with OAuth2, I can recommend reading the 2.1 spec draft[1].

For a spec it is quite readable and pretty short. One reason for this is that it "[..] is an in-progress effort to consolidate and simplify the most commonly used features of OAuth 2.0." [2]

It does away with most of the complicated grants/flows of previous version and basically boils down to just two[3]:

- Authorization Code Grant

- Client Credentials Grant

Authorization Code Grant (which now implies PKCE) is what you most probably want in most cases. So instead of complicated decision trees[4] for most people it will be:

"Learn one grant and be done".

[1] https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-...

[2] https://oauth.net/2.1/

[3] There is also the Refresh Token Grant, but it cannot be used alone. There is also an extension mechanism to define new grants and we will have to see if it is used.

[4] https://alexbilbie.com/images/oauth-grants.svg

Author here.

Yes, I think that reading the OAuth 2.1 spec is a great place to start. They've taken 10ish years of real world learning and condensed it down. I first wrote about it 2 years ago[0] but have been following along on the email list and they seem to be getting closer and closer. I don't think it is at 'last comment' yet, but I know they've done a lot of work.

I am also excited about GNAP, which is a 'from scratch' attempt to solve much the same problem. They've made some great progress[1].

I briefly cover both of those in the article, in the "OAuth's future" section

0: https://fusionauth.io/blog/2020/04/15/whats-new-in-oauth-2-1

1: https://justinsecurity.medium.com/the-gnapathon-57ee110508ac

Ah, I just found out the Device Authorization Grant didn't make it in there, which is a bit sad - that was a really pragmatic way of building user-friendly authorisation for all kinds of devices where you don't have your password manager available. I even used it to build a CLI authorisation flow once.

It seems I have missed quite a bit of development here and will have to catch up, so thank you for the links!

I think oauth is abused as a way of protecting your API. Often it’s just an unnecessary complication which makes your API harder to use and your libraries bloated.

If Stripe can get away with just public and secret key pair, there is a big chance it’s enough for you as well.

Amen! I do not understand the rash of OAuth-style authentication mechanisms for pure server-to-server applications.

Simplicity on the the resource provider's end?

So I implement separate IAM flows for my website, app and third party api access?

Or... Make use of OAuth to standardize it, using code auth+pkce for web/app, and client credentials (not user/pass, this is client id + secret, scoped to permissions you want) for third party APIs.

The scoping is so useful, api keys tend to ignore that, so platforms like digital ocean only give you an all or nothing key, meaning I can't plumb an acme client in to do DNS challenges, without also letting it spool up a VM...

Simplicity on the client side. I didn’t understand the rest if your reply. Scoping is easy to implement on a user/key.

What user authentication does Stripe handle? As a user I've interacted with Stripe's services a lot, but I've never needed to authenticate myself to them.

https://stripe.com/docs/api/authentication

Even pure server-to-server applications with "no user involved" are starting to use OAuth authentication which I don't quite understand.

https://www.verygoodsecurity.com/docs/payment-optimization/a...

the `client_credentials` grant is literally made for server-to-server applications

But why bother with the extra call to request an access token? Makes everything harder for modest security gains.

Access tokens are short lived and can easily be revoked (when opaque).

Sending your service credentials to every service is arguably less secure since they can have lesser security or log unwanted things.

Revocation is actually a major problem with JWTs.

I totally agree with you and so I'll put a shameless plug here: I've created OAuth Hopper[0] exactly for this reason.

It essentially takes a resource behind OAuth, strips OAuth from it and exposes it with Basic Access Authentication. I'm using it to let clients that don't support OAuth access services that force it.

[0] https://github.com/bjesus/oauth-hopper/

Twilio as well

I agree: https://www.rdegges.com/2015/why-i-love-basic-auth/

I don’t really disagree with your statements, but I’d like to point out that:

1. Just because a large company does something does not make that thing safe or correct—especially when your risk appetite and attack surfaces are not the same.

2. OAuth has it’s place, but it alone isn’t enough to secure an API. With or without OAuth most applications need well defined identity management, authorization mechanisms (distinct from authentication mechanisms), and all the other pesky implementation details one needs to consider when there is a desire to guarantee the authenticity, integrity, and/or confidentiality of requests to and from said application.

It'd be useful to capture why OIDC exists (it addresses actual vulnerabilities that crop up when you try to do OAuth as a single-signon scheme rather than just as a way of delegating out the ability to schedule a tweet with a 3rd party service), and to pull out JWT --- you may not be able to avoid JWT if you're doing OIDC, but it's helpful to know what problems JWT addresses and what ones it doesn't, because if you can avoid JWT, you'll almost certainly be better off.

The only real advantage of OIDC is that it enforces a strict user structure, so different services can understand common fields like id and email. The security aspects of it can all be implemented with plain OAuth2 as well without the added OIDC layer.

I can see the benefit of OAuth if you have an API that multiple customers will use since it allows your customers to manage the security themselves, to create and revoke tokens. I'm wondering, is there any benefit to OAuth if the web service is for your own company, between your own servers, even if the traffic goes over the internet, compared to using a simple token in a header that you generate by hand and the api checks for a match?

Between your own servers – not really. For that use case OAuth2 will be just as effective as a single shared secret.

It gives you a single place to manage roles and access, RBAC. Saving you the hassle of managing api key generation- and provisioning for N consumers multiplied by M services. Second thing would be that you don’t have to think about api key rotations and revocation everywhere, the only auth is done to the idp (possibly with mfa) and from there everything else is short lived tokens.

I really hate the idea of OAuth. I expect that your API can be used with an app-generated password that I can stuff into an Authorization header. I don't like APIs that are not curlabel.

As someone who has never used OAuth2 before, this article was not helpful at all. Calling it a "complete guide" is a total overstatement.

Author here.

@brap, sorry it wasn't helpful. It's aimed at folks who want to protect APIs, but maybe I missed the mark.

This article (which I co-authored) may be more helpful: https://fusionauth.io/learn/expert-advice/oauth/modern-guide...

It was also discussed here on HN, if you want some different perspectives: https://news.ycombinator.com/item?id=29752918

Not op, but I found it a bit lacking given the title. From the article:

While I’ll dive further into how you actually use OAuth to protect an API in your system below, including code examples, [...]

I failed to find much concrete around how to use OAuth to protect an API, and no code samples. Is this a part one in a series? If so, it should be more clear about that perhaps.

edit: For example, I just had to secure my non-Azure API using access tokens issued by Azure AD. Lot of details[1] about how to verify that they're valid, actually issued by Azure AD (one does not simply download a certificate), and it's issued for the right thing.

That said, I found the why and standards section very nice, and the grant selection sections were helpful.

[1]: https://docs.microsoft.com/en-us/azure/active-directory/deve...

> I failed to find much concrete around how to use OAuth to protect an API, and no code samples.

I think this may have been an oversight during the final editing process. I ended up having to split this up into two articles, as you surmised.

The second one is a blow by blow of the authorization code grant (and how an API should validate a token) and will be published later this week.

But I should have caught that we promised the code in the intro and removed that.

Just to expand a bit on my criticism. It's called a "complete guide to protecting your APIs with OAuth", yet there's no discussion of:

- Authorization server, roll your own, run your own (like Ory Hydra), or use third-party (like Azure AD)?

- Tokes, if you can choose, should you go with opaque tokens, JWT or something else? What about token lifetimes?

- When you get a token from a client, how do you validate that the token is valid?

There's probably more, but I feel at least these points should be included in a "complete guide".

Change the title and it's a fine OAuth blog post. edit: for example, add ", part 1 - introduction".

Thanks for this critique. I have limited ability to edit the post, but will try to answer these here.

> Authorization server, roll your own, run your own (like Ory Hydra), or use third-party (like Azure AD)?

If by 'roll your own' you mean 'use an open source library like Python's OAuthLib', all three of these are valid options. Really hard to give general guidance because it depends on the complexity of your application environment (1 app in rails? 10 apps with mix of custom and COTS?) and operational maturity (do you want to run your own server for compliance or control reasons? do you have the skills to do so?).

> Tokens, if you can choose, should you go with opaque tokens, JWT or something else? What about token lifetimes?

"if you can choose" is an important clause here. Many times it'll be dictated by the Authorization server you choose. All other things being equal I'd lean toward JWTs because of their widespread support, but I know others have other opinions.

Re: token lifetimes, this is something you need to threat model out (what happens if a token is stolen?). Generally recommend seconds to minutes for access tokens and longer lived refresh tokens. Also whether you should use token binding or can live with bearer tokens.

> When you get a token from a client, how do you validate that the token is valid?

That will be discussed in part 2 of this series, but in general you should validate the signature (if it is a signed token, often a JWT) or use introspection to ask the Authorization server if it is valid. Then you should validate the claims.

There is definitely more, but thanks for the feedback!

Thanks for the reply, good stuff. Most of this could go into part 2 I think.

For part two I would definitely want at least some discussion around choice of authorization server. I realize a full deep dive is too much, but at least list the options (use a lib, use your own server, use third-party) with some pros and cons or something like that.

If you do go into token generation and usage, and mention JWT, do mention the requirement to verify the alg field against expected value. Just given the issues it's caused in the past.

Anyway, hope I wasn't being too negative, that wasn't my intention.

Hiya,

No worries, I appreciated the feedback. That's one of the magical parts of HN comments.

Unfortunately part 2 is already edited and ready to go and so won't cover the choice of authorization server; it kinda assumes you are using one.

Here's an HN comment I made that covers the dimensions of what to think about: https://news.ycombinator.com/item?id=26411197

I also have collected a series of essays around what to think about w/r/t outsourcing auth (including what kind of auth server to choose): https://leanpub.com/theultimateguidetooutsourcingyourauth . Here's a coupon for 50% off that book: hn-april . Or you can email me and I'll get you a copy; my contact info is in my profile.

Thank for the feedback about JWTs. My general recommendation is to use a well vetted open source library for the signature verification, which should handle the alg check. But calling it out is a good idea.

I do appreciate the effort, and sorry for being a dick with my earlier comment (bad mood). I do feel though that this lacked concrete examples, and maybe should have taken a step or two back to explain the more basic concepts. For example, who owns the authentication service, is it me (the todos developer) or a 3p? What exactly is the difference between all of these different tokens? What’s the actual protocol for issuing and using them? Etc. I think something like a “Todos API with OAuth2 from scratch with code snippets” could have been super helpful for noobs like myself.

Your sentiments and questions are all perfectly valid. I believe the majority of developers implementing oauth2 on their servers or integrating another service they need oauth2 for have not fully understood what they are doing. I know that's the case for me and I successfully implemented solutions for OIDC, saml, three-legged oauth2 and two-legged oauth2 on both the server and client. It's all an overengineered mess.

A "complete guide" to this is still sorely lacking -maybe because it just is that complicated and can not be dumbed down?

I think this is pretty good (I wrote parts of it) but it still lacks everything (mobile use cases, custom scopes, token lifetime): https://fusionauth.io/learn/expert-advice/oauth/modern-guide...

I found these resources helpful:

* Solving Identity Management in Modern Applications" walks through the identity life cycle in detail, from initial provisioning to deprovisioning. Concepts, standards, not a lot of code. Lots of focus on the workforce use cases (rather than customer). But still great: https://link.springer.com/book/10.1007/978-1-4842-5095-2

* "OAuth2 in Action", in contrast, builds an OAuth and OIDC server in JS, from scratch, so has lots and lots of code. Great section on tokens, and covers stuff beyond the standard OAuth grants, such as dynamic client registration. https://www.manning.com/books/oauth-2-in-action

* Podcast episode about OAuth with the author of "OAuth2 in Action" https://www.se-radio.net/2019/08/episode-376-justin-richer-o...

> I do appreciate the effort, and sorry for being a dick with my earlier comment (bad mood).

No worries!

> I think something like a “Todos API with OAuth2 from scratch with code snippets”

Here you are: https://fusionauth.io/learn/expert-advice/oauth/modern-guide... and here's the github repo for the code in it: https://github.com/FusionAuth/fusionauth-example-modern-guid...

That assumes you aren't implementing the Authorization Server (it uses FusionAuth, but should work with other OAuth servers like Identity Server or Keycloak). If you want that experience, then I'd recommend buying this book: "OAuth2 in Action" https://www.manning.com/books/oauth-2-in-action which takes you through all of the nitty gritty. Seriously, you'll implement the whole thing. It's pretty cool.

Maybe calling it an introduction instead of a 'complete guide' might quell some of this.

I sent an email to the editor, we'll see if he can change it. :)

They were able to add a 'part 1' to the title. Phew.

The biggest thing you have left out is one API calling another API which requires on-behalf-of flow. This is an extremely common scenario and yet mostly glossed over until the day you have to do it in a real enterprise to flow the delegated identity.

I have also made an attempt at writing an article for developers that want to implement Oauth2.[1] Not a complete guide, but a short(8min) read that aims to get you started.

[1]https://engineering.intility.com/article/implementing-oauth2...

Am I the only one who thinks Oath is over-complicated bordering to security through obscurity and vendor lock-in ?

The protocol could be as simple as this (using password)

    app => auth-server (can user:pass login to app?)
    auth-server => app (no | yes)

Or using "crypto":

    app => auth-server (can user login to app ?)
    auth-server => app (only if user can solve challange=XXXX with answer=YYYY)

Asking for capabilities:

    app => auth-server (what permissions do user have on app ?)
    auth-server => app (user has access to: X, Y, Z)

First scenario: user's password can be captured and stored by the app.

Second scenario: could work if we're talking about a trusted third party app (e.g. Authy) generating the response. Otherwise presumably you enter your secret into the app, where the OTP is derived. This has the same problem as above.

Third scenario: no authentication required to query for capabilities? Puts on black hat Ok let me quickly check [A-Z]{1,18}@whitehouse.gov and see which accounts I'm most interested in.

Sure, if all you need to know is "can this user login" and you don't have multiple servers that want to authorize the user, you probably don't need OAuth. You can use sessions and cookies. See my comment elsewhere in this post about this: https://news.ycombinator.com/item?id=31007003

Interestingly, your first scenario is the resource owner password credentials grant: https://datatracker.ietf.org/doc/html/rfc6749#section-4.3

The second scenario is, I believe, the JWT bearer grant: https://www.rfc-editor.org/rfc/rfc7523

Of course in the OAuth grants, the auth-server returns a token to be presented to other resources instead of a "yes" or "no" answer.

> In general, use the Authorization Code grant if there is a human being involved and the Client Credentials grant if you are performing server to server communication.

This is the crux of the article.

We're integrating with a vendor now that has imo chosen incorrectly to use auth code instead of client credentials.

This leads to several problems. One is that we need to store their refresh tokens, which go stale after 2 weeks. Since it's theoretically possible that the customer doesn't use the integration for 2 weeks, we need to preemptively use the refresh tokens ourselves behind the scenes every 2 weeks to prevent this. When the vendor changes tech behind the scenes, they just chuck away all existing tokens, so we then need to contact all customers asking them to log in again in order to get new refresh tokens. It sucks and it's brittle as hell.

Tldr; don't use auth code grants (the ones most people think oauth is) for server to server integrations.

Holy crap... either you're someone I work with incognito or this is more common a problem than one would expect....

We hacked together a laughable solution that uses an Event Bridge schedule task to invoke a Lambda that goes out and refreshes the tokens daily, lol.

I've got Opinions and Thoughts on OAuth2 (& all those other protocols, SAML/OIDC)

They all model sharing trust over to untrusted boundaries using signed messages generated by one signing authority. There's always a client, different clients have different properties, they're trying to access a resource, and there is a privileged signatory.

Lots can be learned from their specs, but please don't limit yourself to them, and don't even worry about following the spec accurately (no one really does). Just know WHY the spec is designed the way they are so that you avoid the same security pitfalls. And understand WHAT you're doing: granting signed assertions based on some user interactions

Not only that, but just because it got through the IETF gauntlet doesn't mean they don't have problems. These specs slide around like crazy. My OAuth is different from your OAuth. The antidote? Just write Good Docs and provide good libraries for your integration path. A smart kid in a vacuum solving the same problem OAuth solves will come up with OAuth. Try to emulate that smart kid. And understand why certain concessions are made on the protocols you're taking inspiration from because you might actually not need to make those concessions

This stuff hasn't been fully figured out, so push the needle forward and don't be limited by the past. All protocols are incorrect given a large enough time span.

> A smart kid in a vacuum solving the same problem OAuth solves will come up with OAuth. Try to emulate that smart kid. And understand why certain concessions are made on the protocols you're taking inspiration from because you might actually not need to make those concessions.

I have a different perspective. I work for an auth company now, but before I did, I didn't want to think about this at all.

I wanted to get some code shipped and build differentiating features.

I agree that the IETF and other standards bodies are not perfect (there are specs that get published that never get implemented widely and standards that are implemented that are retrofitted specs).

I do think that building on top of a spec is a great way to "stand on the shoulders of giants", get access to people's experience in use cases you haven't seen yet, and accelerate your development effort.

Also, there are a lot of folks who've written client libraries, docs, and tutorials around OAuth that people can leverage. How many devs will have experience with your "antidote"? I dunno, but my guess is the number rounds to zero.

I'm often frustrated by no clear OAuth-like system for non general auth. Ie you have a server and a clientside JS app. How do you secure it? Cookies are the obvious thing to reach for, but in general i was never clear on exactly what the options are. What are pros and cons of various types of implementations? What are the do's and don'ts? And even if i use something i often felt i was implementing all of it (due to obscure languages choices frequently), but the specifics of the impl were often difficult to find.

It's embarrassing to say but auth has always confused me. And of course managers always instantly say "Just use OAuth" -_-

> you have a server and a clientside JS app. How do you secure it?

I'd love to hear what others have to say, but I'd suggest encrypted session cookies. This is what some frameworks support out of the box (rails has docs here: https://guides.rubyonrails.org/security.html#sessions ). Would store the session in an HttpOnly secure cookie.

If you need to scale your server side horizontally, you'd use redis, a database or something similar for your session store.

This feels like a problem usually solved at the framework level (and has been for years) so maybe that's why there are not as many "best practices" docs around?

This OWASP document may be a useful read too: https://cheatsheetseries.owasp.org/cheatsheets/Session_Manag...

Appreciate the link. I also like encrypted cookies. I've not used them before, but i was really tempted because .. well, they seem an elegant low-cost way to achieve it.. iirc i was a bit unsure how to avoid any pitfalls with the encryption side of things though. Ie i don't want to allow encryption to bypass security because i chose/made a bade encryption method. Really depends on how general purpose the encryption for the cookies are. If it's very general then i suppose i can take any encryption library, but even many of those are notorious for footguns heh.

I would definitely use a framework if at all possible, but symmetric encryption where the key stays on the server side shouldn't be too big a footgun.

A smart kid implementing encryption from scratch could come up with RSA or PKCS or EdDSA on their own. What's more likely is that they will have major holes in their implementation that decades worth of research in the area has already addressed.

Similarly, while it is good to understand the various OAuth2 RFCs and why certain decisions were made, you should also just follow them instead of rolling your own auth exchange mechanism.

it would be great if everyone did "just follow them", but I haven't seen a single spec compliant implementation in all my years

I dunno about anybody else but I simply stopped using oauth2 because I found it made no sense. Things never worked, the constraints on things working had nothing to do with my needs, and the libraries were all out of date and incompatible with each other, then being replaced by new ones that didn't work yet. I also had a sincerely challenging time understanding the security model, and I'm quite comfortable with encyrption, access control, client certs, etc.

I work as OAuth2 implementor, I found this guide to be excellent because it links all the necessary RFC's and categorizes them. It's really easy to navigate and I believe it took a while to write it up, this is some quality work. Thanks for writing it up, it's pretty much BRUTALLY useful for me personally and as a reference guide / explanation guide. Great work, 10/10 will read multiple times :)

Looking forward to part 2!

Can someone explain the increasing usage of Oauth credentials for pure server-to-server applications (like Stripe payment processing)?

I haven't worked with Stripe recently, but if I had to guess, I'd say it is some combination of:

* Time limited (rotation built in).

* If using signed tokens, integrity of contents can be verified.

* Lots of libraries and knowledge around how to use it.

* Tokens allow for more interesting/useful payloads than an API key.

So, as a user trying to access a service protected with OAuth... why am I being forced to use a limited list of providers none of which I might want to use / create an account with (happened 2 times already), rather than just pick one myself ?

Wasn't there an on the making successor to OAuth2 from the same people that was supposed to be a totally different take? I remember hearing something about it but I cannot recall or find the source.

I think you are thinking of GNAP. The doc is still in progress, you can see the 9th draft here: https://datatracker.ietf.org/doc/draft-ietf-gnap-core-protoc... (scroll down).

I looked at it a year ago and wrote up some of the benefits: https://fusionauth.io/blog/2021/01/07/gnap-next-gen-oauth

thanks!

Authentication is just one piece of the "protecting your APIs" puzzle. Other pieces including rate-limiting and billing. Those are more challenging.

Yes, also don't forget securing API credentials properly. As a matter of fact, I have a blog post coming about this topic soon.

Would love to see it. Whether tokens or API keys, I'm always interested in how folks recommend securing them.

There's also some interesting stuff around token binding happening. DPoP is moving towards becoming a standard: https://datatracker.ietf.org/doc/html/draft-ietf-oauth-dpop-...

I was expecting how to implement OAuth2 in pseudo code or JavaScript...

Reply on news.ycombinator.com