Ask HN: Suggestions for books about API design? | Hacker News Reader

Ask HN: Suggestions for books about API design?

298 pointsmcrittenden90 comments

In my case I'm specifically building a REST API with Lambda, but I'd be interested in any API design books you found valuable.

View on HN

Not a book, but the best recorded talk ever given about API design is (IMO) by Joshua Bloch, in a 2007 tech talk at Google. "How to Design a Good API and Why It Matters".

He was one of the creators of the Java Collections API and the Java Executor Framework, two of the most durable and most widely-copied APIs ever. That may not seem relevant for an HTTP/REST API, but it is. The design principles are timeless. Here's that reference:

https://twitter.com/amontalenti/status/1260030567501840386?s...

When I had to design a widely-used "HTTP/RESTful" API a few years back, I wrote a summary of the design principles here. It also includes a reference to an old O'Reilly book on the topic, which I summarized:

https://www.parse.ly/help/api/restful-apis

I really cannot agree with the "just use GET" suggestion. GET must not be used for actions that modify the data server-side. Also, POST and PUT means some action is or is not idempotent.

Using different methods than GET is easy in the browser (easier than JSON-P) and avoids a huge class of problems that come with abusing GET.

Yes, honestly this was written at a time when browsers (specifically, the XHR API) did not support non-GET requests reliably at all (e.g. pre-CORS), so there was no other option. (And once you publish an API, it's hard to change.) You're right that today there is better support for POST/PUT, although there might still be some issues lurking in the corners.

I don't think there's ever been a time browsers haven't reliably supported POST.

In a cross-domain context, when you are expecting the browser to make its XHR for JSON data via JavaScript to a third-party API. Think about making a call to the Flickr.com HTTP API from your own WordPress myblog.com domain. In that context, the XHRs didn't support POST/PUT. Remember, even the fetch() API is pretty recent. POST-based forms are irrelevant for API usage.

This MDN doc covers some of the complexity:

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

And here is when browsers added support for CORS:

https://caniuse.com/#feat=cors

I dimly recall a pre-form web, but it could easily be they just took over a little-used Berners-Lee verb. (For instance, PUT support was atrocious for a while.)

This is not about pre-form web. It's about pre-CORS web. Which is pretty recent web. CORS was "widely & properly" supported by "recently released" browsers around 2012-2013. Of course it's only as of very recently that you could safely assume 2010-2013 browser versions were completely out of circulation.

My bad; hadn't parent'ed enough, and the "using different methods than GET" clarification would not have been anachronistic even in the previous century.

PUT can still be weird. For example, PHP doesn't process multipart/form-data if it's a PUT instead of a POST, I'm working on an API in Laravel and for methods that need to PUT a file I have to use Laravel's _method spoofing.

Agreed about the idempotency concern. Safari for example will sometimes make past GET requests while making a current GET request, so you can get oopsied if you put a non-idempotent request in a GET.

On the other hand, if multi submitting causes bad behavior, a failsafe should probably be baked in to the endpoint logic itself rather than relying on POST not being abused.

I got to take a course on API Design with Joshua Bloch and Charlie Garrod (CMU's 17-480) in the spring. While I don't think the course content is available online, there's a good amount of tactical advice that goes with the talk [1]. The most memorable things to me were an emphasis on user testing to validate your design - we would draft APIs and then interview people and make them try to write code with them; and being willing to go back to the drawing board to change things if necessary. I think these are common ideas in (visual) user interface design, but for some reason are overlooked in coding.

There was an anecdote about how the creator of the Makefile (Stuart Feldman) originally hacked together the implementation so that tabs were required for each command (rather than spaces), and didn't want to change it because dozens of people were already relying on that behavior at this point... [2] seems to confirm this.

[1] http://fwdinnovations.net/whitepaper/APIDesign.pdf

[2] https://beebo.org/haycorn/2015-04-20_tabs-and-makefiles.html

Do you have any tips on how someone can craft persuasive arguments which lead a manager or their team to start seeing API design as valuable?

Really need to couch in terms of "value" to the individuals and org - point to bugs/regressions caused by loose interfaces, copy pasted code, able to use cool tools if you follow REST, JSON-API, GraphQL, etc.

Most folks seem totally unpersuaded by "maybe one day 3d parties will use this API!" hypotheticals.

The hard part about this is learning what individuals and organisations find valuable.

It really depends. Who is the API for?

I agree with the sibling comment that showing concrete failures that were caused by or contributed to by poor API design is effective.

If it's a public (customer-facing) API, perhaps you can compare your version to a competitor's: to achieve the same task, what needs to be done?

Internally... I haven't had a lot of luck here. Push for it in code/design reviews and sneak a refactor into your PRs? One thing that helps is to have ready examples, so you can say "Hey, I think we need a higher level abstraction here or else we'll be copy-pasting around the same 12 calls every time we want to X, just like we already do with Y API." and your teammates hopefully recognize that pain.

Thanks for the link. You might want to fix the missing 'of' in the first paragraph:

This page describes some the principles behind the design of the API.

Will fix! How embarrassing, I think this page has had that typo on it for like 8 years :)

I dunno how much I'd trust a Google speech about api design. Few of their API follow the same rules and most are horrendous to work with. It's the wildwest over there.

I presume you are a bigger API design expert than Joshua Bloch. Please point us to your talk/insights.

Because public speech's make one an expert? How many Google api have you consumed, I'm guessing few, or you'd know what I'm talking about.

A Josh Bloch talk is not a "Google speech"

Actually he was employed by Google when he gave that speech at Google.... So I'd call it a Google speech.

My bad, I misread, it was given at Google. Guess their engineers didn't listen. :x

This talk looks to be a bit dated these days. I wish there were more alike with recent languages as well :-(

Don Norman’s “The Design of Everyday Things” (https://en.m.wikipedia.org/wiki/The_Design_of_Everyday_Thing...). Yes, API design has its own gotchas and specific practices—but some of the best (and worst) APIs I’ve ever interacted with were good (or bad) for the same reasons any kind of design can be good ( or bad): affordances, feedback, coherent conceptual models, and so on.

Yep, I second this. Don Norman really and truly understood the user in the deepest way. It’s hard to find people like him who put their entire consciousness in the shoes of the user, simulate what the user goes through when interacting with the world and provide insights about how, we the designers, on the opposite aisle think about design as not some aesthetic shit but as a tool to solve problems.

Design and designers, vast majority of them are chasing aesthetics and trends. Just go on Behance to see it for yourself.

That's about as accurate a judgment on designers as calling programmers "code monkeys."

Follow a couple designers on Twitter for a few weeks and you're almost guaranteed to see someone quote Steve Jobs saying "Design is not just what it looks like and feels like. Design is how it works."

All designers know that.

To be fair, most programmers are code monkeys. But that's true for most people in any profession. It's not a fair comparison to compare the average worker to workers of excellence.

I came to say exactly the same thing! An API should guide a user to form a mental model that is consistent with the actual inner workings to avoid frustration and making errors (the importance of this cannot be overstated for critical functionality like authentication/authorization).

Isn't that the opposite of the lesson to learn? Design should be sending the correct outward signals and function the way users will expect. Users should not need to understand the inner workings.

Like it or not, users develop a mental model. The fridge example in the book shows how hiding too much of the inner workings can make your system less intuitive to use.

> Like it or not, users develop a mental model.

Agree 100%, but good APIs aren't just a raw interface to a snapshot of how a system happens to be implemented at the time the API was defined.

Agree 100%

I think that's the crux of good design :)

It doesn't mean to unnecessarily expose implementation details, but I take it to be the other way around: implement things in a way that things work consistently without having to know the nuts and bolts.

The lesson from the book is that users will always form a mental model of how the machine works. That's fine even if it's wrong, as long as it works. But when things don't work and users have faulty assumptions about what may be the problem, it becomes hell.

Stripe blog posts are unparalleled

https://stripe.com/blog/api-versioning

https://stripe.com/blog/idempotency

It goes beyond "what is rest" and real life UX of apis

I’d highly recommend reading Roy Fielding’s dissertation paper where he introduced REST.

For anyone who hasn’t read it, it will fundamentally change how you think about REST and APIs for network services, and you’ll create much better APIs armed with that deeper understanding.

Paper: https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

Chapter introducing REST: https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arc...

Do you think it's still relevant for modern client web apps / data backend (I have no doubt it is for websites though).

REST is fundamentally about how do to useful communication between networked services. I'd recommend reading the paper for anyone doing anything related to that. I happen to use GraphQL at work, but that doesn't stop me from learning a lot from the paper.

Can you share any examples of where it changed how you think about REST and APIs?

I went ahead and gave it a read but didn't dig in deeply to get all the context. As such, I feel like some things must have gone right over my head.

His book Designing Hypermedia APIs is also quite good.

https://apisyouwonthate.com and their book(s) offer good advices.

Not a book, but MS REST API Guidelines are pretty good although they don't cover funky important details like how to properly do REST operations/actions.

https://github.com/microsoft/api-guidelines

This is short one based on RedHat VM engineer experience:

https://restful-api-design.readthedocs.io/en/latest/intro.ht...

Manning has a solid books on topic:

https://www.manning.com/books/the-design-of-web-apis

https://www.manning.com/books/api-design-patterns

I've found https://aip.dev really useful! In fact this is a canonical resource for building REST apis at Google.

I would suggest having a close look at the OpenAPI specification as a way to bound and constrain your REST API design: https://swagger.io/specification/

Once you start thinking in terms of requests, responses, parameters, schemas etc the rest should easily fall into place! You also get the benefit of having a machine readable definition of your API if you write your design with the spec.

About REST : REST API Design Rulebook by Mark Massé (O'Reilly).

My reference when I'm designing a REST API.

https://www.oreilly.com/library/view/rest-api-design/9781449...

Agreed. Simple and clean, with an approach that makes for consistency.

Domain Driven Design: https://smile.amazon.com/Domain-Driven-Design-Tackling-Compl...

It's not about designing APIs, it's about designing your software to reflect the reality you're encoding into your software. It's a book of first principles that you can apply to any software you design.

Not a book, but the current issue of https://increment.com/ is about APIs.

I hadn't heard of Increment before. They have some very interesting articles! I like how they group a number of articles from different authors on a related topic. Thanks for sharing.

A philosophy of software design - John Ousterhout

This has an interesting generic discussion of API design which is technology agnostic.

The most influential technical book I have read over the past 10 years for sure. The author guides the reader though the process of managing complexity with some intriguing concepts such as "Define errors out of existence" and "Deep modules". I definitely recommend it.

I found JSON:API to be a great starting point, especially for small teams that want to get started right away. You can either adopt it or use it as a guideline for all the topics you need to consider.

https://jsonapi.org/format/

The Little Manual of API design

You can read it in an hour or two. It cover a lot of things that you should consider

Here's the link for reference :) https://members.loria.fr/JCBlanchette/api-design.pdf

Which one? There is 20 of them.

I was looking for this specific title in the suggestions, as I got this recommended myself a few years back. If we have the same little manual in mind, it's by Jasmin Blanchette. I can't say anything about the other 19.

Check out Google AIP

https://google.aip.dev/general

Do people still design rest-ful APIs? I often find I end up needing to do RPC-like things that make the API end up as a weird hybrid. I'm on the verge of abandoning the idea of trying to be rest-ful at all, so it's less incohesive.

One of the things we insisted on building our internal ML platform was that every functionality should be accessible through an API.

This is scar tissue from our many past projects that delivered value to clients, but could not easily be leveraged as building blocks to do something else, especially when you have different stacks.

So now, we always strive to make everything that the platform can do possible through an API call, from scheduling a training job, to deploying a model, or invoking functionality from dynamically loaded applications, which also expose their API.

"Anything you can do with point and click should be accessible with an API call." guides us.

Do people still design rest-ful APIs?

For internal software, you use whatever works.

For the public API of a major service? Yes, REST is definitely still around. REST or at least REST-like APIs are still almost universal in my experience.

Surprising question, as I think REST is probably the leading paradigm for public APIs. At least in terms of what people call their API. What RESTful actually is varies from person to person.

Honestly, prefer GraphQL -- it's more like designing a schema which you may already be doing in your database anyway. The client decides which data in what shape it fetches.

If you're designing an internal API to power a rich client app, yes. If you're designing an API for developers to interface with you SaaS, plain old HTTP is a better choice, even if it's only RESTful in a hand wavy sense.

Yeah, agree - if your product is an api, I'd have REST first, then graphql later perhaps if people are interested in that.

You are right - most API's are RPCish and RESTish at the same time. Apparently REST core is not enough.

I don't have any book recommendations to make since API design is more art than science. As such I'd recommend that you beta test the API (route names, payloads, etc) with a few potential users to gauge durability and the ease with which users can intuit. The last thing you want is to release new version quickly or to require users to always refer to your documentation.

Some of these problems can be solved with an SDK (highly recommend, as it can greatly improve developer experience especially in a modern text editor or IDE) or switching to GraphQL (not without its own problems).

If you use gRPC, I think you can see Google API Design [1] useful.

[1] https://cloud.google.com/apis/design

Consumer centric API design

https://github.com/tlhunter/consumer-centric-api-design

I just read through a good chunk of this, hoping it'd be a good resource.

There's a lot of good stuff in here:

- Higher level than a spec, but still detailed.

- He outlines the anatomy of an HTTP request and gives examples.

- Lots of good suggestions of conventions and rules of thumb, e.g. for URL structure and naming.

- Good demonstration of why using fitting HTTP verbs is important. Explains idempotent and safe.

On the flip side, his ethos isn't the best at points, he presents some subjective takes as fact, and even includes some plain errors against the specification.

For example:

- JSON keys should be snake cased, or camelCased "if you're one of _those_ people". (I'm a snake case guy, but really?)

- Return a 400 if there are any problems with the request. (400 explicitly means request body can't be parsed, e.g. syntax error in your JSON. The spec isn't perfect, but 422 is often more fitting if the consumer input didn't pass validation. Either way 400 clearly violates the spec.)

- He misses the nuance of what PUT is in the spec. You're saying "store this resource at this URL". Not technically update. Although in practice it's usually update, and I'm sure this point will be viewed as pedantic by many.

Overall a decent resource, but take it with a grain of salt.

I read this book for a public API project I was working on. At the time I had done several Rest APIs but for internal use. When it came time for a public API, you really need to up your game. A public API says a lot about a company. It's essentially marketing and sales plus a way to interact with your company programmatically. This book covers all your basics of Rest API design.

Another topic to read up on intent API design (intent resources). If you stick to nouns as your resources, you will encounter issues with naming APIs that have several steps or are really executions of business processes/procedures. With intent resources, you can continue to use the noun paradigm for naming your APIs but still be able to have complicated APIs (more than just simple CRUD).

This book is really good: https://www.manning.com/books/the-design-of-web-apis

It even spends the first few chapters talking about API design in general (not just for REST) with really great, intuitive examples.

Please check "The C Interfaces and Implementations" book by David Hanson [1]. Someone has implemented the concept using his library that you can check and use [2].

Another excellent reference is the "The Linux Programming Interface" book by Michael Kerrisk that documents most of the API available under Linux [3].

[1]https://www.amazon.com/dp/0201498413

[2]https://github.com/gon1332/gonlibs

[3]https://www.amazon.com/Linux-Programming-Interface-System-Ha...

API design in C++ by Martin Reddy is a good text.

Agree. It is very good, but I think thee things he is concerned with are very c++ specific.

Maybe Lauret's The Design of Web APIs? [0]

[0]: https://www.manning.com/books/the-design-of-web-apis

Ebook from APIGee is pretty good, at least for me.

https://pages.apigee.com/web-api-design-register.html

Great recommendations! For another perspective, I'd add O'Reilly's "Continuous API Management" with some chapters around designing APIs as products and suggestions around how to implement a lifecycle. Also, strictly for design I'd also do what was suggested here and check API design guidelines from several sources. I like the Adidas one.

I really liked: The REST API Design Handbook by George Reese.

It is everything you need to know and thinks about the angle of the people who need to consume your API. Quick read too.

https://www.amazon.com/REST-API-Design-Handbook-ebook/dp/B00...

I analyzed 24 blog posts on API design (rest/graphql/rpc) and compiled a list of the books they recommended (sorted by the most popular ones first) https://www.readthistwice.com/lists/best-api-design-books.

Ignore the fact that this is a book about .NET, it's an excellent source of wisdom about the design of APIs:

https://www.amazon.com/dp/0321545613/ref=cm_sw_r_cp_apa_i_wn...

Not a book, but the best advice on API design I've ever seen was in 2004 by Casey Muratori on his talk "Designing and Evaluating Reusable Components".

https://www.youtube.com/watch?v=ZQ5_u8Lgvyk

Not a book, but this is a great talk about [How to Design Great APIs - Parse Developer Day 2013](https://www.youtube.com/watch?v=qCdpTji8nxo)

I like Framework Design Guidelines by Cwalina, Barton, and Abrams. The 3rd edition was recently released.

It's targeted at developing .NET frameworks with a broad audience so some of the guidelines aren't appropriate for internal APIs or applications.

Casey Muratori (Rad Game Tools, Handmade Hero) talk from 2004 covers reusability and design for ease of use.

https://caseymuratori.com/blog_0024

I wrote a popular blog post on this one:

https://www.vinaysahni.com/best-practices-for-a-pragmatic-re...

Moesif has many free resources for API design, development, and maintenence.

https://www.moesif.com/blog/api-guide/

https://github.com/microsoft/api-guidelines

REST API guidelines from Microsoft

for REST APIs, https://apisyouwonthate.com/books/build-apis-you-wont-hate book is my recommendation if you haven't read it already. I read it as a beginner, so it was kind of an eye opener for me on why we should do things a certain way when it comes to API.

Google published their guidelines for designing APIs [0]

[0] https://cloud.google.com/apis/design

Sure, read some books.

But know that they're all wrong. Some fatally.

The best you can do, ever, is to be consistent. So pick a style and stick to it. Honor the principle of least astonishment.

Reply on news.ycombinator.com