* How do they handle deprecation?
At some point, they're going to realize that their API design is wrong, could be better, or could take advantage of new patterns, methods, or even customer usage patterns. As a result, they're going to have to restructure and potentially deprecate things. Do they handle it with transparency, honesty or even just make sure your stuff still works?
At Twilio, the 2008 endpoints have been deprecated for 5+ years but still work and - to the best of my knowledge - there's no sunset announced. Yes, all the examples, docs, sdks, etc encourage you to use the new endpoints but the old ones are there. Compare that to Facebook, Google, Twitter, etc which turn on and off APIs and break things with little regard for end users. Which leads to the second question..
* What's their business model?
If they make more money as you use the platform more, that's great! In biz speak, your interests are aligned! They're going to make sure you get started as quickly as possible, minimize downtime, and make sure that your stuff works now and forever. Once again, that's the Twilio model.
If you scale from 1 to 10 to 1M requests/day and they make the same amount of money (or none in many cases), then they're going to be less concerned if you can't make 1, 5, or even 10% of those calls. Even worse, if the API itself is primarily used for market penetration purposes (once again: Twitter), they may treat you as a free R&D group and let you do the exploration and prove the market before they step in.
* Early Twilio employee, current Okta employee, wrote a book on API Design
Examples:
* A REST API needs 3 calls for $foobar. A GraphQL API only needs 1 call. The vendor will loose money if they optimize their API in a way that you can make less calls.
* Some of your data does not frequently change. If you pay per request, you can save money by caching. However, that'll make your implementation more complex (esp. if the API is supposed to be used directly from mobile apps).
IMO it's better if the API provider can charge based on business metrics. E.g. Stripe charges based on payment volumne. It doesn't matter if you do 2 or 200 API calls to make a payment.
1. Do they provide an easy way to create a sandbox/staging/test/dev environment ? Good example: Stripe. Really bad example: Paypal. Why ? Well, for starters, you need to create a completely separate account just to get a sandbox env in paypal. Then there is no guarantee that sandbox is identical to production in terms of ability to build and test.
2. What are their Rate Limits ? Some APIs are well designed but have terrible rate limits.
3. Authentication mechanism and security. How do they authenticate the API access ? This can matter for building critical apps.
4. Do they use REST or not ?
5. Do they have good documentation ? Can you do a quick test using their documentation or is it required to call their support ?
6. Do they have good support ?
7. Do they version their API changes and are those changes backward compatible ?
I am sure there are more but these are some good ones to start with.
Network and system hiccups/outages cause non-trivial error rates with non-trivial load, so understanding how to recover from those errors is critical (to your ability to sleep through the night because your on-call pager stayed silent).
But it would still be reassuring to see this stated explicitly!
In the real world even bigger players such as AdWords would time out on our bulk requests to them if making >100M changes, without giving any errors back (the jobs just simply tend to disappear; or the jobs never get submitted in the first place).
Truth is, you may or may not be discussing the appeal of the REST; but the issue in this context is solved already - and it’s only a matter of professionalism and therefore, the resources required to build something that just works.
But in the real world we get stoned students writing the majority of the APIs; having read a few how-to’s.
(Yes, it can be done programmatically, but that's not always feasible/practical, especially if I want to keep certain customer accounts alive.)
Similarly, after deleting a customer, it takes you back to the customer list, but it's not always up-to-date. Sometimes the front-end can detect that updates have occurred in the list (i.e. something has been deleted) but it requires a manual refresh.
Meh. REST semantics really only works well when you're doing CRUD operations. They get clunky when something's modeled around actions rather than entities. That said, make sure their response codes and methods make sense.
If you expect that you might need to significantly change what you're doing in the future, having a more simple, composable model around resources that you can call as you need might save your ass down the line compared to if they model things as specific RPC use cases and you find yourself needing a particular variation they don't support out of the box.
Sigh, xml hard at work solving a problem we never seen, but creates such a bloted horrible mess where json would have been simple and better.
The downside is it has to be pretty complicated to enable the magic, so building it can be complicated, and debugging it is a nightmare. You could say the same about a lot of modern JS, and that's been somewhat solved by amazing tooling.
In my data oriented mind, the primary thing that matters is the data that is sent, the second is other semantics like idempotency.
Neither SOAP nor XML solve or helps: there might be no wsdl bindings in your language, or they are badly implemented (true in my experience).
XML make people nest structures where it could have been flatter, SOAP even more, and annotates everything whithout a need for that.
This cost for no reason but to entertain the CORBA like pipe dream, or should I say nightmare.
We just resorted to pulling out data from xml with xpath. And more or less template the response back, no wsdl and very simple.
But why inflict this pain on your API users?
please think through your API if you are to design an API, the tools will not think for you and you can fool yourself instead...
Swagger will not magically solve anything. I have seen and experienced that you can really create nonsensical but pretty and auto documented APIs using swagger in no time.
There are (at least) three senses of "REST" in use in the development world, and none of them are important in an API.
The first is the original definition, laid out in Fielding's thesis. (Normally I call this "the correct definition".) This includes requiring HATEOS, which is far from a necessary property in a good API, and arguably a bad feature because it increases payload sizes.
The next is the definition popularized by Rails [1]. This is never precisely outlined anywhere as far as I know, but most people seem to think it requires using URLs with particular patterns (to see an example, just do `rake routes` on any Rails project that doesn't use weird custom routing in routes.rb). Also most people seem to think it requires using PUT for overwriting updates and DELETE for deletions. All of these conventions are not-bad. But they're also not particularly good; they amount to a variable naming convention that happens to be pretty reasonable. Other systems of organization are also perfectly good! But anyway, this is not an important feature in an API.
The third definition you'll see sometimes is "our API uses HTTP, therefore it's REST, we're buzzword-compliant". Fuck your buzzword compliance, no that doesn't make your API RESTful. And in some cases, these bozos can't even use HTTP correctly (for example using GETs for side-effect-ful requests). This is pretty much indefensible, but it'll still happen. So if you DO care about one of the first two definitions, don't take it on faith that someone who says "our API is RESTful" knows what that even means.
Also, the first two definitions both work in terms of "representations of resources", and all three of the above apply to HTTP [2]. It's not at all clear that either of these are good features in an API. They both have the advantage that they make implementation more predictable, so that's often (usually? almost always?) a great idea. Yay! But resource-representation is a bit bloaty, and so is HTTP, so for some situations, it's more appropriate to discard either or both of those design parameters. For example, suppose you're monitoring some terrifyingly dense IoT deployment, and you want your swarm of sensors to send updates ten times a second. Differential updates over Apache Thrift (or over Google Protobuff) is certainly not over HTTP, and you have to squint pretty hard to to see the differentials as resources. (Mind you, you'd probably also want to send some checkpoints less-frequently, and you could characterize these as resources without too much difficulty.)
In summary: REST means different things to different people, and there's a lot of good sense in those communities (I mean, not the third-listed liars, but the other two). BUT, probably you don't actually care about RESTfulness per se, in that cute conventional URI/path patterns are unimportant, and HATEOS is somewhere between nice-to-have and undesirable.
HTH
Footnote 1: I'm not claiming that authors of Rails (DHH et al) were confused about the definition of REST. As far as I can tell, when they said "make your routes RESTful", they weren't claiming that doing so necessitated using their cute method/URI convention, they were simply claiming that doing so was compatible, and that following some convention is a good idea, and here's a good convention. Yay! Well said! Unfortunately most people who wrote simplifying articles on the subject got it wrong, so now we have an industry full of semi-literates quoting non-existent scripture.
Footnote 2: Strictly speaking Fielding's thesis is about what he thinks HTTP did well, and is intended to be largely advice to designers of future protocols intended to compete with HTTP. That's why so much of the "architectural style" is about stuff that's built into HTTP, such that the designer of an HTTP API cannot fuck it up. API authors are not the primary intended audience of the thesis!
Agree with codegeek about sandboxes. With some vendors it's hard to test your code because their sandbox doesn't look like their live service, or hard to come up with instructions in those cases where you need to ask your customers to interact with a vendor. Some vendors make our tests that integrate against them flaky because the sandbox is flaky :/
Look carefully for things that smell either like too little attention to detail or great attention to detail. For example, for a while a big payment gateway had, basically, a webhook for telling you about payments/payment failures on recurring transactions, but there was no retry mechanism, so if a connection was lost for any reason, so was your data. At the other extreme, it's a good sign if they go above and beyond in some areas, e.g. Twilio and Stripe have great documentation, their approaches to deprecation/compatibility are good, and Stripe offering the idempotency-key mechanism showed they'd spent time solving the whole not-charging-people-twice-because-of-an-inconveniently-timed-network-glitch thing.
This is probably a quirky personal preference, but I'm encouraged when vendors offer APIs for single-item and bulk operations and the bulk ops perform OK with a lot of items. We have some generally alright vendors whose APIs turn out to be painfully slow when we sync data for our largest customers back to our servers. This is a problem that would not necessarily have shown up from testing a high request rate; it's a matter of scaling well with amount of data _stored_.
Some services have "old" and "new" APIs that are not fully integrated, e.g. auth credentials needed and object IDs might differ between them. Sometimes you have to use both to get all of the functionality you want. This hasn't tended to correlate with a good developer experience. On the other hand needing to make your new API distinct is probably inevitable sometimes due to business history or other practicalities--I wouldn't make a decision just based on that.
Comprehensible pricing is a good sign, though of course that's more achievable some times than others. Something like AWS or GCP has accumulated a lot of cost/pricing details to sort through since there are a lot of types of resources they offer, and there aren't really shortcuts to get around that.
Wouldn't that be a 206?
There is 207, I think created by WebDAV, where you give back an XML document detailing the status of sub-requests. I feel like that'd be confusing in many contexts as well, and a clearly defined custom response is probably easier to handle for the user.
Error response? 200 and a human-readable string describing the error.
- Request size limitations (implicit and explicit, such as only allowing GET and being limited by what can be URL encoded)
- Response size limits, especially for reporting or data retrieval APIs.
- Concurrency limits in addition to rate limiting (e.g. Requests per second limit AND number of queued requests limit)
- SLA have been talked about, but you always need to quadrupole check this and your contract outs based on number of incidents, etc.
- Consider upstream/downstream dependencies, for example Twilio itself is not likely to go down, but one of the regional telcos/carriers/rate centers are. What options do you have then? (Twilio allows SIP trunks from other carriers to be connected for example.)
This is a terrible assumption. This is one of the things that differentiates vendors. No two vendors are the same, especially things like staging environments, capacity limits, etc.
But other things are:
1) Do they have a real-time status page?
2) How many outages have they had in the last year, and how severe were they?
3) Where is their support based? And is it 24x7? What if you have an issue at 2am on Saturday morning, do they have support?
4) How attentive are they to fixing bugs?
I just looked back at some old web projects from about 6-7 years ago that I'd mostly forgotten about, to see if anything was worth keeping. Every last one that called out to an API is now non-functional. Oh well.
2.) Documentation quality. Can you read it and understand? Does it contain details and examples or rather fuzzy generalities?
3.) Do the company seem stable? How likely are they to disappear (or get into trouble) three years later?
If this API is mission-critical, write drop-in replacement drivers for other vendors and/or get a commitment from them in writing.
How essential will any given vendor be to your biz model? If they are acquired, say by a competitor, where will that leave you?
How might any of these "relationships" effect your exit strategy? That is, what you find palatable and necessary your suitor might not.
* Ability to customize / extend
I'm assuming your API is targeted at a business domain (vs. an infrastructure API like an email API). The API vendor will do his best to cover all major use cases within that business domain. But expect that your project has a minor use case, or is the first project to ever need feature X. The vendor most likely won't want to implement this feature. (This isn't necessarily bad - in fact it is good, if the vendor would add any random feature to his API, it would evolve into a frankenstein that is a nightmare to work with)
However, the vendor should have features and patterns/best-practices on how to solve those cases. E.g. in addition to the API vendors data model, you should be able to add your own data (like https://stripe.com/docs/api#metadata) and you should be able to react to events to add custom behaviour (e.g. https://stripe.com/docs/webhooks).
* Quality of documentation and support
A lacking documentation, especially in combination with bad support, can slow down development significantly. Some APIs also have a good community outside of the vendor (e.g. it's active on Stackoverflow), take that into consideration as well.
Also, are the POs and Devs active in support? If yes, this is a good sign IMO, because they're trying to stay in touch with their customers. And it'll also raise the quality level of the support, because - if you run into trouble - you can escalate to someone who really knows what is going on.
* Good SDKs in your language
Some API vendors offer great SDKs. They can be a huge timesaver, e.g. if they provide models for all their endpoints, they provide additional helper methods ontop of the API, and they convert data-types into language-native objects (e.g. Money isn't a native data-type of JSON, but of most languages). It is also valuable if the SDKs (and docs) come with lots examples in your language.
Some SDKs are okay-ish, especially if generated from Swagger/Raml. Some generated ones are a bit strange and it may be better to work directly with the API if it let's you craft your models in the way you need them to.
Some SDKs are a nightmare. We had one that leaked threads and killed our servers after a couple hours... not fun!
* Development workflow
As a dev, how can you use the API for testing? On staging? Does this incur additional costs?
You may need to reproduce a bug from your production API. It's helpful if you can copy the production data into your dev project easily.
(I'm an API dev @ commercetools)