The Light and Dark Side of the API Economy

> Hundreds of millions of people use Excel. What if all the APIs we just talked about were accessible to regular folks, or "Citizen Developers"?

I've been living this scenario recently, leading development of an integration with Okta and SCIM and it has been pure, unabated hell. One of the most insanely frustrating projects I've ever worked on, hands down.

Okta picks and chooses which aspects of the SCIM standard to implement, and intentionally deviates from the standard in some cases. To be fair, all of the major identity providers do this, Okta isn't the only offender.

They don't provide any kind of comprehensive test suite to validate compatibility with third-party SCIM APIs, and because you're interacting with a GUI and Okta's internal system log, which does not provide access to the HTTP layer, things fail in the strangest, most opaque ways. Their backend behaves very differently depending on which kind of template app you start out with, and they don't document clearly how this behavior differs (probably because it would be really embarrassing to put something like that on the record).

Slapping a GUI on top of these HTTP interactions has been an absolute abomination to work with and I'm convinced that "no-code" is total scam as long as the actual developer experience isn't considered, which is why open source tooling largely took off in the first place. Companies that aren't in the developer tools market don't give a shit about developer experience.

The odd thing is that a public API is just as often an attempt to take something away as opposed to give something.

This one is really good

https://metmuseum.github.io/

but the first result on bing is this spam page (boy the "API Economy" attracts spam)

https://www.programmableweb.com/api/metropolitan-museum-art-...

The Met has a generous API limit, 80 requests per second! Usually though an API exists to make you suffer, register, deal with API limits and get access to less data than there is on the website. Around that hover a large number of "integration" providers that boast about the quantity of "integrations" that they have (table stakes are "hundreds") but say nothing about the quality.

There's nothing like having a complete copy of the database or at least a complete summary. There are many things that are obvious when you have a mirror of the whole thing but that's you'll struggle with like a bubble under the rug if you are forced to look at things a record at a time. One of the reasons why the semantic web idea of dereferencing URLs has been at best a partial success is that randomly poking at a data set means you're likely to miss what's important and get stuck in trivia.

I downloaded all the metadata for the met and that lets me ask questions like "show me all the images with an aspect ratio between 0.98 and 1.02".

I am pretty skeptical that no-code will be the major savior of folks who don't know how to develop, though I suspect more and more folks will do something similar, just like people in many professions learned word processing and spreadsheets. My SO has set up a few Zapier zaps to automate workflows for her writing business.

Other options I see:

  * learn software development or a tech adjacent skill.
  * branding. Become the X of choice (hairdresser, massage therapist, etc) and build up a following of folks who want to transact with you, not with any X. Shopify/square can assist you in being on top of the API (in this case, the search API).
  * pick a profession not amenable to being API-ized. Handyman, plumber, etc. I think these jobs are too bespoke to be worth turning into an API.
  * be on top of the API in other ways than telling devs what to do. Sales will never be turned into an API, though there may be excesses squeezed out of it.
  * accept the tradeoff of being below the API in exchange for the autonomy (see this piece by Bill Gurley for more on that: https://abovethecrowd.com/2018/04/19/the-thing-i-love-most-about-uber/ )

Some of these things are new, but some have been around for years and decades. No one HAD to buy a server - co-location/bare-metal hosting has existed for years before AWS. The same for printing. Stripe perhaps made the biggest difference, mostly because this field is more tightly regulated so there is more value to a one-stop-shop solution.

While they might provide "a" solution, those "API" services also don't solve provide a complete/optimal solution (You still need to understand information retrieval to make your search work well), and they sometimes create new problems (as demonstrated by the plethora of cloud consultants).

For non-core activities it could be fine, but for activities the affect the product itself (E.g., search is essential for e-commerce), outsourcing all responsibility to a vendor is risky.