Comment by nyrikki | Hacker News Reader

Comment by nyrikki | Hacker News Reader

Parent Full thread

I disagree, those are all technical issues that are typically blocked by socio-technical and organizational problems.

Retrofitting an API is a context specific problem, with few things that generalize but the above aren't the hard parts, just the complex ones.

I am going to point you to a Public Policy book here, look at page 32 and their concept of 'wicked problems', which applies to tech but doesn't suffer the problems with the consulting industry co-opting as much as in the tech world.

https://library.oapen.org/bitstream/id/f3358d25-56c0-47f2-90...

Obviously most companies won't accept the Amazon style API edicts that require Externalizable interfaces and force a product mindset and an outside in view.

Obviously an API is still a "Cognitively complex problem" as defined by the above.

But what makes most API projects fail is the fact that aspects arise that result in ‘wickedness’ A.K.A intractability.

API gateways, WAFs, RESTful libs with exponential backoff with jitter....etc... all exist and work well and almost any system that you have a single DB you are concerned about killing is possible to just add a anti-corruption layer if you don't have too much code debt, complicated centralized orchestration etc....

But these general, but difficult tasks like adding an API almost never fail due to tech reasons, but due to politics, poor communication, focusing on tech and not users, unrealistic timelines, and turf battles.

You are correct that "APIs require thorough documentation or they're useless" but more importantly they need enforceable standards on contract ownership, communication.

The direction of control, stability, audience, and a dozen other factors effect the tradeoffs there as to what is appropriate.

Typically that control is dictated purely by politics and not focused on outcomes.

That is often what pushes these complex problems into failed initiatives, and obviously this is far more complicated. Often orgs can't deal with the very real uncertainty in these efforts and waste lots of project time on high effort, low value tasks like producing gantt charts that are so beyond the planning horizon that they can only result in bad outcomes at best.

The point being is that unless you are on the frontier of knowledge and technical capabilities, it is almost never the tech that causes these efforts to fail.

When I read OP’s list of wants for an API, I don’t read it so literally.

I read it as an example of the complexity required to implement something like an API.

I agree that it’s also a people problem but I disagree that many of these failed initiatives aren’t due to bad tech as well. A complex initiative requires picking the right tech that gives you a good benefit/cost ratio. Knowledge of tech plays hand in hand with the political aspects and IMO many orgs lack both completely.

> A complex initiative requires picking the right tech that gives you a good benefit/cost ratio.

That choice should never be a one way door, you don't control your customers or the future, it is the frame problem.

The idea is to be able to iterate and adapt, not get things perfect from the start.

It is fine and expected that one will use past experience as a starting point, but you always have to reflect and check your assumptions along the way.

It is a complicated topic ruled by nuances and contexts, but if you cannot pivot away from your initial assumptions over time, that lets you know you are leaking implemention details.

While time is limited and abstractions have a real cost, you need to apply them where they make sense.

Some, like just breaking code onto two files in the same directory are fairly low cost, others are much more expensive.

For several years I was jumping in to save failed cloud migrations.

Whenever tech was the 'blocker' it was because of a myth that there was only one way to do things.

But lets say you are writing machine code for Apple silicon? Do you really think those registers are concrete? They aren't. They are a facade hiding 100s behind a legacy interface.

Vendor mitigation important and often neglected and also results in people producing balls of mud.

But there are a lot of distributed monoliths pretending to be micro services out there. And we had decades of people producing fragile enterprise service buses that were built because people wanted to future proof their systems.

The sizing of components and balancing integration and disintegration drivers is incredibly hard, you will never get it right.

You have to leave options open, no matter if that is through abstractions or keeping biz logic centralized to assist in a easy rewrite of context boundaries that arise from scale or changing needs.

Obviously choices have benefits and costs, but for most needs you can keep options open, if not it is probably best to reevaluate the reason for choosing a solution.

Obviously there are predatory vendors like Oracle that base their entire income on captive customers, but that relates to the vendor mitigation above.

I think the US federal government de-risking guide is a good overview of that topic. Microsoft is a strong driver for this BTW.

https://guides.18f.gov/derisking-government-tech/

For anyone who wants to read more about what the Amazon experience was like, I highly recommend Steve Yegge's platform rant at https://gist.github.com/chitchcock/1281611.

It’s depressing how far Google seems to have fallen since then.

Reply on news.ycombinator.com