Comment by Waterluvian | Hacker News Reader

Comment by Waterluvian | Hacker News Reader

Parent Full thread

I don't think there's one right answer, but my opinion is:

- Treat APIs as immutable.

- Any mutation results in a wholly new API version, not a patch or minor update.

- The developer will learn what changed, and how much changed by reading the patchnotes, not looking at which semver numbers changed. This is probably a healthy practice to encourage.

- Don't change APIs so much. The interface should be very carefully designed and tested and shipped like an NES cartridge: consider it impossible to fix once its shipped.

Therefore just start with `1` and increment each time.

The reason I don't like semver is that it's a developer convenience that leads to sloppy practices. You built your product against a specific API. If the API changes, you need to re-run your entire API evaluation, testing, blessing workflow. If the delta is tiny (what would have been a patch change) then yay, your task is likely going to be very simple. But you shouldn't see a bump version update and decide you can cut corners.

I don’t disagree with your assessment on semver as it currently exists for, say, NPM packages.

I do think that with web APIs specifically, the surface area is a lot smaller — the HTTP interface is literally all you touch — so semver, in its purest form, is actually completely enforceable as long as you understand the API schema.

Our team has talked a lot about either hard-enforcing or automatically applying semver where applicable. My question to you is — does this sound reasonable, and if you knew a semver contract was actually bound to implementation (i.e. guaranteed and not implied), would you trust it more?

Tbh, it doesn't matter. But the semantics should be such that it doesn't matter -- the user of the API shouldn't care whether it's implementation bounded or not. But Everytime you break that abstraction, trust in the abstraction is necessarily reduced.

Bounding to implementation is just the easiest way out -- if your policies, tests and protocols consistently fail to uphold that abstraction, then the you can fallback to this very simple (presumably innefficient) strategy to do so.

But I, as a dev, just want a stable API, and I don't care how it's done.

How would your trust be impacted if there was better communication ahead of a breaking change you could almost agree with? i.e. the experience has been well beyond that which you currently expect and have been provided good reasons proactively.

The gap I see a lot starts with a bad versioning strategy and continues through bad API analytics to bad communications for developers.

The API provider should be able to understand all of the individual clients to the point where they can decide to progressively migrate them one-by-one, if necessary. There shouldn't be a need to use metadata like IP addresses or user-agents to try and identify the clients. Unless someone has walked through these scenarios ahead of time, it's difficult manage when issues arise.

I don't really trust semver insofar as there's nothing to trust, in my opinion. What am I trusting? That I don't have to understand what changed and can take semver's word for it? To me, personally, that's an abdication of my responsibility to know what I'm utilizing and implementing against.

What's the practical value you're hoping to gain by hard-enforcing semver, especially if the surface area of HTTP APIs is a lot smaller? And this isn't a loaded question. I might just not be privy to your team's needs.

Also consider (and this again falls within the realm of personal opinion) the value of achieving _maximum possible statelessness_ as a developer. If I'm reading you right, you're asking me to know and remember, and probably write down somewhere, that your API's versioning means something different than others. So I now have another branching path to maintain: how to behave when your API changes. What I like about the simple numbered API schema is that I can come back to the API usage in a stateless manner. I just see that the number is different and I know what to do: learn what changed and decide if I have to act on it. Of course I could do this by looking at a semver as a unique identifier without semantic meaning, but that kind of leads to my initial point: encourage best practice by not giving the developer a semantic shortcut to which they can abdicate responsibility.

I think you misunderstand where I’m coming from with my line of questioning.

I’m asking questions because we provide tools that standardize the API development experience. Semver is trustable if the tools create the contract instead of the developer that decides to build the API. We deliver the tools that create the contract.

In this sense, APIs built and delivered atop our platform don’t need to “encourage best practice,” best practice is — or at least can be — hardcoded in with no potential for footguns.

Your concerns about semver are well-founded. I’m asking questions because I’m pondering aloud if we can fix them and change semver — with respect to APIs — from a social contract to a coded contract, and include tools necessary to inform end-users of API changes. :)

Does that make more sense?

Yeah it does. Thanks for clarifying. I think there's a very complex conversation to be had here, especially since you're working on a meta-level of tooling to ease API development.

With two days before my first vacation of the year, my brain is a pile of mush. I'm going to politely and apologetically bow out of this conversation. =)

I don't think semver is easily enforceable. Take for example the discussion in this thread about stripe changing the order in which hooks fired, which completely changed the meaning of a null value in the next_payment_attempt field. Just looking at a traditional API schema wouldn't reveal any change, yet this was a major breaking change to some. Maybe you can create an API schema which encompasses the order in which hooks fire, but that still doesn't help if somebody changes the semantics of a field without changing its name or type.

What does a codebase look like with these API versioning exactly?

I feel like code bloat would be hard to maintain. Obviously I could take two endpoint handlers that have the same functionality and move their behavior into a shared function, but there's also tests to consider. As time went on, I'd have a pile of tests running against seemingly random versions.

Eg, `test_blog_create()` might test v1 and v2 of the API as that specific endpoint didn't change, but suddenly in v3 the API _does_ change, so now I need to write a slightly different test to handle that new functionality. `test_blog_create_v3()` or w/e.

I'm not arguing against it, merely noting what I've previously thought about as problematic for implementing versioned APIs.

Thoughts on how best to manage the code base?

Versioning is hard. Versioned APIs are very hard. I'm in an especially tricky spot because we've got customers that might refuse to upgrade our product for years and years, if ever.

Two approaches I've taken, with a blogpost worth of thoughts on them:

1. Every version is a whole new database/webserver stack. You shut the oldest one down when the last customer migrates off it. The bloat is real, but it's dead simple and it works.

2. A single database with a webserver that exposes all the versions. The contract is that the database shall not undergo destructive migrations. Only additive migrations. Or if you must, modify schema in a way that you can deterministically generate every previous API's view. Again, discard older versions when the last customer has been migrated off.

I'm sure you can immediately smell some problems with either of these approaches. Sacrifices are definitely made.

Every version is a whole new database/webserver stack. You shut the oldest one down when the last customer migrates off it. The bloat is real, but it's dead simple and it works.

That means data is different between API versions?

Yes. But that works. Becuase a customer's data lives in only one version. When the customer upgrades their data gets migrated. Weird. Dead simple. Works.

There is definitely multiple ways to handle that, but splitting your API into frontend and backend seems to be a common and good approach. The backend is basically the "real" internal API, subject to changes. The frontend consists of nothing more than forwarding functions, that take their input from the outer world and translate it to backend API calls. How you organize Code reuse depends on the language, but inheritance is not a bad way to do it, although you might run into trouble with the size of your hirarchy. You can circumvent that with maps of functions/lambdas, like prototyping in Javascript.

For testing you then have your usual set of tests for the backend, but for the frontend you just test if the translation between public and internal API is correct. Very easy to test for in most cases. If you then organize your tests exactly like your frontend Version, i.e. some kind of inheritance, you only ever have to add tests for the changes in your new Version.

It is definitely more work than just having one ever changing API, but it's not that much of a hassle imho.

Reply on news.ycombinator.com