Hacker News Reader

Comment by jordigh | Hacker News Reader

Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for downstream users, no matter how wrongly you may think they are using the interface.

It’s an attitude that is difficult to convey because software developers always want to have the freedom to “improve” their own software, even at the possible cost of breaking something for some users (and worse, even telling users that they shouldn’t have been doing that in the first place).

I keep going back to this blog post which I wish more people agreed with (Steve Losh is another person influenced by Mercurial):

http://stevelosh.com/blog/2012/04/volatile-software/

View on HN

Rich Hickey has a wonderful talk about it.

"Spec-ulation" - https://www.youtube.com/watch?v=oyLBGkS5ICk

Which I fully agree with, basically there are no major, minor, patch level changes, in reality every change is a possible breaking changes, because we don't control what downstream is doing with our software.

Oh well, he should try a language with types ...

/me runs ...

What are you trying to say? There are already a lot of explanations for why Clojure is dynamically typed and how it fills its role nicely that way.

I know, I know. We've been having some interesting discussions about Clojure, Haskell and dynamic and static typing on these stories

https://news.ycombinator.com/item?id=15602658

https://news.ycombinator.com/item?id=15593069

My comment was intended to be a wry one referencing those very interesting discussions and hinting that if you have types then you can more easily make breaking changes internally that aren't visible externally.

Dubious. The maintenance argument is weak. https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be...

I'm not sure what you mean. I thought it was self-evident that if your interface is an abstract type then you can change the implementation without breaking consumers. If you don't think it's true then please can you explain how? I thought the Clojure argument is not that types don't provide beneficial encapsulation, but that it's not worth paying the cost for them. Again if you know any different I'd be pleased to hear.

Apples and Oranges. We're talking about changing the interface. Did you watch the video segment?

> We're talking about changing the interface.

Who's "we"? I'm talking about changing the implementation.

> Did you watch the video segment?

I watched the bit where he said if you add a constructor to a sum type you have to change everywhere it's pattern matched. True. I'd love to see Clojure's solution where you don't have to change any existing code!

EDIT: Oh wait, I think it's even worse than that. I think he's talking about product types. In which case you should use projection functions and indeed you need to make no changes to old code!

All my favorite languages are typed and I do agree with him.

Really? How so? If you have abstract types then you can easily make internal changes without making those changes visible externally.

Some examples,

1 - Not all changes can be kept internal, for example adding a new case to an ADT

2 - Internal changes can still alter the semantic of the API, even if from the outside everything still looks the same from the type system point of view

Yes, both are true. 2 is a particularly good example.

As a simple example, suppose I swap out one container type with another one in the internal code. Both implement the same interface/traits, so the change isn't visible to users. But the time complexity of the container operations are different on the new type, and the user was depending upon the time characteristics of the old container. E.g., deletions are now O(n) instead of O(1). I've violated a user expectation without breaking type safety.

Nice counterexample!

> there are no major, minor, patch level changes, in reality every change is a possible breaking changes

It is "possible" that my blowing my nose will disturb butterflies in China.

Internal to the organization that produces the software, they are highly meaningful. Extended to user base that reads the documentation, it remains equally meaningful.

Downstream that resists information and can not be controlled is on its own. Good luck.

I agree that platforms should stay backward compatible as much as they can. That said, if upstream changes, sometimes you've got to change too.

There should be a mechanism for releasing breaking new major versions. And it is this:

1) Release a new version for all your DIRECT dependents. Ideally they should be subscribed to your updates.

2) After they have had ample time to test it, they will release an update for THEIR direct dependents.

At no time should C update A if C depends on B and B depends on A. In fact, ideally, if C depends on B1, ..., Bn then C will NOT upgrade A until all B's have upgraded. C should push the B's to do it and supply patches.

This means version pinning for all dependencies, my friends!!

The alternative is "dependency hell" or making multiple copies of A to live inside each B.

If some B's are taking their sweet time then it may be better for some C to submit a patch to B's repo, which even if not accepted can be pulled by other C's. The question is who "owns" or "maintains" a B project if they don't react to changes of some of their dependencies. The answer, I think, should be - fork the B if it's not being actively maintained.

Edit: I think HN needs a "reason for downvote" field, perhaps seen only by the person being downvoted. As it is, many downvotes convey very little useful information.

If upstream changes, pick a better upstream.

So if a large platform like facebook changes, simply stop supporting facebook? Wouldn't that kind of make your library very limited?

> there are no major, minor, patch level changes, in reality every change is a possible breaking changes

I noticed this when recently writing a spec that defines backwards-compatibility. There's surprisingly few things you can do when you want to stay backwards-compatible.

It's much like an immutable data store. You get one operation: insert.

With full backward compatibility, you agglomerate features, and changes are implemented as optional, with the default being the old behavior.

You can add layers that depend on existing code, but you cannot make existing code depend on those new layers.

>Mercurial, made by another (in my opinion) much more well-spoken kernel hacker, is what really introduced me to the concept that you do not break interface for downstream users, no matter how wrongly you may think they are using the interface.

It matters less, IMHO, the further up the stack you get.

Being right at the bottom of every stack it matters most of all for the kernel, which is why Linus needs to be such a stickler. The amount of damage you can do by breaking userspace is vastly greater than the damage you do removing an hg command.

For various reasons I would not go back to Java, but that was wonderful, your very old code still worked with newer versions, nothing broke during upgrades (even when behind deprecation one version).

We do enjoy breaking changes every now and then. :)

http://www.oracle.com/technetwork/java/javase/jdk-relnotes-i...

Of course, they are never at the level of a Python 3 release.

Yes, but compared to e.g. Scala breaking changes in Java are heaven, thanks :-)

There's a great talk about backwards compatibility in Java by Brian Goetz: https://www.youtube.com/watch?v=2y5Pv4yN0b0

Watched, great talk, thanks!

Go is really good about this too. It's great.

It's not as strong as the kernel. Go does not break APIs but may break undocumented behavior.

> you do not break interface for downstream users, no matter how wrongly you may think they are using the interface

How does Linux manage to never break its API? For example in the event of a refactoring.

I mean, is the answer really that hard? If the "refactor" would break the interface, you can't do it. Maybe you deprecate the old method and make a new one.

How do they handle bug fixing though? What if one application wants a bug fixed, but another application is depending on the bug inadvertently?

You trivialize the broken usage.

The bug they fix is the bug that keeps the API contract intact. The first application gets its bug fixed, and the second application breaks because it was relying on a bug that didn't match the API contract. You program to an API, not to an implementation. The second application did the latter, and will therefore pay the price.

Well, sometimes, but the Windows philosophy is probably to introduce a shim so that an app relying on the broken functionality still works.

If you can afford to do that, it's nice to do it indeed. It can get messy and hairy quite quickly though.

It seems to me a lot of it boils down to "who gets impacted". If I have a bug in my kernel and a lot of apps are somehow relying on it, I need to be mindful of that and perhaps swallow the purity of the contract in order to provide a pragmatic fix.

Sure, it's not a strategy without costs and Microsoft goes to an extreme. But there's probably a middle way in there; if you're aware that tons of programs now depend on the buggy behavior you probably want to deal with it. But if you're working on Windows or the Linux kernel the tolerance for breaks is probably much lower.

That's not Linus' position. The "API contract" for the kernel is "we don't break userspace code," not "we don't break userspace code that follows the specs." He would bounce (and flame) the fix you're suggesting.

In other words, the ABI you have is the ABI that got used, not the ABI you thought you were building.

That sounds insane. Is it written somewhere besides a sporadic Linus rant? (e.g. somewhere here https://www.kernel.org/doc/html/latest/ etc.)

Introduce a new API.

That is a situation they work towards never happening, by never leaving a behavior change in the wild long enough for such a situation to develop.

If you look closely at the kernel code it is easy to find two or more syscalls that have names and behaviors that are very similar, but where one is recommended above all the others.This is because the older syscall(s) were found to be flawed.

But rather than break existing software, they introduce a new syscall that fixes said flaws and leave the old one intact and recommend that in the future software use the new syscall.

Certain parts of the API once able to respond with a value now just spit out '0', it means that whatever application that used it just continue to work even though that part is deprecated.

> software developers always want to have the freedom to “improve” their own software

Developer can still have that freedom in most[1] situations. When you have a new idea for a better/faster implementation that is, unfortunately, incompatible with the existing interface, you can usually provide compatibility with a shim that implements the old interface using the new interface.

Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. Occasionally it will be a bit heavier, which might make the legacy interface slower, but this should be balanced (approximately) the faster underlying implementation. If the compatibility shim is significantly slower (or has other unavoidable runtime issues), an upgrade that is slower for legacy uses that still works is always better than breaking existing code.

For an example and opinionated discussion, see JWZ's post[2] about his OpenGL 1.3 -> OpenGL ES 1.1 header.

[1] Exceptions are cases where there is a *fundamentally unavoidable cost to providing the interface (e.g. where N is the total number of interface entry points, something in the critical path unavoidably MUST take >O(N) time or space, or N is limited externally to a small value ("we only have silicon for at most N instructions"))

edit: [1b] Another exception would be when the legacy interface itself is a serious security problem. If your legacy interface unfortunately forgot to include a way to supply credentials and simply accepts writes from anyone, a prompt fix that breaks that API is necessary and appropriate. Next time you're designing an interface, be sure to remember that security has to be baked in from the very beginning!

[2] https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-... (you might want to copy/paste that URL if your browser betrays where you came from with a Referer header)

> Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions.

That's more or less the thesis of Steve's blog post too.

There are cases where a major change to a software project caused a handful of minor regressions but added new functionality, vastly improved performance in the majority of cases, etc.

In these cases, a Zero Regressions policy is a bit silly. You can flip things around and look at the new code as baseline and from that perspective the old code has many more regressions relative to it.

There seems to be a pretty stark divide between these mindsets.

Why not simply bake these contracts and tests into the actual software APIs so they are impossible to break?

That way the app would be "backward compatible certified" if it used this layer. The layer would only run during deployment.

I guess an approximation is the Passing indicator on Travis or GitHub.

Some contracts are so complicated that we need 100-page RFCs to express them. What makes you think we can express these in a regular type system? (Unless you use Idris or Coq or something like that.)

I was about to post that it was funny this URL popped up when I read the Lobste.rs discussion until I saw your username :)

If only compiler developers had this attitude.

I think Rust has done this quite well, managing to have the best of both worlds. Unstable features are only available on the nightly build and need specifically enabling (and their interface may change). Once a feature is considered stable it is carefully maintained. They do have some breaking changes, but they're not common and for unusual edge cases:

https://killercup.github.io/bitrust/

This explains the Rust approach:

https://blog.rust-lang.org/2014/10/30/Stability.html

Rust also has yet to develop a stable ABI, I know they’re hard at work on so many things so I don’t say this as some snide remark - but until this is the case and Rust supports proper dynamic linking there’s whole domains of tasks I’m not comfortable using it for.

I can assure you we’re not actively working on a stable ABI.

This is a thing I've been wondering about. Is dynamic linking (in the way that C/C++ programs use it all the time) not a usecase that Rust wants to cover?

You can do dynamic linking, you just can't do it across compiler versions. This is how all of the Linux distros are doing it, for example.

It's a spectrum:

Rust: recompile the world when you update the compiler

C++: recompile the world when you change ABIs or when the ABI changes (My understanding is that MSVC++ changes the ABI every ~2 years? I could be wrong.)

C: never recompile the world

It's not that we don't want the benefits of a stable ABI, but it's a monumental task, and there are more important things for now.

Don't forget that you can get a stable ABI today by using the C ABI. You just lose out on the fancier features Rust has.

Starting with vs2015 microsoft promises more stable abi.

> C++: recompile the world when you change ABIs or when the ABI changes (My understanding is that MSVC++ changes the ABI every ~2 years? I could be wrong.)

This is only half true: every new release, the ABI of the runtime library changes. But the Visual C++ compiler ABI hasn't changed in decades.

If you design your own library ABIs so they don't pass std::foobars around, you can link C++ libraries built with VC++6 with ones built with the latest version, and the whole thing is going to work fine - the trick is that multiple versions of the runtime DLLs can coexist in the same process.

GCC/libstdc++ hasn't changed its ABI in more than a decade, since release 3.3 or 3.4; they even did incompatible changes to std::string in GCC 5 without breaking ABI:

https://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c...

Great, thank you for the clarification.

Unfortunate, but expected. On that note, would there be any breakage loading multiple Rust .so's with differing compiler/stdlib versions into the same process? This is one of my main concerns about the lack of ABI stability and the static-linking-first design of Rust. With C++ at the very least I know Red Hat isn't going to break the ABI within a major release, but the Rust 1.21 packages are already in epel-testing.

If you want to ensure that there's no breakage, the only way is to use the same version of the compiler on all the Rust code.

Well, Rust has a stable ABI in the same sense that C++ has a stable ABI: you get a stable ABI if you use an extern "C" and repr(C) layer at the linkage level, or you use the same version of the compiler to compile all your code. It's not ideal for some use cases (though I think not having a stable ABI is the right decision at this stage), but it's not worse than C++.

Rust also uses a sizable fraction of all publicly-available Rust code as a regression suite for compiler updates. This allows them to check edge cases that are "breaking in theory" to find out if there's actually any code that would be broken, or if the edge case had literally not yet been encountered.

By sizable fraction, he is referring to Cargo.io, Rust's public package repository.

They literally run all tests against every crate published to the package ecosystem. That is amazing.

Makes you wonder why no-one else came up with that idea before. I mean, it is pretty revolutionary, but also really obvious in hindsight.

http://cpantesters.org/

I'm curious why this is seen as a new idea. The capability to pull it off is somewhat new. In particular, at such a large scale.

But the idea is far from it. Commercial compilers probably had this more than open source ones in the past. Though, oddly, it was probably a weakness in some respects, since it almost certainly slowed feature development/releases.

Crates.io

Do they not? It certainly seems to be a guiding principle of C# compiler developers, who have almost always opted to preserve unintended quirks of the compiler/spec rather than "fix" them.

gcc and clang are _fairly_ good for this, and even MSVC has been getting better.

... just recompile

Minor point, but he puts Python in with software that "almost never breaks" on update. Really? Am I missing something? Maybe he just hasn't tried running Python 3 yet...

You're missing that Python 3 is clearly labeled as a breaking change? He is talking about changes that are not labeled as breaking changes but do infact break things.

Python 3 is a similar language with a similar name to Python 2. It shouldn’t be considered an update, nor applied as one.

Yet the Python 3 executable has the same name as the Python 2 executable...

Only on Arch and Windows... All other Linux distribution have python3 and macOS binary is also python3.

No they don't. At least one of them has a number after it.

I got both Python 2 and Python 3 installed on my machine, from the official distribution, and both have "python.exe", no numbers.

Yet a huge number of Python modules are both valid Python 2 code and Python 3 code.

Sure, they're similar enough that a single file can be valid in both with less effort than most language pairs. That doesn't mean it's an update, any more than C is an upgrade to Lisp because I can make a file that's valid in both.

I get the idea, but this is user hostile. If it shouldn't be seen as an upgrade, it should have had a new name. :( Especially if it is going to be a forced migration someday? (I'm assuming python2 support will eventually stop?)

One major breaking release in how many years, clearly announced as such and with parallel support for the old version for a long time (so you didn't just update python and it broke). Fits quite well under "almost never breaks".

Or have their been issues with minor releases? I haven't come across any I can remember, but I might be wrong. Maybe some things were deprecated and then removed?

If you're talking about Python 2 -> 3, I think you're misunderstanding his point.

He says Python packages, many of which bend over backwards to maintain backwards compatibility. (I wish Django were one of them)

Can't edit orig comment...yes I misunderstood the point about explicit breaking changes. I recently watched Rich Hickey's "spec-ulation" talk referenced elsewhere in this thread, which takes an extreme view (tl;dr SemVer sucks and we shouldn't have breaking changes ever), so that's where my mind was. Thanks for the clarity, all.

Reply on news.ycombinator.com