Hacker News Reader

Comment by panic | Hacker News Reader

According to the survey, ~16% of people experience breakage with minor upgrades. Isn't this a pretty serious violation of semver?

View on HN

The Rust compiler has been very stable after 1.0. The issue lies in the ecosystem, and people is still figuring out the best design and/or solution for doing things, while occasionally breaking them. The biggest ecosystem breakage so far was termed "libcpocalypse", which was a massive hit for everyone (transitively) depending on libc crate for FFI. The proper resolution is still in under way [1].

[1] https://internals.rust-lang.org/t/solve-std-os-raw-c-void/32...

This percentage includes people using explicitly unstable things (e.g. me).

I believe (but cannot verify) that these breakages were due to libraries breaking, especially libraries which evolve in lockstep with Rust internals like aster.

The main reason I believe this is because each release is tested against the entire ecosystem, and stability regressions are fixed. The only time that breakages in the ecosystem are okay are during a soundness fix where the broken library was doing something unsound.

Doesn't the large portion of people using unstable libraries / things depending on unstable internals mean that the versioning isn't covering a large enough API? I feel like blaming the user isn't really going to help make the ecosystem more stable.

(Context: I went through the tutorial at one point and got really fed up that it wasn't possible to do it without involving crates because there wasn't any random number generation in the standard library.)

I of course understand that you folks are still working hard on it and it'll come in time; it's just that all other recent languages (Go, Ruby, Python, heck even the . Net and JVM ones) come with a large and usable standard library.

  > isn't covering a large enough API?

We stabilize lots of stuff every release, but we also don't want to stabilize things that aren't ready. Once we do, we're stuck with that interface forever.

A significant portion of nightly users are on it due to compiler plugins, even for libraries that work on stable, because it makes development a bit easier. Stabilizing those immediately would mean stabilizing compiler internals, which is a pretty huge drawback.

We have plans for addressing that specifically, as well as for the more general "libraries change" problems. They will come as part of an actual announcement, not a leaked first draft blog post :)

I came from languages where a big standard library was de rigeur (python, java), and I quite like rust's approach personally. For one thing, having RNG in the standard library would make it much harder with rusts stability guarantee to iterate on the API design in meaningful ways. For another, it's way easier to maintain external dependencies in a rust project than in others. Cargo is like night and day compared to pip, for example.

Yes, it's harder to iterate on API design once it's in the stable API. But that's the point, basically; it's harder to keep code working if the thing you're using keeps iterating.

And yeah, cargo is really nice. I just think the part where you have to go to it for all the things, with no direction on which alternative is (API- / crash-) stable, maintained, etc. is frustrating. That's the thing a standard library gives me - curation.

I wasn't talking about the stdlib, I was talking about crates. Rust tries to keep its stdlib small so that crates can evolve on their own (and have alternatives). I think that's a valid choice, just because needing to fetch a crate for random numbers is different doesn't necessarily mean it is bad. It also means that the versioning can be different -- the URL library can make breaking changes if it wants with a major version bump, and that's okay.

Sometimes, a breaking change in the ecosystem is marked as a minor version bump, and sometimes you need to coordinate versioning across crates. This causes breaking changes for downstream users.

------

Most of the actual libs with unstable rust dependencies out there are compiler plugins. "Things depending on unstable internals" exist in many communities and Rust is no different. That doesn't necessarily mean these internals should be stabilized.

A big one is clippy, which I maintain. This lints code by hooking directly into compiler APIs, which will never be stable to the extent clippy needs them to be, since that would freeze compiler evolution. Similar tools for other languages either reimplement a mini-parser+typechecker internally, or hook directly into the compiler (gimple passes, etc). There is a solution being worked on for clippy specifically. For tooling/IDEs in general, the Rust compiler will eventually get an "oracle" API that can be queried for type information and whatnot (which probably won't have enough info for clippy to work, but will have enough for general IDE stuff)

Another big one is serde, the community serialization library. There is an official one that works pretty well, but serde is better. However, the serialization codegen is done by a plugin. It can also be done with a build script on stable, but this is a bit more unwieldy. Some libraries use the plugin instead of the build script, and often break.

There is work going on for a better procedural macro/metaprogramming interface too.

That is mostly the extent of things which depend on Rust unstable internals and are used widely. There are a few more things, but they are less used.

  > Sometimes, a breaking change in the ecosystem is marked as a minor version bump,

... because it's still a young ecosystem, and a lot of packages aren't at 1.0 yet, so this is valid per semver.

To be clear, you're saying moving from 0.x.y to 0.(x+1).z is valid to be a breaking change, right?

(For other readers, Rust/Cargo uses a---more useful---variant of SemVer where an upgrade from one version to another from assumed to work if the most-significant non-zero numbers are identical, whereas SemVer says that any version number change pre-1.0 could be a breaking change, even 0.1.0 to 0.1.1.)

Semver says that for x.y.z, x == 0 means no guarantees about anything. Cargo currently treats x == 0 as the same as x == 1.

I disagree that this is more useful, personally, for exactly this reason.

Hm, I'm interested to hear why you think 0.1.0 -> 0.1.1 being considered just as dangerous as 0.1.0 -> 0.2.0 is more useful than treating 0.1.0 -> 0.1.1 as "semver-safe" and having only 0.1.0 -> 0.2.0 be breaking.

Thanks for replying!

And my view is that having a smaller stdlib and pushing all the responsibility to crates means that the language isn't actually usefully stable, since it means downstream still can't depend on not having breaking changes of its dependencies. The URL library can make breaking changes and have a major version bump; but that means everybody ever that used a URL will need to deal with it, which can be annoying if you happen to be using a crate that does so half a dozen layers in. Saying that a library is broken because its dependencies changed shifts the blame but that doesn't actually help the end developer. I totally understand though; that's more of an artifact of having a young language. I just still find it frustrating.

As for unstable compiler plugins: sure, it's looking at dirty compiler internals and whatnot. But that means people are using non-release compilers, and makes it more likely to start using the same in their code. The same could have been achieved by having a compiler version (completely separate from the language version, sort of like how GCC version numbers are decoupled from the C language); instead, we've got instructions lying around encouraging people to use unreleased toolchains.

I guess I really just want a stable, battery-included thing to build on top of, but while Rust is exciting it isn't quite there yet. Really thankful for the Rust team (and friends) for working on all this and letting us have a look, though, and looking forward to the day the ecosystem gets there :)

I don't know the details in rust's case, but I wouldn't consider it a violation of semver if the code which broke in a minor update was doing bad things (maybe accidentally exploiting a bug or limitation in the language or implementation, probably by violating some requirement that wasn't checked properly, etc) and simply couldn't be correct code if left alone.

How do you define a "bad thing"? The semver spec is quite clear about "even the tiniest backwards incompatible changes" (http://semver.org/#if-even-the-tiniest-backwards-incompatibl...) requiring a major version bump.

Cargo does not (yet) ensure that the library maintainer did follow the semver's doctrine.

But there are people working on it.

https://github.com/semantic-rs

Rust has its own definition of what it defines to be stable. Soundness fixes and bug fixes do not count, though as much as possible they will try to fix them in a way that breaks very little.

http://blog.rust-lang.org/2014/10/30/Stability.html

It may not be considered a backwards incompatible change to rust if it no longer compiles invalid code.

If the code never worked correctly, but silently compiled, one would think a breaking change was welcomed.

Semver does not define what a "compatible" vs "incompatible" change means, though. Manish linked to how we define it, but SemVer specifically leaves that up to the project.

Reply on news.ycombinator.com