Hacker News Reader

Comment by mumblemumble | Hacker News Reader

> Starting from now users should expect more frequent releases of major versions as we move fast and break things to bring you even better experience.

I was in the process of evaluating Celery for a project, and it was looking promising, but this sentence alone might have prompted me to pull the emergency brake.

I've had past experience with projects that have a policy of shipping frequent major releases in order to have frequent breaking changes, and it was never a fun time. It's not just that the breaking changes themselves are troublesome. It's also that projects that are overly liberal about removing features tend to become overly liberal about adding features, too. So that, over the long run, they have a tendency to become bloated and clunky at a faster-than-average rate.

View on HN

Yeah, as a long time Celery user, I'd say you're making the right call to reevaluate your decision. Upgrading Celery/Kombu/Billiard has been enough of a hassle in the past with breaking changes, regressions, undocumented "features", and things getting dropped without any sort of deprecation timeline. I'm not looking forward to that becoming "more frequent" (and will likely be exploring other alternatives)

Just piling on here to agree with this sentiment. I've spent weeks of my life debugging celery/kombu/billiard-related issues, and I found that celery's behavior in many cases was too flakey to properly isolate a specific root cause. If I was starting a new project from scratch, would look into an alternative like Dramatiq.

I'm interested on people's thinking in this vein. If the current version meets your needs and you don't need to update to a newer major version, what's the problem if they keep releasing new major versions with breaking changes?

Say you pin your project at version 5. An exploit is found. There is no 5.1. Instead they fixed the exploit in 7.

However 7 breaks your pipeline.

What do you do? Fix the vulnerability yourself on an outdated version (high effort) Upgrade your entire pipeline to the new version (High effort) Leave the vulnerability in place (terrible idea)

You left out "switch to a better suited dependency (high effort)"

That's honestly the only viable choice with such libraries if you want to deploy your software in production environments.

Thankfully, there are alternatives around and you're not forced to migrate every task at the same time

Not getting patches since they don’t support previous versions for very long?

”As we’d like to provide some time for you to transition, we’re designating Celery 4.x an LTS release. Celery 4.x will be supported until the 1st of August, 2021.” [from the op]

An LTS version supported for less then a year from being designated as LTS?

Many projects have long term support (LTS) releases which will receive security updates for 2-5 years without gaining new features. Security updates are generally not backported to other older versions, though it depends on the project/community.

To add to what others have said: Staying on an old version (and only upgrading when your current version reaches end of life) may be a viable strategy if you accidentally find yourself dependent on a project with policies like this, and don't like the situation.

But that's a mitigation strategy, not a place you actually want to be. I would not consider selecting a dependency with the intent of doing things that way to be a sound policy, because infrequent big-bang updates like that have a tendency to be expensive and risky. There aren't a lot of development ethics that I find less palatable than "move fast and break things," but "move slow and break things" is definitely one of them.

well one problem is that Celery still has bugs and issues, and each new release fixes some bug of the month (it sometimes introduces new ones unfortunately).

There was a sequence of releases in 4.x that each had their own game-breaking bug for us, which has meant we have been stuck for a while (I've tried personally contributing fixes to this project, and fixed one bug, but another got introduced in the same release).

I would be happy about more granular releases, but I hope they could make more "bugfix-only" releases so I can make some forward progress on not having a busted task queue.

To bransonf's point, it's generally in your best interest to stay as current as possible on your dependencies. If a critical severity vulnerability is discovered in a dependency, you suddenly have 72 hours (or whatever your SLA is) to patch or upgrade. If that involves jumping several major versions (each with breaking changes), you're going to have a bad time. If you stay current, hopefully the delta is smaller and the task is less herculean.

Reply on news.ycombinator.com