Never Ship on Fridays

Speak for yourself. In my shop we have CI/CD and we ship whenever we're ready to ship regardless of the day or time.

I don’t think this take is complete unless you include complexity and the risk of unintended consequences. We pretty much understand roofing, but software doesn’t struggle with the chains of the physical world and the base case can be significantly more complex.

> Just another reminder of the absolutely terrible state of our industry. Imagine anyone in any other industry taking this ridiculous stand. "Oh we don't fit roofs on Fridays, in case our roof leaks and we need to bring the roofer back in on the weekend"

Agreed, the industry operates on a much more janky level than many other industries, but this syndrome is by no means limited to the field of software development.

If your roof is functioning perfectly well, fixing a non-critical problem on a Friday is less safe than fixing it on a weekday because if something goes wrong the next day, it's better that the "next day" be a day when the roofers are available. All other things being equal, I'd say fix your roof on a Monday so you have the next four days to call the roofers back if things go wrong.

It's the same reason you're safer booking an early flight, or scheduling surgery early in the day: if something goes wrong, at least you will get bumped to later in the day. If your originally scheduled flight or surgery was later, and something goes wrong, you will likely get bumped to the next day.

Obviously this rule doesn't apply to emergency fixes: if your roof has a bad leak, and it's a Friday, and you know it's going to rain on Saturday, you're still better off having it fixed on Friday.

I really don't agree with any of your examples here, or the combativeness of your comment.

Roof work is usually a fair bit more deterministic than software.

For car work, it's not uncommon where I am for the tire guy, the vehicle inspector guy, or the guy who knows how to work on my brand of car to have unavailability.

My dentist has scheduled work earlier in the week with the explicit intent that if something goes wrong, I can be seen quicker. I appreciated this gesture, even if it wasn't exercised.

Actually what's happening in those examples is that issues DO happen on the weekend, the customer just suffers or calls someone else to patch things up till it can be addressed correctly. Difference with tech work is that your employer typically suffers.

Roofing isn’t unskilled, but building codes make it sort of a commodity. If you just invented a new type of roof, I would not want you to install the very first one on Friday night and then send home every available expert to party or travel or whatever for the next two days.

The thing is that the one roof doesn’t affect all your entire customer base. That isn’t quite a fair comparison to make due to the nature of software scale, but I can relate to the sentiment.

The core of your argument is that it's the worker deciding this, and that they don't want to fix issues over the weekend.

That's not true.

It's the business deciding that they don't want to ship bugs on Fridays and then take the hit for customers having a bad experience over the weekend if the issue can't be resolved. What if you let someone deploy on a Fri and then they go skiing for the weekend? Sure, someone else can step in, but maybe it takes them an extra 2-3 hours to solve the problem. That might cost you a renewal.

The consequence of a bad deploy can be millions of dollars for a client or business. Depending on the industry, it can be a million dollars in the course of an hour. If a dev team works Monday to Friday, then it's likely they're going to be harder to reach and coordinate with during the weekend. It usually costs less money to wait an additional business day than to get a team of engineers in on a weekend to try and fix a problem in production.

If a roof leaks on a Sunday and the roofer doesn't work Sundays it's going to be similarly difficult to get the roofer to come and fix it. The difference is the order of magnitude of money lost for a leaky roof vs a software fuckup at a pharma company or a business conglomerate.

It's ultimately the responsibility of the developers to test their code sufficiently so that there never is a weekend problem to begin with, but the reality is there will always be bugs that crop up in production, and you can only control that so much. You can usually control deploying on a Friday though.

It depends on the type of software. A B2C SaaS that has continuous small releases with easy incremental deployments and rollbacks is incomparable to a bespoke B2B release that integrates with dozens of the customer's other systems.

The latter are typically deployed over the weekend with multiple companies/teams on standby as unexpected interactions are at a certain point of complexity a given.

Your quality aim is for instrumentation and verification to rapidly detect and access and triage issues since assuming you can hit 'deploy' and walk away because your code is perfect (delusional) ignores that 95% of the systems you integrate with aren't even under your control, and 'it worked in test and was signed off' is not an acceptable answer for a client's production system being down on Monday.

Maybe it's just me, but if I knew the dentist will be away tomorrow, there's no way I am going to get my wisdom tooth removed today.

There’s actually another maturity level:

3. The processes are automated, and high enough quality that we can be confident that clicking the "go" button" will either result in the build/test/deploy pipeline halting due to a problem, or a good release going to production. But we still prefer not to release on Friday, because no CICD methodology is perfect.

At my workplace we release small fixes on Fridays, but prefer to wait until Monday.

I don't see how we could trust ci/cd to catch all errors. Tests are great at detecting expected errors, but it's not possible for us to get 100% confidence.

We also need some sort of tradeoff. Creating a perfect system with few bugs take a lot of time and resources that a startup does not have. We accept that our releases may include bugs, because we can respond to angry customers quickly.

In our B2B industry, most of our customers sales happen during weekends. A bug has a much higher financial impact.

Completely agree. Also, we have significantly lower usage on Friday into the weekend. Would rather catch a bug and quickly rollback Friday with fewer users impacted, rather than arbitrarily force devs to wait until Monday to queue up their releases during our peak usage.

As in most things, context matters. Tired of "always do x" and "never do y" opinions.

This is really really ridiculous. I've never worked for a company that has releases every day, even every day of the week except Friday. That's not how it works. The most I've seen was 2 days (Tue, Thu) but 3 was fine if you insisted too too much. In my current company we have only 1 day a week release: Tuesday. No other day is allowed or is even possible without someone spending tons of time doing manual things that are supposed to be automated. Moreover, if your code is allowed to be released next Tuesday it has to be merged by 4PM this Thursday. I still spend 100% of my time working, including Fridays. Because not all coding is releasing, and not all software engineering is coding. It's hilarious that someone would think if you're not releasing on day X you're not working. Sounds like straight out of Dilbert cartoon where some management type thinks I'm releasing my code every day.

Until we have formal verification that works across an API boundary where the parties are at different companies or much more extensive than usual test suites, changes to one end of an API endpoint will pass all manner of testing and still fail a customer use case. On Sunday afternoon in industries that ship over the weekend and start working again on Sunday.

It’s all a trade off.

We tend to avoid Friday deployments because they give us no tangible benefit while increasing the chance of an out of band issue. At a practical level, our customers literally do not notice if something ships at 4pm on a Friday or 10am on a Monday.

As for velocity, we can move way way faster by risking some bad deployments. Catching bugs is an exponential time trade off. We could catch most bugs, but we’d move like molasses. Instead, we catch what’s reasonable then ship during periods where it’s likely for us to recover quickly and easily.

Most companies will benefit from fastest shipping velocity at the trade off of occasional bad deployments.

And on what grounds/experience can you so quickly hand-wave all code that doesn't ship on Fridays as "shit"? This is lacking in nuance that anyone with experience would otherwise have.

Bugs happen. Sometimes bugs take a day or two to surface and aren't caught in dev/QA.

I would suspect there is a planning issue somewhere if an engineering team is brought to a standstill if they prefer not to ship on 1 workday of the week.

We save boring work for Fridays. We knock out quality-of-life patches/scripts/cleanup tasks. We'll review external and internal docs. We'll even explicitly verify the functionality of sufficiently large features, which has saved us a few times. Friday is our day to do tasks that improve quality.

I've seen enough weekends saved by not deploying on Fridays for me to be skeptical of any contrarian dogma that demands it's universal removal.

I’ve literally never seen a codebase with 100% branch coverage from integration tests. There’s always some risk of handing a corner case badly with invalid writes. Now you need experts to write ad hoc tools to repair what’s in your datastores, and you’re really screwed if enough of those experts are unreachable until Monday.