Hacker News Reader

> Nothing against CI/CD and continuous delivery, but the hamster wheel has gotten to a point where we have to release all the time. Corners are cut on everything, and testing is given lip service.

Just to call it out: Having CI/CD doesn't mean you have to do all these things, or do scrum. It's imho just good engineering practice. I have it in my side projects, and I've had it at work on a Kanban-driven team (as well as on scrum).

It removes disputes over build process (eg, nobody can ever say "well, it complies on my machine"). It is a form of build/deploy documentation. It removes bottlenecks ("only Tom knows how to deploy that service, but he's on vacation") and stupid mistakes ("whoever deployed this last did it wrong and left a bunch of old files around").

I also have found deploying regularly is stress-reducing. For one, with CD the development environment is running the same steps, and we know it works because we do it dozens of times a week. The longer it's been since last prod deploy, the less confident we can be it'll go smoothly, whereas when it gets deployed every few days it becomes a non-event.

When something breaks, having only 1 or 3 changes makes it really easy to figure out why. Recovering after a big release with 40 PRs in it is absolutely painful.

None of this means you have to release everything on a regular schedule (like a sprint). You can still do long running branches or feature-flag things off, Just try not to go too long without deploying something.

> When something breaks, having only 1 or 3 changes makes it really easy to figure out why. Recovering after a big release with 40 PRs in it is absolutely painful.

This is, by far, the most important reason to practice continuous deployment. I've been part of enough of these fire fighting sessions following big releases to see that it's not a sustainable way to deploy software. And yet, I've never been able to convince any boss I've ever had to adopt CD because they're worried it'll introduce more regressions into production.

The way I’ve explained this before is “would you rather small issues that we can fix one by one until anyone really notices or would you rather we hit all those issues in one big go?” Your mileage may vary.

Intuitively, I think most managers think that tackling everything in one go is somehow easier or less risky.

Speaking as a manager: any managers that think that are wrong.

Sure, but that doesn't mean it's not pervasive.

In the nearly 30 years of doing this, I’ve had more success delivering good quality software that meets business requirements under waterfall than any of the SCRUM implementations I’ve worked with, and I am saying that without disagreeing that waterfall has its problems. In the agile space, kanban is the only one I’ve felt is productive, but it has trouble scaling to multiple teams.

In my experience, waterfall forces business to think more about the project and plan accordingly. They have to think about how this feature interacts with this other feature. Business also usually has a better understanding of how the software will work because of the planning phase.

In my experience, with 2 week sprints, the business doesn't really have to think about anything outside of bite sized chunks or even how those bite sized chunks affect later bite sized chunks. They can make a snap decision without thinking about it because it can be rectified in the next sprint. Almost no-one has a "big picture," view of the software.

Before agile, we essentially did 4 releases a month (this was back on the burn a CD days), 2 major releases and 2 minor patch releases. It worked really well and we didn't get the burnout as much by sprinting a marathon. With sprints, I always feel burned out. No sense of accomplishment, no letup, working hard just gets you more work. It's like laying bricks one at a time on a wall with unlimited length.

> No sense of accomplishment, no letup, working hard just gets you more work. It's like laying bricks one at a time on a wall with unlimited length.

Also feeling bad every 2 weeks because planning was really bad and now you need to say you're sorry the task has to roll over the next sprint and your manager has to explain to their manager, etc, etc.

Yep. Somehow scrum leads to more rigid specifications than waterfall.

Waterfall forces management to get it's act together. These processes are all about giving management more power and less accountability.

Product Management has fallen a tremendous distance from where it used to be. PMs who have come up completely under Scrum seem to think stories and epics barely require more than 1-2 sentences and the developers should have to figure everything out from there.

Even senior people in their early 40s have completely forgotten how to specify or document anything well.

It makes me wish I worked on something that involved physical products. The whole thing with agile/scrum is that management can change their mind at any time for any reason and the process gives them justification. It works in pure software, it doesn't work in anything physical because they had to sign the POs to buy materials and parts.

>Before agile, we essentially did 4 releases a month

Correction, 4 releases a year, or one every 3 months.

> and testing is given lip service.

It's one reason I'm really thankful to work in a certified/regulated industry. It's mandated to effectively prove your work to get the sign off. And since you can't really sell too well in the area without that cert, it's a big part of development.

But holy hell is it still hard to get specific people to test their work. They just don't even think about it.

Well you are a manager, can you at least change how things are done for your team?

ex manager here. managers are in an even worse wheel. biz demands managers show up to even _dumber_, less organized meetings than devs think they have to deal with. eng to eng meetings tend to be fine, but the rest of company culture at BigCorp weren’t trained in structured problem solving. it’s negotiating with goldfish half of the time (generally friendly goldfish), and no one actually practices any formalisms or larger cohesive pjm. “just give me a gantt” like the author says would be an absolutely monumental improvement from how i see things get done

With software you have a situation with two problems.

First is the "gap" between those doing the work, and those writing the checks. (When it's the same person, this problem disappears.)

The guy editing the checks likes to understand progress is being made, and that the project both has an end and will be successfully completed.

The second problem is that by it's nature software "never ends" and many (dare I say most?) projects fail and are simply abandoned.

The moment the check writer is not the direct manager of the development you have an intractable problem. The person in-between (quite literally middle management), is often not technical. But he has to convince the bean-counters that this project is "on time and on budget".

He can't help but feel sometimes that he's herding cats. He's an irritant to those who are "doing the work" so they treat interactions with him as a waste of time. Inevitably he starts trying to measure things. (And we all know what that means.)

His job is hard. He's stuck between developers who don't want anything to do with him and higher-ups who want reassurance, bit don't really trust what he's saying.

The miracle is not that this process sometimes fails. The miracle is that it ever works at all.

And sure, you may not like your meetings, but at least understanding the game might help you understand why his job is the crappiest of all of them.

This is really interesting. Do you know of any articles/blog post that goes into these ideas further?

No, alas no material to refer to, other than my own experiences at several places in the food chain.

At the moment I'm having fun developing again, but I have had experience of management as well.

Obviously my experience does not speak to all cases, and I'm fortunate that most middle managers I've dealt with are competant.

I am in the fortunate position of being able to "speak truth to power" though, and generally been able to dispassionate translate technobabble into managerspeak, and help all the layers understand each other better.

I've found that when both sides understand (as much as they able) the complex demands being encountered, more realistic expectations and demands are made (and usually achieved.)

But I've been lucky. I've had good dev teams, good middle management, and good product owners. Who have all been pulling together to make projects successful. When one group is pulling the other way, things get tougher.

It's beyond software: any domain that requires oversight of non-trivial work should really have leadership from the ranks so they are able to ground their observations.

Managers with no domain knowledge are just an extra layer of indirection and overhead without any of the benefits. Would any IC want their career gated by someone who has zero insight into the team's day to day duties, doubly so if it's at a supposed "tech" company?

Trust and rapport are gained in time, often in short order. Small artifacts of evidence should generally prove to the bean counter that they are cutting checks for something!

> at least understanding the game might help you understand why his job is the crappiest of all of them.

Well, sure, but that's what the daily standup is for. So that the problem solvers on the team can understand the process, the problems it faces, and, importantly, find ways to optimize it.

But it seems, for how crappy you claim it to be, those in that position don't want to talk about it in fear that it will get optimized away. Job protectionism trumps all, I suppose.

The meetings between engineers are rather unstructured. I hadn't really thought about what meetings between business types would be like. Decisions get made in those that affect whether the company is a viable going concern. That makes for a very uncomfortable line of reasoning.