Hard deadlines are not user-first | Hacker News Reader

Hard deadlines are not user-first

jatins.gitlab.io

467 pointsjatins246 comments

In my experience when people focus too much on sprint deadlines, they miss the purpose of sprints.

Timed sprints exist not for the benefits of managers, but for the benefit of developers. They exist so that the development team has a shield against managers making last-minute decisions and changing priorities without notice.

If the focus and discussion is regularly on short term deadlines, then the developers are not driving the process, the managers are. It means that sprints are seen as a management methodology and not a development methodology. And then everyone has a hard time.

Managers who think this way know they’re not “allowed” to make mid-sprint changes so instead they focus on the end/deadline.

The managers should be spending their energy supporting the team and figuring out what the priorities are for the next sprint(s). If they’re spending their mental energy on deadlines then they’re doing everyone involved a disservice.

Scrum is for when you have a central dev team in a company that's shared between multiple groups. Once every sprint, you get all the managers in a room and let them fight it out. The result is a list of features that will be in the sprint. Everyone more or less gets their turn, depending on how powerful they are and whether something is legitimately urgent. Even if something is urgent, they can probably wait two weeks if they don't get it in this sprint. The dev team gets to work on something without interruptions for two weeks. The alternative is having everyone pushing around the devs, continuously changing requirements, overloaded devs, and general chaos.

The critical factor there isn't the sprint, it's the product owner. As long as the PO has the power to prioritise work, and as long as they are the only gateway for getting work into the team, chaos (from this direction, anyway) is minimised.

PO or scrum master as gateway for new work is sort of misguided malpractice that continues the insulation of developers. It may work if PO or scrum master has a clue, "lead" by asking/helping the team, but often in practice they are not technical themselves or external to the team. Having stakeholders fighting for priority and say, builds up helluva toxic environment that trickles down as well.

I see both options being shot down here so what's one possible alternative that does work?

What works depends heavily on the circumstances and the team (people first). What is to be accomplished: software delivery, operational quality and/or user/customer interactions? Are the responsibility boundaries clear or not? When overall structure is clearer, it is more straightforward to decide who are the best decisionmakers for the team..

What works for one team, might not for another. It's a good idea to have semi-stable teams as there is significant overhead creating temporary teams and disbanding them.

When there's no true need of urgency or the deadlines are outright fake, kanban and maturity may make a good alternative to scrum/sprints. It may even allow for some flexibility how to allocate people and borrow temporarily from other teams. You don't want teams to silo down, which they do the minute they are formed to start healing the wounds. Teams will need clear responsibility boundaries to be able to make effective decisions, while also granted the slack, incentives and psychological safe learning environment that fuel the necessary and logically next steps.

There's a perfectly healthy middle ground where POs are the gateway to new tasks but don't isolate the developers from the rest of the organization, nor disregard developer input.

And since we're talking about Scrum, PO and Developers (traditionally) supposed to sit together with stakeholders for Review meetings, gather feedback from them and provide input on the next tasks.

Sprints exist so that both management and devs can realize that there is more work than there is time. It forces devs to be focused and scope their work. It forces management to prioritize and scope down as well.

Keeping the deadline of the sprints is supposed to force management and devs to agree on reasonable scopes, such that the output of the dev team becomes predictable with time.

This enables devs to get trusted by management, breaking the poisonous relationship.

Sprints also have the pleasant effect of forcing some coherence onto work.

In the Old Days, teams might build software like a layer-cake: All the lowest layer, then the next layer, then the next, and so on to the top. So at the beginning, we’d be building infrastructure based on projections (a fancy word for “wild-ass guesses”) of what the subsequent layers would need.

You can, in theory, do that in sprints, but with an emphasis on delivering “working software,” sprints would help devs and stakeholders pick very small end-to-end increments and build whatever was needed to make them work.

This kept everyone focus on work that was known to be needed, and since you were building a complete increment, the team would be building the infrastructure at the same time it was designing and building the functionality relying on the infrastructure.

This kind of thing has its own failure modes, of course, but it eliminated a number of catastrophic project failure modes without even getting into the value of delivering working software sooner and revising priorities based on feedback.

> This enables devs to get trusted by management

I am curious about that “trust” part. In your model what happens when management doesn’t trust devs ? do devs trust management ?

Having a steady stream of deliverables and seeing that the devs can deliver on their promises, enables management to shift from control to guidance. That's not to say that it always happens, but I've seen it work.

Getting development done has traditionally been a game of getting developer attention. Scrum is one way of clarifying the contract between the stakeholder and the development team. I'm sure there are others too.

When management doesn’t trust devs, they micromanage. Often on things they’re way out of their depth on.

Though in my experience, devs do not trust “management” writ large. But they will trust individual managers who see their job as representing their teams within the management hierarchy and protecting them from the suits. That trust isn’t automatic and usually has to be earned.

Define "trust". As an industry I think we often talk about the wrong things here. People look at this from the point of view of controlling IT & developers (controlling costs, deadlines, functionality, etc.) when really everyone should be looking at it instead from the point of view of maximising value to the business. That's what Agile is really all about - it should really be about trusting that people will make good decisions based on the currently available info (which will likely be different than when you started the project), do good experiments, and find good compromises. This is especially true if unforeseen delays to a team's work has a knock-on effect on other people (which a good manager will try hard to decouple and avoid, deliver incrementally to lower the risk, or at least include healthy contingency for).

IMO, you should only really care about imposing deadlines as an answer to the question "how much time is it worth investing in solving this problem?" If the scope of the solution becomes larger than expected, and it starts to look like it won't pay off, abandon it and start doing something else instead. You had a good experiment which was entered into in good faith based on the data available at the time. That's the nature of experiments. Learn your lessons and try the next one.

Trusting that your teams have good plans that fail early and incorporate fast feedback loops is generally the only real way to make sure that they have realistic scheduling/deadlines for those plans and that they are likely to generate real value. It amazes me how many people in management seem to put their efforts and emphasis mostly on scheduling the plans, rather than the plans themselves. I guess it's easier to come up with estimates for things, than it is to come up with good strategy and business value propositions/justification.

I really recommend reading "A Seat at the Table" by Mark Schwartz (ISBN 9781942788119) for a good exploration of how agile meets management trust issues, budgeting and longer term planning, and how to try to break down the barriers between "IT" and "the business" (when in most modern companies, IT is the business).

Very good points.

Reading the comments, I think what I was bugging is the notion that would give a task not trusting it would be done.

People point at micro-management as an effect, because it basically means providing more limited tasks with limited risks.

But I think in the long term the direction we tend to take for people we don’t trust is to set penalties (bad evaluations, demoting, firing included).

In that sense, in average there should only be people you trust at the appropriate levels of an org, so I am not sure it is important to focus on gaining trust as it builds up just by doing ones job.

That’s where I like your idea of management trusting a plan, more than focusing on the team.

Yeah, in my experience this is “boundary setting” — and it’s absolutely critical to a healthy dev culture.

When I was a baby manager, I focused hard on Agile process and deadlines / roadmap commitments. And you’re right; it completely ruined my relationship with my team. They saw me as a representative of the oppressive corporate overlords, which is totally fair because those were the people I was trying to impress.

It took a few failed projects for me to get it through my thick skull that “doing things right” meant protecting my team from this behavior pushed down from above. It meant giving them space to do the job we paid them for. I realized I was letting my anxieties overrule people who knew way more about the issue than I did. It meant listening more and pushing back on unreasonable requests.

And it turned out my boss didn’t care about missed deadlines for the most part — he cared way more about production incidents. The best way to reduce production incidents is to work deliberately and not take unnecessary risks. A side effect of this was that we had way more power to say “no” to other teams, which made maintaining the code base a lot easier.

> protecting my team from this behavior pushed down from above

OT, but it's super common to hear in management circles about how managers are supposed to "protect" or "shield" their teams from the nasty CTO or CEO or some powerful stakeholder. IMO this is just not sustainable.

A CEO or any powerful person pushing a manager too hard accomplishes absolutely nothing at best, and is a disaster at worst. This kind of thing really hurts the morale and culture of the whole company.

Glad to know your boss is one of the good ones. My current CTO is like this and compared to the company I was in before it's night and day.

Tbh this is exactly what the article is talking about :)

If an organization has the right customer focus, internal deadlines should be meaningless. Customer-focused orgs are generally better places to work overall because there is a shared sense of what is important.

If you can't manage 2 weeks of work, you can't manage 6 months worth.

but it is very possible that you can create 'something' in 2 weeks vs. 'nothing' in 6 months.

and in most cases, 'something' * 12 > 'nothing'.

(although that 'something' might not be very changeable, or complete, or 'good')

and thus agile, getting things done with people that haven't done things before. (not meant to be snarky. long design iterations require experience in the problem domain. now days, no-one has experience. or the problem domain is new).

> If the focus and discussion is regularly on short term deadlines, then the developers are not driving the process, the managers are. It means that sprints are seen as a management methodology and not a development methodology. And then everyone has a hard time.

Right. This is the thing that's nearly impossible to achieve in most places and why so many people hate Agile: because they're dealing with "imposed agile" where the managers are driving the process.

Yep, the article argues that "locking sprints is a terrible process" because you're either going over or under the mark set upfront. As if those are the only options.

The idea of locking sprints was a reaction against the problem of going _sideways_ , i.e. "changing priorities without notice" and requests for unplanned work to be done "now" without any thought of the effects.

Sprint's don't have deadlines. That's the point of "agile". They have checkpoints.

Sprints are one of the ways that management, with the help of a cadre of consultants, took control of agile while nominally endorsing it.

That is not to say that sprints do not serve a purpose -- several, in fact...

>An ideal solution to this would have been to just go talk to the stakeholders - product/sales team - ask them if the delay could be potentially customer losing. And if it is, by all means put in the extra hours, work those weekends.

Strongly disagree, it's not on the shoulders of developers to ensure faulty promises can be kept. I will for sure not sacrifice weekends, this is far from ideal

If we read between the lines that means the layers of management are not needed at all if developers are talking to sales / users - see "Developer Hegemony" and "Developer Anarchy"

I second you. All stake holders will always say that it's critical. And they will make you work on weekends. My experience from consultancy.

Have to agree here. I sometimes find quite a big gap between deadline setters (sales/management) and the ones who have to meet them (developers).

I find it to be a very difficult topic, as everything has to coexist: no sales, no company but no developers no product to sell which is then no company.

I find that the deadline setting sometimes gets out of hand because it's not the setter who has to meet the deadline. For example if I set a deadline you have to meet and I can not influence the work being done (help or somehow make it easier to hit the deadline) how could I ever understand the "true" consequence of setting the deadline? In other words, not my weekend so I probably won't care as much as you.

What has worked for me (sometimes) is setting up the consequence; if we have to make x for deadline y, we won't be able to make a for deadline b. And then try and involve "sales" or the deadline setters in how we cut down whatever has to be made so we do make the deadline.

But that completely aligns with the post as that rarely is user centric or building the best solution for users. This simply gets stuff done within an arbitrary deadline set by someone, to meet a contract etc.

Bear in mind these experiences are from startups and not corporates.

I've ran into this with teams before. I think it's critical to ask ourselves "Who really are our stakeholders?". More often than not, the answer boiled down to a.) our customers and b.) our sponsors. If someone from sales, marketing, support, infra, or anywhere else lobbies for anything that'll turn into work, we work with our PO to make sure this is something the product team (customer by proxy) or sponsor(s) really think are important.

Time and time again we've butt heads with the sales team because they promise the world to one or two leads before checking back in with reality. Those interactions are met with checking in with stakeholders to validate assumptions. we gained the support of our executive sponsor or customers who are actually paying for the product to be built. Asking that question insulates us from greasing internal squeaky wheels.

If the sale is easy it’s because the lead thinks like your existing customers. If the sale is hard, it’s because they don’t.

Any feature you use to entice a new customer isn’t necessarily going to please your existing base. It might, as someone else suggested, actually be a considered a liability.

So the idea of building a new expensive feature, or a new feature in an expensive way, is suspicious. There’s no guarantee you can amortize that cost across all of your customers. And the times we really butted heads with sales? The features cost us more than the value of the sale, reducing margins.

But I suppose that’s the sort of metric dysfunction you invite in when you measure salespeople by sales.

I agree with your disagree. Work at a sustainable pace and if all your stories are slipping to the next sprint every time then you know you are underestimating. Likewise if you never slip you are overestimating. Forcing people to always finish each sprint will either burn out your team or cause estimates to be far too conservative leading to slower progress. That said, there are actual dead lines, where if you are late the software is useless and you are going to be returning a customer's money, or at best losing a customer's confidence. Those deadlines don't happen every two weeks.

If you are selling through sales people, and have enough new customers, then it’s possible to have a situation with a deadline every few weeks, unless someone intentionally brings sanity into this process.

Yes, sales getting ahead of production has caused a lot of burnout. Growing production teams as fast as sales can grow is hard.

The only time I've come close is when we fostered a culture of continuous improvement. We don't want to just produce features faster. If that's all you ask for you get people finding new and better ways to cut corners.

Learning productivity tricks and building tools actually lets you do the same amount of work in less time. Some quality initiatives reduce rework, which also increases throughput. And these practices also reduce stress during exceptional events (production issues). That effect may be mostly obvious but what we don't ever talk about is the fallout of stressful events and how they affect the quality of the code produced immediately afterward.

Burnout, as you say, is the end game of not doing this. But I think the symptoms are detectable long before that point (and I might argue are part of the feedback loop that leads to burnout).

Right, because the promise the developer is making(in most cases) isn't to to stakeholders or the users but to project managers. If the PM can't fulfill their promises on time, that's on them. Their responsibility is to better estimate what can be reasonably done in a time frame, provide buffers where possible, and shield developers from the business responsibilities that go above them. PMs and stakeholders need to be able to adjust their expectations to reality, not punish their developers by working longer or harder, or by changing the makeup of their development team, because it's not the minutes-spent or # of lines of code that make a good product.

Beyond that, working extra hours and weekends is a bad idea. It should only be done out of desperation because working overtime can quickly become the expectation.

Granted, the lines can be pretty blurred at some companies, namely startups that give lots of responsibilities to their developers. But most non-startups seem to have more refined command structures.

Definitely. That's phase 2 of the world's worst game. Phase 1 being "Please give us an unbiased estimate, but we will yell at you until it's the number we like." Under those conditions, working weekends is just feeding the beast.

I do think it can be ok to work extra hours to seize an unusual opportunity or deal with some truly unexpected problem. (E.g., the destruction of the World Trade Center in 2001 required a lot of tech people to scramble.) But that has to be balanced by something good for the workers and painful for the business, like an equivalent amount time off or significant overtime pay.

But workers shouldn't have to spend their spare time making up for managerial incompetence and malpractice.

I like bringing up this barometer the next time I'm asked to work weekends:

"Did a bomb go off in our datacenter? No? It can wait then."

I was lucky enough to see management get caught in their bullshit during one of these "hard deadlines" when I was just out of school. They said that December 1st was the deadline and that if we didn't get the release out we would lose millions in revenue. People run around like crazy until the end of November when we realize the EULA document is just a place holder and that Legal has not given us the new EULA. All of a sudden the deadline is mid January when we expect to get the new EULA.

What drives me crazy about this is how it shows that some employees are expected to bust their asses to meet deadlines, but others (legal) are not -- oh, if legal won't be done in time, we'll just move the deadline, no problem.

While some people -- healthily or not -- are willing to bust their butts (with or without extra compensation) when they think everyone is in it together, it will generally kill morale and willingness to do such when you realize when someone else 'doesn't have time', the deadline is moved, it's only you expected to work evenings and weekends and extra hours to meet the deadline. Why is your work-life balance less important than theirs exactly? (Only because they can get away with it...)

And while not always as stark as your example, people are often shown this all the time, that not everyone's inconvenience is in fact treated equally, that the "deadline that can't be missed" really only applies to some people, and is moved for others.

"Poor planning on your part doesn't constitute an emergency on mine."

It does, if you genuinely care about your customers or want to live up to your organization's MTP :/

If you make other people's planning problems your own problem, then over time everyone who planned poorly comes to you to save your day, and you burn out.

And then you're somewhere crying on the floor or in a hospital, and not coding. The organization around you has adopted to poor planning, because you encouraged them to do it.

I've started telling myself something and it's helped me to curb a bad habit:

"I'm not wasting this food by throwing it away, I wasted it when I let it go bad in the refrigerator"

If we really do care about the customer, we should be fixing the problem at the source, not trying to scramble to fix things after it's too late.

One thing that leads to those faulty promises, IME, is that we often conflate estimates with commitments.

When you work in a scrum team with a well-established workflow, and you have the experience that we usually finish around 20 story points per sprint, then shouldn't commit to finishing the next sprint with 20 story points.

You also know that that effort and technical challenges etc. vary greatly, so you can only do a commitment to, say, 8 story points per sprint with a straight face.

And now, try to tell somebody why you only commit to "so little" work being done by the end of the sprint when you tend to get much more work done... Mostly impossible, in my experience.

> One thing that leads to those faulty promises, IME, is that we often conflate estimates with commitments.

I want to meet the person who went around convincing PMs and scrum masters to abandon the word "estimate" in favor of "commitment".

I believe commitment has now been removed from the scrum guide.

One thing I've noticed on a lot of teams is that sales/product believes their own hype and overestimates the importance of their software in people's lives. There's this belief that customers/users are waiting impatiently for features when 99% of the time, they're just not. Yes, features affect users, but often users don't know what features are coming, because they just don't care enough to know.

Software I use every day: Firefox, Slack, Vim, MacOS, Python, Bash, GCC, Android, StrongLifts 5x5 App, Gmail, Google Docs, Google Maps, Simple (Banking) App, VLC.

All of these (except maybe Vim?) have released some significant features in the past year. Of these features, I only knew about a few Python features that were coming before they came, and none of them were ones where I was thinking "I can't wait for this to come out! If this doesn't come out soon I'm switching languages!"

On the contrary, features take a long time after they come out for me to learn how to use them and for them to affect my decision making. In fact, my initial reactions to new features are much more likely to be a liability than to be an asset to the software company. I'm considering switching away from Simple because their expenses update doesn't really fit my needs, since as a freelancer I'm not paid on a regular schedule. And I'm definitely switching away from MacOS since their deprecations have broken some tools that are key for me. If they had taken more time, maybe someone would have noticed the problems and avoided them.

Sure, early on things are different: first-to-market advantage is real. But for most of the lifetime of a project, you aren't releasing killer features. You're releasing incremental changes that have a much higher chance of losing you users than of gaining you users (in the short run, at least). Never releasing isn't an option--death by stagnation is a thing--but releases should be done with painstaking attention to detail rather than urgency.

Another thing sales forgets is that the features you already have are more valuable than the new features to actual users. They always want to sell the new things, but provided features are added with most valuable first the earlier features will be more valuable than the new stuff. This gets really silly when they rush releases to get the new features and end up breaking old features, which actually decreases overall value of the software compared to releasing nothing new at all.

This is fairly controversial. Is there any example of a startup that actually made it by employing this strategy?

One of the truisms I've heard and have always followed is you need to release something you're embarrassed about, because that's the quickest way of getting user feedback, and understand if a user actually _wants_ that feature, and if it's actually worth spending time improving that feature.

It's easy to waste valuable capital paying attention to details that the user does not really care about.

EDIT: I've noticed that most of the software you posted here is very well established, most > 10 years. I'd be surprised if any of them follow "agile" by the books.

Not the OP, but I believe he and you are talking about two different stages in the life of a company. He's describing mature products and organizations (as is the blog author), and you're talking about a company that is attempting to complete their first offering.

> Is there any example of a startup that actually made it by employing this strategy?

Is there any example of a startup that actually made it by releasing features that were broken or poorly thought out?

Is there any example of a startup that actually failed because they released features a week or a month after an internal deadline?

Do we have enough insight into the internal strategy of enough companies to make any claims about this at all?

Your argument is basically looking at existing startup culture, but realize that most startups that use a fail fast strategy do exactly that: fail. I'm not saying that's causal: I think success is hard and there's a myriad of reasons why startups fail. But I don't think we can look at the fraction of startups that have succeeded and point to this one element of their strategy and say, "This was a necessary component of their success."

I'm not looking at the whole industry because I don't think there's enough representative data there. I'm looking at my own experience as a user and noticing, "Hey, I actually don't even know what features are planned for any of the software I use, yet I'm still using it. So maybe future features don't have anything to do with why I use software. And maybe other users are similar."

Let me ask you: can you name 5 features that you're looking forward to in the software you use? If those features take a year longer than expected to materialize, do you think you'd switch to a competitor? That's rhetorical: I am sure some HN people can answer those in the affirmative, but I'm also sure that's not the norm.

> One of the truisms I've heard and have always followed is you need to release something you're embarrassed about, because that's the quickest way of getting user feedback, and understand if a user actually _wants_ that feature, and if it's actually worth spending time improving that feature.

Sure, but then you don't know if the user doesn't want the feature because it's not useful, or because it's badly implemented since you didn't spend time to get it right. You're collecting data faster, but you're also confounding the data so that you don't know what it means.

> It's easy to waste valuable capital paying attention to details that the user does not really care about.

Agreed. One of the details I'm saying the user does not really care about is whether your feature is released today or three months from now.

> EDIT: I've noticed that most of the software you posted here is very well established, most > 10 years. I'd be surprised if any of them follow "agile" by the books.

...which is a pretty damning statement about '"agile" by the books'.

I'll say what I've said before many times: the entirety of agile is contained on this page: https://agilemanifesto.org/ . "Agile by the books" is an attempt to make money off a successful strategy, but you can't sell six lines of text, so people write books on how they implemented agile, which if you understand agile, is fairly useless for understanding how YOU should implement agile. If you really grok those principles, then it's really clear that releasing early to get user feedback doesn't follow the principles:

Individuals and interactions over processes and tools The deadline is a process rule that you're prioritizing over literally every individual in this situation.

Working software over comprehensive documentation Releasing broken software so you can document user response doesn't follow this principle.

Customer collaboration over contract negotiation Customers should be involved long before you're releasing features.

Responding to change over following a plan The deadline is the plan. The fact that the feature isn't ready by the deadline is the change that you need to respond to.

In my last job we were sufficiently sophisticated users of AWS that we routinely had conversations with product managers to learn if specific features were on the roadmap, and if so, if there was an ETA. We had these conversations because we were debating whether we wanted to write a solution of our own, or if we wanted to wait for a generalized one to be released.

That said, the product managers generally left us with information such as "this is actively being worked on", "this is planned work in the upcoming quarter, but it could change", "this is on the roadmap without a date", and "this is not on the roadmap". No specific dates promised, but it was still good enough for our purposes.

I hope they weren't using our requests to push for specific dates, internally.

It is worth noting that ALL the agile values you spelled out are actually important. That said, the values on the right do not take precedence over the values on the left.

Processes and tools are important so long as they don't diminish individuals and interactions.

Documentation is important so long as it doesn't get in the way of building software that works.

Contract negotiations are important so long as they don't get in the way of a collaborative relationship with customers.

Following a plan is important so long as it doesn't get in the way of responding to change as a competitive advantage.

It's more of a marketing-first approach than user-first.

There is something I'm impatiently waiting for though: Optimization so it doesn't take 5+ seconds for Photoshop to load and be usable. Why doesn't it load within 100ms on $3,000 modern hardware? It's image editing software, not Grand Theft Auto!

That's where some energy could be put and make a real, tangible difference to everyone.

It kinda is Grand Theft Auto, though. Most of the opening time is spent locating assets and plugins to build menus (and all of those can be added or removed without launching Photoshop).

In our case, our sales team makes promises to customers about features, and puts it in the contract

Bingo. That's sooo bad for engineering that a company does that, and probably bad generally as well. My company does this and its infuriating, like, you don't even know what's involved, why would you do this? My company also, for most of its lifetime, has relied on contracting out software development.

My current project in fact started out life offshore but the work was so slow-going and awful iterative results that they then rev'd up a real software team to complete it. Initially they wanted us to complete the project without scraping the offshore code base, which was in baaaaad shape. It took another consultant, hired for a different reason, rather than listening to their in-house leads (of which I was not one, I was a junior at the time), that they needed to scrap that code.

> My company does this and its infuriating, like, you don't even know what's involved, why would you do this?

Sales commissions?

When the customers are telling your sales people that they are going with the competition because they have features that you don't, one should take notice, though with the caveat that the customers are not necessarily being completely transparent about their motives.

With saas products you can call bullshit on these situations. The second time you are presented with a demand to produce feature X by date Z you can take a look at the results of the previous time that you were told that feature A needed to be done by day B and show exactly how many customers used that feature on the due date and exactly how many new customers signed up at that time

There are 2 normal ways to do things: work until you achieve the functionality you want and release it when that happens or work until an arbitrary date and release whatever you have at that point. Surprisingly, as a developer I always prefer the latter. As a coworker of mine if fond of saying, the time you know the most about the project is when you are finished. The time when you know the least is when you are starting.

By choosing to release "whatever we can do" at a certain time it encourages you to make difficult prioritisation decisions as the "deadline" approaches. Are we really OK with releasing with just this functionality? Maybe we should axe X and replace it with Y since we don't look like we are going to get both. Maybe Z is an even better compromise that we didn't realise was an option at the beginning of the project.

If you try to hit functionality, it allows the organisation to avoid rethinking its planning decisions. We aren't releasing until the ideas we had 6 months ago are all implemented exactly as we envisioned it back then. By the way, why is it taking so long?

Of course trying to hit both at the same time is just a recipe for disaster.

The problem with little upfront planning is the team might spend too much time on certain features in the beginning and get crunches at the end to deliver an MVP experience at the agreed upon deadline. By signaling early that a deadline may be missed, there is an opportunity to make course corrections in the spec or shift folks from other projects to deliver a higher quality experience at the agreed upon deadline.

Clear-as-possible descriptions of uncertainty are the only real hedges against this, because they create the opportunity for directed experiments.

If the organization is mature enough to rethink the project scope when approaching deadlines, couldn’t it also take the first approach and rethink the delivery as features are getting worked on ?

There is effectively organizations that look at ongoing R&D subjects and say “this is almost shippable, we’d just need X. Let’s just priorize X and ship it”

Depends. Some can, but my industry cannot. I write seasonal software. A release in May won't be used until the next March, so there is no point in any release that isn't in January/February. (A march release will be skipped because they are scared of bugs, February releases gives customers time to upgrade and test before then critical time when they need it)

Thus for me it makes the deadline or it doesn't. Early isn't a benefit, late is a problem.

But you don't quit in April, you are working on the next February release. So you don't have a deadline, you have a checkpoint.

Sort of. I go on the next project, but we let some contractors go at the end of the project. Ultimately I'm putting a computer on machines that were common in 1890: there isn't much innovation to add that previous engineers couldn't think of. We can get larger than past generations, a little more accurate and reliable: but in the end the designs from 1890 weren't that much worse that today's.

Exactly this, and what a lot of less senior developers don’t understand is that the fixed timeline is actually a defensive mechanism for the development team. If the date is fixed the functionality has to be variable and the best people to assess what fits or not is the development team who thus can be in control of their own planning.

No, for many managers the fact that the deadline is fixed means they can death march you for all the functionality they decide needs to be complete at that date.

The idea that adding additional process to manage your managers is laughable at best.

Manager can't hold your fingers and force you to type code. Agile's short sprints mean that managers learn in 4 weeks instead 16 that death marches don't work. Managers can be managed as much or little as reports are.

I know currently of several companies being managed into the dirt by people who dont learn no matter the scheduling mechanism or level of machiavellian maneuvering by their employees.

So no, you cant manage your boss, you can inform them, maneuver them, help them, cajole them; but at the end of the day you cant get them to change their behavior if they truly don't want to(and they can just fire you for being too annoying.)

In the current environment one can get away with a lot of push-back because managers with any fraction of a clue know if they piss you off too much you'll be giving them your notice, new job in hand, in a few weeks, if not days. That won't last—which is why if we're going to organize for better leverage against abuse in bad times, now'd be the time to do it, but I guess enough people've been convinced unions will impoverish them that we're not gonna, maybe next boom, see y'all then—but for now it's pretty nice.

I agree, but I think there's a still better way. If the "deadline" is too early, you risk unnecessary failure. If the "deadline" is too late, you end up developing to an arbitrary feature list again.

I'm a big fan of going hard for a minimum viable product. The notion is that you find the absolute minimum that will let you close the primary feedback loop: seeing what people do in the real world. Maybe it's only one set of users, like early adopters in a particular market segment. Maybe we leave out a lot of "real product" things, like help and docs, making up for it with handholding. But we ruthlessly cut scope until we have something in users' hands.

At that point, I think it's possible to shift from an internal focus (e.g., an arbitrary date or a made-up list of features) to an external focus, where we look at who's using it, who we're delivering value for, how we can sell it, and what the next most valuable improvements are. W release weekly or more often, driven by hypotheses and data. For me, that's when the real magic happens. Then we can focus less on internal politics, and more on what we're learning from the real world. And there's always something to learn.

> There are 2 normal ways to do things: work until you achieve the functionality you want and release it when that happens or work until an arbitrary date and release whatever you have at that point.

Wise words. Herb Sutter wrote a long blog post on exactly this, with regard to how C++ language versions release on a schedule, whether or not your favourite feature is ready yet.

https://herbsutter.com/2019/07/13/draft-faq-why-does-the-c-s...

At Pivotal, I learned that having sprints is silly. They have the concept of one week iterations instead. The PM just keeps a rolling backlog of stories and devs just pick the next one off the top. Dev's are responsible for pointing the stories and the sum of those points averaged over time (usually 2-3 weeks) become velocity.

Just keep the velocity as consistent as possible and things work out for the best. If the PM wants a story moved up the list, then they can see how much work will fit into an iteration and how it will affect schedules.

All of this is managed in their homegrown, but publicly available tool called Pivotal Tracker. You can read the website documentation for a more thorough explanation. Once I learned this methodology and started practicing it like a religion, deadlines went away and PM's could estimate due dates many weeks in advance. It was a life changing experience that I highly recommend at least giving a shot at your own workplace.

This sounds exactly like every other sprint-based process I have ever encountered.

Sprints have deadlines and a set amount of work... 'A two week sprint.'

Iterations are 1 week and there is no expectation that a story will always fit within the current iteration.

The only tool that I have found which can accurately visualize this difference is PT.

What is the point in an iteration?

Good question. An iteration is a marker to generate velocity.

All of the points collected in that week are summed together and the average of those points across a configurable threshold (typically 3 weeks) is the velocity. Once you have a velocity, you can start to make predictions about the future.

For example, if velocity is 10, that means 10 points worth of stories can fit into an iteration. If the PM moves another story into the iteration, the last story would be automatically pushed into the next iteration. Makes it easier to answer the question of 'when will a story be done?' as well as 'how will changing the flow of stories affect deliverables?'

Highly recommend you read the documentation, it has been formed over many years and is excellent.

https://www.pivotaltracker.com/help/articles/quick_start/

Because then we'd call them weeks and Sprints would be Fortnights :)

In all seriousness I'm curious about the story pointing. Does the team meet together for that at Pivotal?

Repeated from above:

Pivotal has 3 regular meetings: daily morning standup (short), iteration planning (monday), retrospective (friday).

Any new stories during the week that haven't been pointed get pointed in the morning meeting.

Sounds like Pivotal combines elements from scrum and kanban in a way that works well for them. That's what you're supposed to do with agile -- keep adjusting the process until it's working well for your organization.

this ^^^^^^^^^^^

In practice, most places I've worked treats sprints much closer to how you describe your iterations than to a fixed amount of work.

Most of the time, the sprint planning just provides some ceremony around ensuring important stories are discussed and points allocated, and priorities set, and the planned sprint is pretty much a line in the backlog that suggests an aspirational goal that is likely to change. The point is rarely to hit exactly that, but for it to serve as a means of talking about what needs to be done first - whether something needs to fit into this round, or can wait for the next. It also serves as a means of talking about inter-dependencies and sequencing of changes.

I think you've been lucky in where you have worked. You read some of the other responses and it isn't the case at all. I've seen some really tragic examples.

Pivotal has 3 regular meetings: daily morning standup (short), iteration planning (monday), retrospective (friday).

Any new stories during the week that haven't been pointed get pointed in the morning meeting.

out of curiosity what value do you see in morning standups? 15 yrs and i still don't see it (i've really tried).

I agree that the whole typical 'what did you do yesterday' is indeed lacking value.

The standups are mostly just pointing the stories that need pointing, covering any questions and picking the pairs for the days work (pivotal is 100% pair programming). The meetings can be done within 5-15 minutes at most.

I'm currently living at a yoga retreat center in the mountains of Vietnam. I can see from the practice that the first few minutes of silence is good to clear the mind. I think having standups is a similar good way to start the day with a positive attitude.

I also find value in standups, but the problem is they constrain the team to some daily meeting time. For example, I start work around 6am. A co-worker does morning kid duty and tends to start closer to 10am. There is no good way to start our days at the same time, and there really is no need.

What we have been trialing for awhile is a simple Slack checkin. Everyone reads them when their day starts, and if there are any questions or comments a Slack thread is started from the checkin message. It's worked quite well.

Pivotal solved that by requiring everyone to work the same exact hours. It is a bit military in that regard, but it works for them. You don't get a job there unless you buy into it. Their reasoning is more around their culture of pair programming.

It sounds like you have found a solution that works for your company and that is great! The way I look at it is that as long as we are shipping software around the time when we say we are going to and the quality is high and people aren't killing themselves (weekends) to make it all happen, all good. =)

I just asked the same thing because I've had that experience too. I'm curious: are you usually working on things that require close collaboration between team members?

My hunch is that standups are pretty useless on teams where each developer is working on an independent change where there isn't much opportunity to get blocked by teammates aside from waiting for code reviews and the like. That's most of the teams I've worked on, and the standups inevitably turn into a verbal form of, "Hey manager, here's my status report for the day even though you already know all this because I've already said it in Slack," because you have nothing to ask of the other people on the team but you're required to say something every day.

Where I've seen standups work effectively is when an engineer has a bit of context on a problem that someone is dealing with, or an insight that might unblock someone. I've had many times I've mentioned in an (async, written in slack) standup status update, and someone's chimed in with something like "Oh, I've dealt with that before - let's pair for 10 minutes so I can walk you through some gotchas".

Another plus for me personally is that it has a focusing effect on myself as a developer - if I say I'm going to get something done in standup, I'm going to force myself to focus on that task, and not get distracted by something else unless it's really important. I can see how that's a totally personal thing though - I need to have some sort of external "deadline", even if it's self-imposed and soft, to keep me focused.

All of the above can be solved with async updates in Slack though, and that's probably actually a better medium. Having folks get in the same room seems a little pointless to me, especially when you have remote folks and timezones are an issue.

The problem I've had with async updates is that they often get ignored because everyone that could help is busy at the time, so you still need the actual standup to just remind them about it. How do you get around this?

I don't have a great solution for that, really, other than ensuring that updates are highly visible in a room where people are likely to read them and have a good culture around helping folks.

I've been in environments like the one you describe, and I think at the end of the day it's more of a cultural issue than something that can be solved using a tool.

this ^^^^^^^^^^^

I get around it by pinging people directly (usually I reply to my question as a thread in Slack and @-mention them, so the main channel doesn't get cluttered with my reminder) if a couple hours go by without a response. Still usually faster than waiting until the next day to ask at standup.

   My hunch is that standups are pretty useless 
   on teams where each developer is working on 
   an independent change where there isn't much 
   opportunity to get blocked by teammates aside 
   from waiting for code reviews and the like

Personal anecdote, but I like to know what my teammates are working on even if we're not collaborating directly.

I could (and do) read their commits and PRs, and standups are not a replacement for that, but I think you can usually get a better idea just by chatting for a few minutes each day.

I view the cost of a daily standup as very small (they should be only a few minutes long!) so while some would call the benefits modest, I think the "time spent vs. value obtained" ratio is still very good.

For me the cost is much less about, "It's just a few minutes long" and much more about, "Having to get up from my desk and talk to people about their projects kills my flow state." It's a short meeting but it's still a meeting and has the same effect on focused thinking as a longer meeting. Losing focus is totally fine if there's benefit to it, but for standups where I neither learn nor teach anything new, it's all cost and no benefit.

I too like knowing what my teammates are working on, but I already get that from the team's Slack channel where people talk about what they're doing as they do it.

I agree about the frustration of having my flow disrupted, but my like of daily standups is related to my experience that they create an overall reduction in interruptions.

Of course, that's my experience. Others may have had different experiences and I respect that. If you formerly had 0 interruptions per day and now you have 1 interruption per day, that would be frustrating. I would also like to work where you work!

>I view the cost of a daily standup as very small (they should be only a few minutes long!) so while some would call the benefits modest, I think the "time spent vs. value obtained" ratio is still very good.

My problem with a morning standup is that it completely destroys my productivity if I come in early and really get cracking on a problem, then have to stop and task-switch. I'd much rather make it optionally asynchronous.

Yeah, I feel the same way sometimes regarding the timing. I don't think they should be in the morning. The older I get, the more of a morning person I am... that's my productive time.

I personally would rather do those meetings in the early afternoon, like before or after lunchtime.

Pretty sure there are plenty of studies that show early afternoon is our least productive time anyway.

(TBH, and I'm being completely serious here, my ideal schedule would also involve siesta time in the early afternoon)

Standups are for "weakest" devs, to break silo of "I am working on my thing don't bother me".

"weakest" devs - you know not everyone is outspoken and will communicate without issues, new young devs will be afraid to ask an experienced guy. Some experienced guys (also "weak") will be assholes to young devs for asking questions. Just getting team to talk to each other at least once a day is positive. Unless you work only with senior people who are experienced and don't have any issues talking to each other, I would still try to get people to gather for standup.

Weaker? Senior? We have something to learn from every one.

Nobody should be an asshole. Nobody should be working alone.

Both of these are great ways to create a really toxic working environment, please don't perpetuate it.

How would you express that thought, to get through to a person I replied to who wrote he has 15 years of experience and he did not see any value in standups?

Do you think he is not a senior person? Is he a bad person for pointing out his years of experience? Maybe I am bad person trying to express in BOLD that there are people with less experience, social anxiety, assholes and we have to work with all those personalities. With strong emphasis that standups are good tool for getting rid of toxic work environment and getting teams together.

Counterpoint: comment was fine and non-toxic, describing siloing as weak no matter who's doing it is an accurate and a pretty tame criticism, and ensuring that's not happening is indeed the only real point of scheduled daily stand-ups. As long as we're gonna have the term "senior"—and we do have it—I'd abso-friggin'-lutely reserve it for people who can avoid siloing without a formal, daily ritual, and a team of such folks don't need stand-ups. Don't worry about it, ozim.

> Nobody should be an asshole. Nobody should be working alone.

Ozim advocated neither being an asshole nor working alone, so is perpetuating neither. Standups have a place and it's to fight both of those problems, when some ICs on a team can't handle it on their own.

The best dev on my team is also a very reluctant communicator.

This guy is a beast at getting things done, and is a good communicator once you get him talking. Overall, a joy to work with. However, you do typically have to make the first step. He kinda stays dark without scheduled meetings.

To me the key is to keep the daily standups VERY brief. They help the "reluctant" communicators. And they don't cost the "active" communicators very much time.

I am a fairly "active" communicator myself. I'm gonna spend 15 minutes talking to my freaking team every day anyway. So a brief 10-15 morning standup is no problem at all.

First off, I'd say that a good daily standup meeting shouldn't be more than 15 minutes. If it's longer than that, you are probably getting bogged down in details that don't belong there.

Let's be generous and call the "real" cost 30 minutes of productivity... 15 minutes for the actual meeting/call, and 15 minutes of context switching cost.

That is 6% of an 8-hour workday. You can still code up to 94% of the time, not bad. And I think it has been shown many times that most developers don't really pound out code for 8 hours per day. We might work 10+ hours per day but it's not necessarily 10+ hours of coding. So I feel the cost of these daily meetings is very small.

What are the benefits?

1. For one thing, it is good to know what the rest of your team is working on and what challenges they are facing. This jogs a lot of useful conversations in my experience, where Developer A turns out to have some experience in the area in which Developer B is stuck. You may be thinking, "good teams communicate and can do this literally any time... why schedule a daily meeting for it?"

Yes. If your team is operating and communicating flawlessly, sure, standups are redundant. However, I've been at this for 22 years and this is not typically the case, and the brief daily meetings are a real upgrade.

2. These meetings help prevent "drive-by shootings." That's what I call it when management randomly stops by your desk and wants to know what you're doing and if you're stuck and wants to see your work in progress. I hate hate hate hate these. I would rather just briefly chat with leadership on a predictable basis. And when I am leadership, I would prefer not to subject my team to these "drive-by shootings." A lot of teams are already paying this cost, which I feel is much greater than the cost of a brief morning standup. Again, if your team is already communicating at a super high level, maybe you don't need standups. I feel this is not the case for a majority of teams.

3. For remote teams, these give folks a chance for some regularly scheduled face time and chitchat. We kept our morning call to < 15 minutes today and talked about Star Wars. As a remote worker myself I truly cherish this. If you have a remote team I would call video chat standups more or less a requirement.

Out of curiosity, how valuable are these three meetings in a typical week? What do you get out of each of them as a developer?

I've worked at a few places that follow more or less the same practice and I've found that the planning meeting is hugely valuable but that standups and retrospectives are a waste of time more often than not.

Remember, meetings are what you make of them. If you can't find value in them, either change the topic to make them more valuable or don't have them at all.

The retrospectives at Pivotal are actually quite fun. Usually involves optional booze and doesn't last for more than an hour. The goal is to make them about anything, not just talk about work. It is like an end of the week team building exercise. Think about it, most of us spend more time with these people in the office, then we do with own family. So, we might as well have an hour where we put down the phones and keyboards, reflect on how the week went, find things that can be improved, talk about life events or anything really. Find the positive aspects and focus on those as well as quench any negative things so that they don't carry onto the next week.

Maybe. But the point is that there isn't really that kind of sharp delineation, but a spectrum where you really can't assume that people talking about "sprints" means a fixed amount of work.

The process you describe fits precisely within the Scrum framework; a key point of the retrospective in Scrum is not just to understand what has been delivered but what caused deviation from the estimate at the start and why. The assumption throughout is that the forecast from planning is just a forecast and that what will be delivered will pretty much never be exactly that.

Maybe there are other things you do that does not match Scrum, but the above fits Scrum better than attempting to turn it into static/fixed releases - after all the entire point of pretty much all iterative development processes is the acknowledgement that we do not know how to precisely estimate software development with sufficient accuracy to create precise estimates for specific sets of tasks.

If you want to put a name on it, Pivotal is just a type of scrum. They have their own sauce. I personally called it the Pivotal Process... but I'm probably the only one.

At Pivotal, the forecast of what will be delivered changes all the time and it is tracked in a single tool that everyone can look at and keep up to date with (along with a few short meetings to keep everyone on the same page).

> They have their own sauce. I personally called it the Pivotal Process

What Pivotal does is fairly vanilla Extreme Programming. Indeed, the vanillaness of it is something of a selling point. In the London office, we would start projects by giving client PMs and programmers a copy of Extreme Programming Explained and telling them that was what we were going to do. It's easier to coax them into following a process if it's a documented industry-standard one rather than some special sauce.

Pivotal does have a few extra bells and whistles that fill in spaces around the XP core - "discovery and framing" [1] and "inception" [2] being two that spring to mind. And as you say, there is less emphasis on iterations as a unit of work - the planning meeting is weekly, but it feeds a continuous flow of development and delivery. But the daily pattern of work should be familiar to anyone who knows XP.

[1] https://content.pivotal.io/blog/what-we-did-on-our-summer-in...

[2] https://www.infoq.com/articles/project-inception-meeting/

I was told that the main thing that makes Scrum Scrum is that the team chooses the sprint goal, from amongst the tickets at the top of the backlog, and the team commits to hitting that goal.

In my experience more typical is that the PM chooses a set of tickets and asks the team whether they can commit to them.

According to the constraint above, neither that, nor the “pick from the backlog” process you’re describing would qualify as “Scrum”.

I doubt it’s luck. In every place I’ve ever seen Agile in use, they treated 2-week sprints like you describe iterations. It was generally expected that portions of stories would roll into the following sprint and there was a high degree of trust and autonomy in letting teams organically decide if & when any stricter deadline needed to be imposed.

One week sprints are also fine, but I’ve always found everyone hates them for doubling the number of planning / backlog / etc. “process” meetings.

One month is often too infrequent for these meetings, one week is too frequent, so 2-week sprints appear from this sort of naturally. As long as you don’t conflate the timing of the sprint with the timing of work tasks (eg accept there are single work tasks that cannot be chopped up into two week incremental subtasks) this seems fine.

The problem I've seen over and over again, and others have mentioned here in this great thread, is that when you have a 2 week sprint... if a feature misses the sprint, it is another 2 weeks before anyone sees anything... so now it is a month, which can be really unfortunate.

It gets especially bad if there is a lot of fire fighting happening as well because someone might get pulled off that feature and now the time goes to infinite...

The idea is to try to prevent that type of common delay.

That’s not how I’ve seen it work anywhere. If you do 2 weeks of work and the feature is a few days away, the portion of work that continues into the following sprint is just a few days, and then you see it.

Fire fighting is just as likely to prevent completion in one sprint as the other, so there is no net change in how the effects of context switching for firefighting slow progress.

Of course, restructuring teams and work so that firefighting effects are reduced is always the goal, I’m just saying it doesn’t matter at all in terms of when the arbitrary cadence for syncing and planning are chosen.

We have tried 1, 2, and 4 week iterations. The problem with 1 week was that we didn't feel the planning and retro overhead was worthwhile for such a short amount of work time. 2 weeks was our sweet spot. 4 weeks was just too much time between larger re-orientations.

Sounds more like a sprint than an iteration since iterations are 1 week.

As I mentioned already in the thread, retros are not just about the work... it is a check in on what went good, meh, bad during the week. It can be about anything, not just work. The idea is to have one hour at the end of a week closeout team bonding where anything can get laid out on the table.

Your iterations are 1 week. An iteration could be any length of time. And retros are exactly what you said. During our retros we have talked about iteration length multiple times and tried varying ones. In our retros everything is on the table for discussion, but it sounds like that's not the case at Pivotal.

'Your' is Pivotal. I'm just talking about how they do things and what I learned there.

At Pivotal, an iteration is a week and it doesn't make any sense to make it longer than that. What would a longer than 1 week iteration really be? Are you making them longer only because you don't want as many retros?

I said: "The idea is to have one hour at the end of a week closeout team bonding where anything can get laid out on the table."

You said: "In our retros everything is on the table for discussion, but it sounds like that's not the case at Pivotal."

There is a disconnect somewhere cause I'm pretty sure we are saying the same thing. =)

> At Pivotal, an iteration is a week and it doesn't make any sense to make it longer than that.

It might. For example, if your team is distributed (eg development in the Pivotal office, infrastructure in the client office), you might prefer to have retros once every two weeks, rather than either drag the infrastructure people across town every week or have retros remotely.

On some projects i anchored, we had planning meetings twice a week, because planning went more slowly, relative to development, than usual, due to complexity of features, multiple stakeholders, etc. We still had retros once a week. Was our iteration one week or half a week?

The truth is that iterations don't really exist. You have planning meetings at some frequency. You have retros at some frequency. You have client liaison meetings at some frequency. You may have various other meetings at various other frequencies. Sometimes the frequencies are the same, but it doesn't mean anything.

I came across as argumentative in my first response. I didn't mean to :)

> There is a disconnect somewhere cause I'm pretty sure we are saying the same thing.

Sort of. In our retros everything actually is up for discussion. If people think 1 week iterations may work better for us, we may try them out (and we have). We got rid of daily standups because of a retro discussion (they may come back if we feel communication is lacking). The point is there are no sacred cows.

You already stated a few things that appear set in stone. 1 week iterations, pair programming, daily stand ups, and everyone at the office for the same hours every day. From my standpoint that appears very rigid, and I wonder what there is to talk about in the retro if the development process is static.

Late to the discussion.

Many Pivotal teams hewed closely to the default practices, but not all. The most dogmatic is Pivotal Labs, because its job is to teach the basics. Having a rigid, predictable structure works better to start folks off.

I've seen every sacred cow at Pivotal slaughtered by Pivots and guests, when it was felt that this was the right and effective thing to do.

The end of the sprint should not be a deadline, but a goal. Wasn't that kinda the point of the article?

No one ever said a story has to fit within a sprint, that's not how agile or sprints are supposed to work. Some stories take multiple sprints. You should always break down large stories into sub-tasks that do fit within sprints.

This iteration thing your describing sounds almost exactly like sprints to me, just reworded to sound like their own thing.

Ah, the difference is that with iterations, there is never an 'end' and goals are just epics.

Correct, all larger stories should be broken down.

To be super clear, this is not my own thing. =) It is Pivotal's thing. I am just explaining the process as it was taught to me.

>Ah, the difference is that with iterations, there is never an 'end' and goals are just epics.

Jira uses epics in the same way. And I'm not sure what you mean by "there is never an 'end'" - how does that differ from sprints? Iterations and sprints are just two words for exactly the same thing.

When you say, "devs just pick the next one off the top", that sounds very much like Kanban instead of Scrum. Kanban is radically different but considered "agile". When Scrum degenerates into mgmt screaming, "you missed all your deadlines - again!", Kanban is often a great source of relief, because the only deadlines that count anymore are the "real" ones. It also allows much more flexibility with shifting priorities, which stakeholders love. Absolutely worth a try when Scrum has become a tool of oppression & misery.

I wouldn't really try to put a name on it other than Pivotal Process or Pivotal Methodology... I was just googling around to see if they even name it and found these articles...

https://infocus.dellemc.com/rahul_bhandari/software-developm... (corporate version)

https://pivotal.io/agile (this one bugs me a bit since it really just compares against waterfall)

https://medium.com/@ryanjjones10/what-is-the-pivotal-process... (this one is more accurate)

Okay well it looks like they've put even more emphasis on pair programming, which Kanban certainly doesn't require, and... I'm not gonna advocate for that part.

Pair programming all the time comes from XP. Almost no one does it, not even Kent Beck. In fact, the only place that I know of that tried "real" eXtreme Programming was the Chrysler Compensation project, which ultimately failed.

I'd be amazed that Pivotal actually does pair programming all the time, except that they are in the business of selling tools to facilitate exactly this process.

I've been in tons of agile places that do pair programming.

Many Pivotal teams do full time pair programming. Many don't. Pivotal Labs is pretty close to 100% because of its teaching role. Large parts of R&D practice 100% as a default with many local variations. Other parts of R&D don't practice anything resembling pairing.

All of this is presumably going to be in flux, now that the acquisition by VMware is complete.

> Sprints have deadlines and a set amount of work...

Well, they shouldn't: https://www.scrum.org/resources/commitment-vs-forecast

You don't understand how agile development actually works and appear to be confused about terminology. There is no difference between a sprint and an iteration; they are two words for the same thing. A sprint is a planning cycle, not a deadline. Many other tools such as Jira or Rally work as well or better than PT. Recommend you broaden your experience.

Sounds like every agile team I've ever worked on - sprint and iteration is usually interchangeable.

The problem is that Devs making estimates tends not to be accurate IME. Some stuff is easy to estimate, but adding in new & significant functionality is notoriously usually hideously mis-estimated (either high or low). So much so that the numbers basically b come meaningless and can't be relied upon because every major new thing was a lot easier/a lot harder than expected.

The reaction to this is to split the task up, but this often requires that you do a lot of pre-work-work to architect and design the work so you know how it can be broken up, and that design work needs to be estimated so you are back at square one with wildly inaccurate estimates.

If the story estimation isn't accurate, then the velocity goes haywire and this shows up in the reporting in Pivotal Tracker.

So like you said, devs learn to get better at forcing the PM's to split the stories into smaller chunks. Smaller stories are almost always easier to accurately point since it is always 2 (1 point for the story, 1 point for testing).

The numbers are not meaningless though. That is why velocity is an average of 3 weeks. A single week doesn't blow things out of the water and if it is a new project, the expectation is that we won't know where we are at all those first 3 weeks. It gives a built in ramp-up period for the project.

Agreed, the PM's role is SUPER important. For this reason, at Pivotal the PM's usually have a developer background.

Aren’t you putting all your faith in the PM in this methodology?

The way you describe it, a developer’s role is to point out stories and jointly perform the work the PM has allocated.

Instead, if those same developers understood the users, use cases and metrics and co-created the work the team does, the team would have 10 or mor engaged minds all working to make a better experience instead of the one super PM.

It isn't black and white like that. The team always has a voice and can express it at any time. Communication is always key and that is the whole point of developers pointing stories. They could just call a story 8 points and the PM would never get it into the backlog. =)

Right, but that is different than the PM asking the development team what we should do this iteration.

The PM is the conduit between the business needs and the developers. The PM and the lead team dev decides the backlog. Devs just pick the next story from the queue, but of course they have input on things as well and can convey this in any of the meetings or directly with the core dev or PM.

Yes they can coney things, but it isn’t by default. Having conduits, etc. is a waterfall approach.

There are lots of devs who have a personality that benefit from having a leader (PM).

Also when you are doing the 101st CRUD type function in an app, you probably cannot be that enthusiastic about managing yourself either.

How can this state of affairs where developers grind out CRUD features and complain about how bored they are be stable? The identifying feature of areas where automation would be profitable is human boredom, and this is something where developers are the ones bored. Why hasn't all of the repetition in CRUD stuff been dealt with by now?

Saying this is saying like "why do we need architects to design houses if all of them are just four walls and a roof?" Software engineering is not that different from mechanical or civil engineering or whatever other kinds of it. However you automate any design process, you'll still need engineers to properly utilize that automation, customize, verify, maintain (oversee manufacturing or software deployment) and fix problems.

CRUD has been automated a long time ago. It's trivial to create forms with 120 fields based on scaffolding and automate the DB access with an ORM. It is time intensive to actually make software that is better than the paper equivalent. The scaffold puts everything on one page so you split it up into 10. You still have to interact with other systems to autofill form fields. Implement app specific access controls (only users in the same department can see each others forms, only X can actually edit). Implement the actual workflow (ask for digital signature from superior, send email, send end result to SAP or whatever).

Teams often work this way in practice. The product owner has ultimate responsibility for choice and priority of stories, but like any competent leader, they will exercise that responsibility on the advice of their team.

If the team's programmers are "product-minded" [1], then the PO will be getting lots of input from them, and in a healthy team will be going along with most of it - the PO is in a "first among equals" kind of position. If the programmers are not like that (junior, new to the domain, just not built that way), then the PO will make the product decisions without them (but still with the support of a product designer, hopefully!).

There's an excellent leadership principle in the UK armed forces (and i'm sure elsewhere) that a leader must consult their subordinates before making a decision, but that once a decision made, subordinates must go along with it. The PM - programmer relationship around prioritization is an example of that.

[1] https://blog.pragmaticengineer.com/the-product-minded-engine...

In my experience, there are a few reasons why you would choose sprints over a continuous backlog like this. Originally, these kinds of agile processes did not use continuous deployment. Often you used trunk based development (TBD) where you merged your work directly into trunk/master several times a day. Continuous Integration servers were invented because this style of CI often broke the build. You wanted the test suite to run after you pushed so that other developers would be alerted to a broken build before they pulled. Of course even better to check before you merge, but the source management systems of the day often weren't capable of doing that easily. Sometimes even making a branch was hard work. The end result of this kind of system was that master was not always deployable. You needed a point in time where the system was deployable. That's the end of the sprint. Although it's a very unusual way of doing development these days, prioritising true CI over CD has definite advantages in some environments.

In the same spirit, the longer you keep code out of trunk/master, the more risk you have and the more difficult it is for people to do refactoring. If you make large changes over a whole week, then you end up with difficult integrations. Even if you aren't doing true CI (i.e. actually integrating your code with everyone else's on a continuous basis), it's important to limit the amount of time your branch is held out of master. If you have a 2 week sprint and you have 1 week stories, you will almost certainly fail every single sprint (been there, done that ;-) ). The only way for a sprint to succeed is if the vast majority of stories are small enough that you are unlikely to start it without finishing it.

The funny thing about having small stories (around 10% of the size of your sprint) is that if you are doing them, then having a sprint end is inconsequential. You may drop a story or two from your board if things don't go as planned, but you will almost never start something and not finish it (especially if you prioritise stories by risk). So the main reason for preferring sprintless backlogs is because keeping the story size small is difficult.

In order to keep all your story sizes small, you have to be doing some things very, very well. First, you need to be doing your analysis very well. If you don't know exactly what you are building before you start, you will have wide variability in the amount of time it takes. It will also be very difficult to break problems apart because you aren't entirely sure where the good break points are. Second, your code needs to be very, very clean. You need to be refactoring literally constantly. The code needs to be simple in order to break your problem up easily. This requires a lot of iterations of design in all of your code as the code grows. All of this actually feeds back to the preference for CI over CD.

It's not a system that a lot of teams will prefer to use, but it has definite advantages. My experience has been that the potential is quite a bit better, especially over time. You need a team that is completely on board, though. It is especially difficult for PMs who usually don't want to think about problems in as much detail as is necessary for this system to succeed. However, I would encourage developers to keep an open mind because when it works, it works amazingly well.

In my experience with sprints, I see testing get dropped before stories do. Pointing should factor in testing as part of the score.

TDD/CI/CD is critical and these are part of the culture at Pivotal.

To be fair, if you are doing TDD, then you can't drop the testing ;-) But, yep. No matter how good your process is, you can't do well with it if you don't follow it.

Edit: a particularly poorly placed lack of negative.

I don't think this in an entirely accurate history of CI.

Continuous integration originally meant that there was a single physical developer workstation that was considered canonical, and to get your work into source control, you would export a patch, walk over to that machine, sit down, merge your patch, run the tests, do any small changes you needed to get the tests to pass, then commit. If you couldn't get the patch working quickly, you would revert, and go back to your own machine and keep working on it.

Changes were serialized by the fact that only one pair could be sitting at the CI machine. Changes would only go into source control once the tests passed. Master was always deployable.

I once used a slightly more comfortable version of this process where you did the merge and commit at your own workstation, but there was a physical token that had to be taken before doing that, so that merges were still serialized. That worked pretty well!

The transition to the modern approach of just yeeting your changes into master and waiting for CI to go red after the fact constitutes a decay of this original practice. I don't think it's a deliberate evolution.

I take the view that the build turning red in CI should be a show-stopping surprise. But given the sheer volume of automation that occurs now I've seen that standard slowly abandoned from sheer signal fatigue. It's hard to notice a red box when you have hundreds of boxes on-screen and there are a sprinkle of reds.

I'm not sure if I like that state of affairs. It feels like eroding goals or a capability trap.

Hi, PM here. I ditched scrum long time ago with the team and setup 1 week cycles. We meet on monday, to kick off and see results at friday. We managed to keep executable development cycles, so from my part, i tend to ask for less every single weekend because that's how I get most of the results, and keep moving. In the end, it's better than seeing results every 2 weeks and some major results in a month or so. I preffer to move quickly, and iterate those "mini-sprints" in case one went wrong.

this ^^^^^

This sounds like Scrum with one week sprints.

we don't do retrospectives, groomings, estimation of stories, etc.

This actually sounds a lot like Kanban.

This sounds a lot like Kanban for Scrum Teams. https://www.scrum.org/resources/kanban-guide-scrum-teams

It strongly depends on the maturity of the team. All the comments in this thread are assuming the point of view of the heroic developer and his tribulations. Managers know that some devs will sit on their hands and do shoddy work without constant prodding. A good process has the appropriate amount of incentives to move quickly and checkpoints for quality and accuracy. The more mature your team, the lighter these checkpoints can be. I've found that a lot of teams will essentially graduate from sprint-based Scrum to Kanban once they've acclimated and gained some trust. Same with estimates. They may be necessary at first, but eventually become less valuable.

> At Pivotal, I learned that having sprints is silly. They have the concept of one week iterations instead.

So you're doing Scrum with a 1-week sprint?

I find being transparent with my team makes it a lot easier. There are internal deadlines and external deadlines. Former are internal ones which have no real consequence if missed and just keep a development rhythm (e.g. weekly iteration, monthly release, quarterly roadmap update catchup etc...).

The other type comes from third parties, generally, such as clients or the board. Once you separate these two types of deadlines, I find my engineers more understanding about having to meet external ones whereas blurring internal and external into one "deadline" simply causes people to either burnout or lose faith in management because everything is marked as urgent.

I've found most deadlines I've worked with to be internal. The only exceptions to that however was legislation dates; Cookie walls and GDPR, new laws for stock trading on an European level (forgot the name), things like that. These are things for which your company can be fined heavily for. Other deadlines are likely to be of the "we'll start earning money later" variety.

I have one additional type of external deadline:

"The marketing campaign already promised it would be ready by X"

Those are BS as well. No one is going to fine company on day 1 of a new law. GDPR was effective 25 may 2018 but companies still had half of a year to implement it.

Basically if you prove you are working on it I bet they will let you off the hook.

I work on a lot of features lately handed down from the CEO of our client. And he places deadlines so unrealistic his own team oftentimes doesn't get us the details until a couple of days before the deadline. For work that takes weeks to do.

Stuff is constantly pushed back. The India launch so far has been pushed back 9 months. A new product launch in the USA was pushed back 3 weeks - he initially said they were launching January 1st, when they get 2 weeks off for Christmas and no one works on new years day. Lol.

Even external deadlines need to face reality.

It sounds like management at your company isn't doing a good job pushing back on these requests or insulating the dev team from this unreasonable client?

But you work for your CEO, not your client's CEO. Even Steve Jobs called Larry Page, not a random engineer, when he didn't like the Google icon on the iPhone.

I think Agile is the heart of the problem. It is a cargo cult that brings predictability to software projects by ensuring that it's always a dumpster fire, that everyone's stressed, and that everything takes longer than estimated. Eating up "tickets" and crapping out code is not how you quickly deliver software to customers and iterate in the face of changing needs. It's so sad there are generations of developers being raised thinking "Agile" and playing absurd, time-wasting games with "issues" is how to build software. Agile is to the 2020s what the Bobs in "Office Space" was to 2000: "Trendy" corporate management absurdity and fury signifying nothing.

The alternative to Agile is Software Engineering, where you do important things like assess your requirements and capabilities, identify how to validate those requirements are functioning properly and support your business requirements, promote testing and QA to a first-class citizen. Most importantly to always have functioning software that is reasonably demonstrated to be correct. That way you release with confidence, people take pride in their work, and the product, team, and schedule all become quite pleasant.

The Agile you describe is neither the agile I practice nor the one advocated for by sophisticated software development organizations who understand productivity and effectiveness.

The things you describe as “software engineering” are part of any reasonable agile implementation.

Many orgs have done a poor job of implementing agile practices, but throwing the baby out with the bath water seems like a great way to continue repeating the mistakes of those that came before us.

Because, as someone observed in an earlier thread, agile has become a management practice instead of an engineering practice.

Things were even worse before agile, or at least as bad. The alternative you describe doesn't exist at scale, nor can it, because the same corporate culture that made agile suck and waterfall suck will make every other process approach suck in much the same way. The most that is possible are sporadic individual exceptions.

I'm far from an agile-ista, but.

> The alternative to Agile is Software Engineering, where you do important things like assess your requirements and capabilities, identify how to validate those requirements are functioning properly and support your business requirements, promote testing and QA to a first-class citizen. Most importantly to always have functioning software that is reasonably demonstrated to be correct. That way you release with confidence, people take pride in their work, and the product, team, and schedule all become quite pleasant.

That just sounds like Agile...

Like, legit just sounds like the Agile Manifesto.

Honestly, if anything is likely to be the issue. It's Scrum. Scrum has the cargo cult. Agile just seems to get drug through the mud by everyone that implements some authoritarian variant of Scrum.

But again, not exactly a proponent of Agile here anyway.

    That just sounds like Agile...

Agreed!

    It's Scrum. Scrum has the cargo cult.

What do you see as the problem: Scrum itself, or the cargo cult of folks who do it badly?

Scrum done properly is very low overhead, at least from a developer's standpoint.

At my old job we did fairly strict Scrum and I estimated that it consumed only about 4 (so, 10% of a theoretical 40-hour week) of my hours per week.

I feel that is very low overhead. I don't know of a lot of development scenarios where you don't need to spend at least 10% of your time communicating with management.

I mean, people really have to think about what their actual dream scenario is. Some kind of world where they only have to spend 1% of their time interacting with management? 0%? These people need to get real. Management/stakeholders exist and you are going to need to communicate with them.

I have had jobs where upwards of 50% of my time was spent in seemingly endless meetings, plus "drive-by shootings" where management randomly stopped by and wanted to see progress and dump new shit onto my plate. Scrum is not perfect, but for most workplaces would be an upgrade.

    Agile just seems to get drug through the mud by 
    everyone that implements some authoritarian variant 
    of Scrum.

I feel it is close to the opposite.

"Agile" is a nebulous term that has lost nearly all practical meaning. Bad management does some weird, loose version of "agile" (usually with some trappings of Scrum) and gives both Scrum and "agile" bad names.

> What do you see as the problem: Scrum itself, or the cargo cult of folks who do it badly?

Scrum itself. Sorry, but when you create a list of ScrumButs and indirectly deride everyone for not being able to take "Full Advantage of Scrum" so that your purists can go out and attack it at full force, yeah, that might be the problem.

The idea that different organizations might actually benefit from not adopting a methodology wholesale isn't that farfetched.

You could argue that's part of the cargo cult, I'd agree to some extent. But it becomes difficult to separate the 2 when you're literally derided for not following it to the religious T.

I guess you have had different experiences than me. We did fairly strict Scrum at my old job for a few years.

I never felt "attacked at full force" or "derided" even though we didn't follow Scrum to a T.

Never saw that "ScrumButs" thing until I Googled it just now.

I still don't feel attacked or derided. I've talked to dozens and dozens of developers over the years with various feelings on Scrum and never experienced those feelings.

But, you do, and it is 2020 so all feelings are valid. I'm sorry you've had bad experiences. People should not be jerks to you about Scrum or anything at all.

Engineering-first culture is almost always the wrong idea. It's product-first or nothing. The entire point of agile is prioritization based on business value. Everything always takes longer than estimated regardless of how you manage your project. That's actually a core tenet of agile and one that stakeholders (and budget holders) have the hardest time accepting. Having worked in this industry before agile was prevalent, I can assure you it was never better than this in the past. QA as first-class is a pretty core principle as well. The point of things like Scrum is meant to be visibility and alignment rather than prediction, but impatient people are guaranteed to ruin your life no matter what.

I too have been around before Agile took off, and I too recall how dysfunctional it was before that. I do, however, attribute the positive development to better tooling: Git (and GitHub) instead of SVN/CVS/Nothing, Cloud computing and containerization rather than dusty-server-in-the-corner-with-curmudgeonly-sysadmin, popularity of safer programming languages rather than dominance of C/C++/early-Java, StackOverflow and full ecosystem of answers to every question you might have, etc.

In general, my experience has been that the most wildly successful projects I have worked on tended to involve the least amount of stress, most amount of personal/professional development, and best relationship with peers. The least successful ones tend to be stressful, frustrating, and destructive. The common denominator is that the most successful ones rely on experience, intuition, and professionalism of your staff, the least successful ones submit to tyranny of agile - and when agile isn't delivering success, to double down on it and all those awful "issues".

I'm sure I don't have the wealth of experience you fine people have. But I've been in enough teams and companies to see that whatever methodology is being embraced, it seemed like the X factor was always what kind of people are practicing it. If someone likes always being in meetings, you're going to be in a lot of meetings. If you're on a team with smart people you enjoy working with, with open lines of communication, then the rest just seems to fall into place as long as everyone is doing their job.

Then again I never worked for Hooli.

Yeah, this is actually my same observation. High-functioning teams were naturally agile even before it was called that. Defining and formalizing agile practices is a way for people who've never experienced it to know what success looks like.

Without turning this into a No True Scotsman, I don't think the problem is Agile. I think the problem is that most companies have no idea what Agile is, and they just implement a subset of its ceremonies in a highly malignant way and pat themselves on the back.

If you have 10 hours a week of mandatory meetings and are asked to make projections about features and deadlines 6-12-18 months into the future, you are not Agile.

> So the developer, being a developer, pulls a couple all nighters, maybe works over the weekend and does get it done.

And does minimal testing, skips documentation, adds some tech debt, copies-and-pastes code rather than doing proper code reuse - because the deadline is the only thing thing being measured.

At the company I work for, our yearly performance reviews are based partially on the metric "Complete 90% of uninterrupted development sprints with no carry over." What I see most of the time, is developers hurrying towards the end of the sprint to meet this arbitrary deadline, choosing the easiest path A instead of the correct path B. Now the project has been run like this for years, you can imagine how much tech debt has accumulated.

Sounds like a great way for the company to make people work hard (not smart), product mediocre results and not have to pay out bonuses. Have you considered either improving the process or finding a new job?

This is a good example of Godhart’s Law (“When a measure becomes a target, it ceases to be a good measure”) in action.

There are better ways, but it requires enlightened management.

Sorry for the typo: it's Goodhart's Law, not Godhart.

Right, "agile" has become a developer metric system.

I've seen many management fads. They come and go. They go because everyone tires of them and they get gamified anyways. At Sun I heard many people say things like "I'm going to code myself an SUV" whenever we got a new gamifiable management technique.

Sprinting too shall pass.

I think pretty much everybody is missing an important point of deadlines: money. If you are not done by the deadline you are dead. Management (with marketing) has a good idea of what will sell, fail to make the deadline and the product won't make money in the market and should be canceled early. This is the point of estimates: to decide early if the project is worth paying anyone to work on. (or course once you are part way in the project sunk costs become an issue)

Note a result of this is deadlines are not something you should come close to: a break-even project isn't worth your time to develop, so there should be plenty of slack in the schedule to cut into before hitting the point of no profit.

That's a extremely high faith view of management's ability to predict the future.

It's not really faith, it an educated guess: CEO (or Director of Department) - Makes an educated guess, that if they can release Product A in next 18 months, they will earn $1,000,000. That means that they need to know that A: 1) Product A will be released in next 12-14 months. 2) They will spend less than $700,000. Otherwise they might be better off buying bonds or something.

It's effectively a job of an exec / director to make bets. Hopefully they are educated guesses, but there are times when somebody just have to flip a coin too.

There is a lot of education in those guesses. We know what previous sales are, we know what industry sales are. We know population growth rates. We know many other things. Everything we know puts bounds on what we can sell.

> make bets

this is literally why a significant portion of high-end intellectuals refuse to participate in this charade, and instead build quality technology in the open. Open source technology, built largely without artificial deadlines and fake drama, define the execution environment that this exchange is taking place upon (in many cases).

The old industrial plant planning algebra in 1950s textbooks, say something like "if we make widget N in six months with a team of 6 and $100,000 budget, we have a 30% chance of breaking even in two years, but if we make widget P in 12 months with a team of 4 and $20,000, we have a 60% chance of breaking even in 24 months" etc.. this is slightly useful in certain environments, but anyone here is going to argue that the "future" in 12 or 24 months is stable, technically?

The capacity of certain mental models to "Reductio ad Absurdum" is amazing to me, in an age of moving literally one billion+ conversations at once across wires and through space.

In finance, it's called risk appetite. Different people and different entities have a different appetite for risk, and they will act accordingly. There is nothing intrinsically wrong guesswork if parties understand the context, and don't lie to themselves.

>12 or 24 months is stable, technically? Hm. Faster iterations are better, usually, again if we understand the context, but I see a lot of companies that are too quick to go into full panic mode. An example would be recession – If you plan to invest for the next 60 month, and a recession hits you on month 20, the right choice would be to ignore the inputs, and keep investing. Those who pull their assets when prices are low usually lose.

Quick or long iterations are not be all solutions, once always have to keep the context in mind and use first principles to understand what is that we are doing, and trying to achieve.

Given unlimited time, a team will ship more labor hours, that’s it. Constraints matter to getting shit done, and time is the most precious constraint.

GP's comment is underrated. Engineering is about effectively managing to make something within constraints. Memory, CPU, I/O bandwidth, network outages, screen size, all constraints that engineers work within. Development time should be treated as a first class citizen among those constraints.

For example, if you're working on embedded software that gets flashed onto ROM as part of a device's trip down the assembly line onto a boat, and the factory is ramped, you can't just say, oh we just need another month because our estimate was way off. Something is getting flashed onto those ROMs starting Day 0. If you're an engineer working on this software, this constraint needs to be taken as seriously as the OS API or memory footprint.

> to decide early if the project is worth paying anyone to work on.

Yes, we know that’s what you think. We keep trying to explain to you that you’re living in a fantasy world where software development can be predicted accurately in significantly less time than it takes to just do the work. I understand why you WANT that to be true, but wanting something to be true doesn’t make it true, to the frustration of MBAs all over the world.

I do think well-functioning development processes & teams, with good integration with business is something every company wants, but very few companies are willing to pay for, and most just cannot accept that they're opting out of what they claim they want by not paying enough, so they try to get it anyway.

Why's it expensive? I think people who are capable of building and managing such teams, and dragging organizations into a set of processes that truly work, exist at a rate of about 1/1000 the demand for them. Meanwhile the people who can discern useful practices from useless ones or good implementation from bad are similarly rare and overlap largely with that same group, leading to an ecosystem of development practices that's almost entirely snake oil and wishful thinking, and where almost no-one can tell when they're looking at one of those 1/1000 folks who've got the knowledge and skills to make it work if you give them the authority, or some delusional schmuck or huckster.

The best most companies can do is to learn to live with a ton of uncertainty and work humanely and effectively with that, and few achieve even this lesser outcome, which tends to come with its own (lower) costs in hours and less-than-ideal predictability that many orgs just can't accept they have to pay to keep things running even sort-of smoothly.

Fully agree.

Honoring deadlines also makes you predictable.

I once managed a handful of products for manufacturing. Our customers planned their down time, upgrades, etc around our releases. Defects, delays cost our customers real money.

I work in consulting and this is the constant knife-edge we walk on. We scope and plan a project and then have to attach a budget to it in a contract knowing full well that our estimates are going to be wrong. It's such a struggle to get clients with fixed budgets to understand how they have to flex on scope and timing and it's not just because we're slow and lazy.

There are various types of reasons for deadlines. I have seen everything from very good reasons (an external API has to be replaced before its sunset date; regulatory changes that must be met when they come in place) to very bad ones ("$manager said it should be done by $date"; when asked $manager why, the answer was "I thought it should be doable by then, but it's not actually that important").

A deadline without the information of its importance is pretty useless, unless you happen to be in an organization where all deadlines are there for good reasons.

> just go talk to the stakeholders - product/sales team - ask them if the delay could be potentially customer losing

In my experience, sales team rarely have any incentive to help out here. They almost always say that delay will create problems and they quantify in terms of dollars e.g. If X is not done by this date we will lose Y dollars. That Y number is almost made up to make things more urgent.

Actually, I have seen sales blaming lack of sales numbers on engineering because the delivery was "too" late.

Agile works well when you’re working in a win win kinda relationship. What you describe happens more often than I am willing to admit.

What helps here is if you hold Business Value meetings with real people involved and try to reason relative priorities of stories. This is the fuzzy, social aspect of Agile development but I suspect it’s really the core idea behind the whole thing: it forces people to talk in person and empathize more, hopefully building up cooperative relationships rather than extractive ones.

If the stakeholders continue to be sociopaths though, you can either chose to give in and do whatever they say or lie. Both options break the spirit of the thing. Frequently there’s a 3rd invisible option where you use back channels to pull some influence. Which also defeats the purpose of Agile...

> If X is not done by this date we will lose Y dollars. That Y number is almost made up to make things more urgent.

Sometimes it's real though.

Clients might have to make a vendor decision by the end of a quarter for example and want a bug to be fixed or a feature to be added before they sign.

In other cases, there might be a huge event and ad campaign where you have a limited timeframe to make a good first impression on a lot of potential customers.

Deadlines suck but in certain cases they're very necessary.

There is a fundamental relationship between schedule duration, amount of work and quality.

If estimates were all accurate, then the amount of work and the sprint duration would be the same. It inevitably takes more work than estimated, so something has to give.

If you constrain the duration and the scope of work, then quality will vary. If you constrain the duration and allow overtime, then you get a mini crunch time every two weeks. Some bosses think that people will slack off unless they are given pressure, so they like this situation.

A better way is Kanban. It's more or less Scrum where quality is the thing that's fixed. If the task is going to make more time than originally estimated, either you let it take more time or reduce the size of the feature, i.e. implement a simpler version of it.

> If estimates were all accurate

then every level of management between the devs up to the CEO could be replaced with a spreadsheet.

This is what Steve Jobs has to say about project managers: https://youtu.be/rQKis2Cfpeo?t=116

Well that hits home pretty hard. I didn't know Steve Jobs did these kinds of interviews, thanks.

Another gem: Steve Jobs on sales and marketing, https://youtu.be/-AxZofbMGpM?t=21

Isn’t that about managers in general?

It is about preconceived notions about what project management should be.

I think many practitioners of Scrum would fall into what Steve Jobs described in that video as bozos.

Scrum is riddled with bozos.

Talking to "product managers", managers and various stakeholders everyday, the problem is at the times when these otherwise good people are disconnected to the rest of the organization and have no operational clue. Then they will demand deadlines, push dates backwards, do all sort of non-intentional self-sabotage to the org. They might get their thing done, badly, while hurting the rest of the organization and creating unhealthy competition. When goals are accomplished, they often are unable to even get the value out of the thing they wanted in the first place. These same people won't be able to prioritize when given the choice, and basically just want it all now, again like little toddlers, with checklists!

So what do you do? Only healthy thing is to push back, give explanations for what is required before work can begin and have good relationships with all parts of the org yourself. Insulate the toddlers. If they need a PM, they gotta provide a budget.

> We'd still be doing MVC in frontend apps if someone at Facebook didn't miss a deadline.

After working with React for some tine I'm not so sure the good old Mvc is a bad thing, not always at least.

That line stuck out for me too.

Is MVC as a pattern really all that bad for the majority of web apps?

I think it depends on what kind of code mess you prefer - client side or backend. I personally prefer to mess up on the backend, feels much more comfortable to me to fix it up and write tests.

deadlines exists because managers believe that the team will focus to accomplish the goal. the reality is that _most_ managers are lousy at estimating the amount of time a given task takes and _most_ developers are cowards to stand up to a manager and tell them this. _everyone_ sucks when it comes for planning for the unexpected. in the end, all deadlines do is cause unnecessary stress and disappointment for _everyone_ involved.

i'm no different and have been caught up in this game more times than i can count, yes, i suck too. later in life i've come to determine, that whatever deadline i have in my head, i just extended by a factor of 2.5. if i think it will take a month, i say two and half months. that is more than enough time to not only get the project done, but also deal with anything that crops up unexpectedly. my serenity is worth more to me than looking good or being a hero.

IME, if the manager is primarily coming up with the estimates, rather than the team, you're doing it wrong. (And are doomed to an eternal cycle of blame, stress and lack of trust.)

You don’t know how this game works. Of course the devs come up with the _initial_ estimates - they have to, or else it wouldn’t be possible to blame them when they end up delivering “late”. But those initial estimates are always “negotiated down” to whatever looks best on the manager’s quarterly bonus review - they might not be brilliant at designing products or scheduling, but they’re geniuses when it comes to re-stating things so that they sound different but are actually the same until you finally get tired of arguing with them (I mean, it’s tricky to argue with somebody who decides whether you get paid or not) and agree to whatever they say and just disappoint them later.

We tend to try to do any vaguely longer term/important estimation as a team exercise, which ends up with people talking each other's estimates up, not down - you explore the problem space more effectively as a team and a lot of "but what about X" stuff comes out of the woodwork then.

If you have a manager who sets your team up for pressure and failure by artificially revising your estimates down, without cutting the scope of the work, then personally, I would find a new manager. Source: Am a manager.

Lack of courage to stand up to the manager might be the case sometimes. But sometimes it's also just realizing that it is futile because they are not open to it hearing it.

I remember one meeting where I watched an engineer make a good faith effort to estimate time honestly and accurately, they said the number, and then the manager came back with a time period about half as long.

In that situation, the manager and engineer viewed the discussion in completely different, incompatible ways. The engineer saw it as a process of attempting to find the answer to a question about the world. The manager saw it as a negotiation. It looked like the engineer was frustrated with the manager for trying to treat facts as negotiable. And the manager was frustrated at the engineer for not being aggressive enough.

I'm not sure what the solution is. I'm sympathetic to the manager in a way because there is a natural tendency to ask for lots of time. Why wouldn't you, if you can get it?

I have been struggling with scope creep for deadlines at my work, I am a junior dev and I have a product owner that means well but changes requirements several times and increases scope mid sprint. How do you push back against this in a nice and reasonable manner?

Document the truth.

If you are employed and a product owner is required to give you your workload, you are kinda of stuck. The best you can do is illustrate (over time) that you are unable to get things done because of changing priorities.

Put together a collaborative document where you write what you understand to be your tasks. When a task changes or a new one is added, add a new entry with a date/time. Invite him to see it and ask him to bookmark it because you want to be clear on what you should be working on.

This is a good thing to have around review time as well, because you may be confronted with 'It doesn't seem like you are very productive'.

Also if it is possible that your priorities are not changing as much as you thought, this is a good exercise for yourself as well.

Be open about the challenges of context switching and how that may lower your overall productivity.

Use specific examples (so everyone can learn) and avoid making about people by focusing on the jobs to be done.

If you're not comfortable bringing this up to boss directly, shoot through a Sr dev first (who may coach you around the issues / validate your concerns), or, Sr dev may speak up when needed to support your cause should you bring it to the group/boss.

There’s an old saying: “people will forget what you said, people will forget what you did, but people will never forget how you made them feel.” It’s supposed to be inspirational, but it’s actually depressing because it’s so accurate. How do you put it into action as a dev? Never argue. Ever. If you argue, you’ll lose the argument anyway (you have no power), and they’ll just remember you as the person who was argumentative. If they say, “can you add AI to this product by Friday?” You say, “yeah, sure, let me look into that.” In a sane, rational, just world, everybody would be working together on a shared, common vision, and they’d realize that by giving you herculean tasks with impossible deadlines that nobody could achieve they’re actually hurting themselves as much as (if not more) than they’re hurting you, but the grim reality is that that isn’t the world we live in. If they add scope, just shrug and accept it. Don’t kill yourself with nights and weekends trying to meet their deadlines either - just say, unapologetically, “I was able to do ___ and I’m still working on ___”.

Don't push back. That is, don't try to stop them doing this. It is absolutely within the product owner's purview to change and enlarge requirements at any time.

But make it clear to them when they do this that there is a cost. "We estimated building the profile page based on doing it in one language; we can certainly internationalise it now, but it will take quite a bit longer". It's then up to them to decide whether the cost is worth the benefit.

As long as you are working as well and efficiently as you can on the work you have been given, you can feel satisfied with yourself.

Unless of course you're being judged for schedule slips caused by your product owner, in which case you're screwed, and you should quit and find a job somewhere less awful.

"This is not what the story said when we started the sprint. Could you please make this change new story for the next sprint?"

I tend to take a fairly dynamic approach to design.

In many cases; especially where you have heterogenous projects, done by multi-disciplinary teams, dated milestones are pretty much required. You have to have some thing, by some time. I worked like that for many years, as part of massive, multi-national teams.

But these days, I have the "luxury" of generally working alone, or as a member of a casual open-source team that doesn't have any specific deliverable deadlines.

What I do, is define what "done" looks like, and set some very fuzzy milestones. At each of these inflection points, I reexamine the work I'm doing. In some cases, I decide to bin the whole effort, and either rewrite it, or simply scrag it, and move on to the next in the FIFO. In other cases (less often than I'd like), I am ahead of expected schedule, or the functionality of my work is better than expected. Boo-Ya. Go Me.

The important thing for me, is that I have a solid idea of a deliverable. It may not be associated with a date, and may be subject to revision, but it exists. At some point, the project needs to be released.

That may require some compromises, and it will require some tedium, like signing and provisioning, as well as producing marketing copy and graphic assets. That's part of what you need to do to have shrink-wrapped releases.

A lot of times, I can put off some of the sexier stuff for a 1.1 release.

The essay is just a long winded way of saying the quote from Eisenhower: "Plans are worthless, but planning is essential"

Which is of course a short winded way of saying that the article makes a lot of sense. We for example have sprint goals - these are things that the team commits to actually getting done during the sprint. Then there are tasks in the sprint which may or may not overlap with the sprint goals and may or may not get done during the sprint.

It's always about trade-offs.

Reading HN comments makes it sound like engineers think they're the only ones in a company who have to deal with deadlines. It's kind of hilarious. Go ask your sales team how they cope with hard deadlines of closing a specific dollar amount every quarter based on the presupposition that 1. the product delivers what it says without major issues, 2. competitors don't come out with feature or pricing changes that blindside you, 3. other departments (marketing) delivering them enough interested contacts to talk to... and 100 other things. Not to mention the main "stakeholder" in play for them -- the customer -- is ACTIVELY trying to deceive the salesperson in order to get the best deal for themselves. Like, they will literally lie and say they intend to sign a contract with you by end of quarter to get the salesperson to go the extra mile for them. With zero intention of actually following through... Yet the sales team gets it done every quarter. And if they don't? 2 bad quarters in a row = expectation is you're out of a job.

I don't think the main point being made is that deadlines are a PITA, I think it's that the trade-off is unmeasured. Successfully meeting a sales deadline looks the same as signing the same contract after the deadline (other than the deadline), but the argument for deadlines in software being bad is that they materially erode quality and long term viability (shortcuts, tech debt, bugs, usability). The product delivered within the deadline is materially worse than the one delivered after. The problem is that it's hard to attribute these issues to "deadlines" although the intuition is that they are a major cause.

> Successfully meeting a sales deadline looks the same as signing the same contract after the deadline (other than the deadline)

This is so far from the truth. I encourage you to spend more time with your sales team and even your stakeholders who are buying $100k+ software from vendors. There are so many parameters of a deal that go beyond contract signature date. Deliverables, SLAs, roadmap commitments, length of contract, legal definitions, opt-out conditions, payment terms, pricing, and more. Deals aren't just "on time" or not, they vary greatly in terms of quality. And that quality largely determines your company's ability to hire more engineers. Or fire them. ;-)

Bad deals happen when you're down to the wire and have to get it done on time or people will lose their jobs. Not unlike bad software. Work in the real world has deadlines. The people doing the work will always advocate for the loosest deadlines (or none at all, as is often the case with engineers). The people who need the work done will always advocate for the fastest turnaround. Without these checks and balances, the work just does not get done fast enough or well enough to stay competitive.

Deadlines are for managers. If developers need to be aware of the deadlines then the managers are not doing their job.

Maybe - for many product inclined engineers, participating in picking and managing deadlines is an extension of authority and autonomy.

Additionally, some people need deadlines imposed to finish work. The alternative would be to fire them.

I don't quite agree here, developers need to have an idea of what they're working to so they can manage their own implementation time. If you don't make developers aware, you put too much control in the manager's hand which isn't a good thing here (unless you're referring to a developer lead but even then, individual developers should have an idea of when they need to complete by).

The post is solid on criticising sprint deadlines, but doesn’t give many suggestions on what works better. Except for this:

> Another idea is to experiment with more elaborate probabilistic ways of setting shipping estimates rather than gut based estimates.

In my experience, teams setting estimates and holding _themselves_ accountable is a great way to go with multiple benefits. This is why I’m a big fan on estimating software projects with a few ground rules [1].

Sprints and deadlines could be a tool to get here - as with a bunch of things, context, people and culture matter just as much as processes do.

Iterating and learning from iterations is how good software is produced, with a team that is motivated and continuously growing.

[1] https://blog.pragmaticengineer.com/yes-you-should-estimate/

One estimation method I found good and worked at a risky project in my job (involved a regulator, they love deadlines) was:

1. Do the usual breakdown of tasks and point game to get relative difficulty estimates

2. Calculate the std dev points for each task instead of just averaging. The assumption is that this is valuable information about how uncertain developers are about the task, so we don’t want to throw it away.

3. Create happy-path and worst-case scenarios considering +-1sigma and +-2sigma

4. Prioritize tasks by deviation, to remove the most uncertainties at the beginning of the project

With this you start w/ a more realistic view of the uncertainty of the project, and it’s supposed to converge to the avg as the project goes.

This article has beef with artificially imposed internal deadlines, not deadlines in general.

A deadline can be for a ton of legitimate reasons, like delivering a signed contract by the agreed upon date.

The only deadline I set are for customers. That is either internal customer because they asked can it be done by x day and I either answered yes or no. Or an external customer because I got on a phone call or someone in sales/support indicated an opportunity or at risk customer.

Ok fine and sometimes I set deadlines because I’m like OMFG how have we not done this yet. Let’s get it done by x day. For example upgrading framework x to latest version y

What was so wrong with the author’s chosen article title that it had to be manually sanitized? I get that HN admins may not like the snark, but it’s the author’s title, it’s his snark, his tone, his article, and by sanitizing the title, you are basically making an editorial decision on his article.

The HN guidelines call for changing titles that are misleading or baity. That one was baity, so we replaced it with representative language from the article body.

Generally we err on the side of preserving articles' original titles and favoring authors above everyone else! But not when the title breaks that guideline.

https://news.ycombinator.com/newsguidelines.html

If it weren't for deadlines I would never do anything, pretty sure I'm not the only one.

This goes for external (outgoing deadlines) as well as for internal. When you don't tell the world to expect exactly this on THIS day, they won't be unhappy. They'll simply be happy when it happens - and happens correctly!

This is a very well written article. TLDR: don’t be too steadfast with sprint deadlines, keep them fuzzy and understand how missing them will impact stakeholders before committing to working over the weekend to meet them.

Many Scrum Fanatics, especially the ones holding various Certifications, will scream bloody murder at the idea of tampering with their precious sprint. But the sprint is a fucking artificial construct that was meant to be fuzzy. If it isn’t good at representing how software is delivered you can either change the sprint or try to force your team to sync with it.

Unfortunately I’ve seen the agile processes be misused in many ways because it’s inherently so fuzzy and ill defined. It doesn’t really make much sense to bikeshed over minutiae but to focus on the outcomes and tinker liberally with the process to suit your teams style. Just keep your stakeholders informed of the changes.

I don't want to criticize but you spell critic wrong on the first period: "crtitic"

Pun intented.

Well, they are user-first... if you look at humans as resources and managers as users.

Part of me wonders if the Author works for Amazon, because that sounds totally like Amazon.

I've done a lot of scrum like projects in the past 15 years. I say scrum like, because absolutely nobody does it "by the book". And if you do, you are doing it wrong because you are being rigid and dogmatic. Every agile manifesto signee will tell you that (I've heard several of them talk about this). I meet a lot of "you are doing it wrong" types. You know the type. Self styled agilists that throw the book at you and seem to love having lots of post its stuck on their walls that provide the illusion of progress in the absence of exactly that.

These days, I prefer Kanban style processes for several reasons. Most importantly, the practice of ignoring new requirements for two weeks (aka. doing a sprint) is not sustainable in most projects I'm on. It's just not a thing. Stuff changes all the time, deal with it now instead of in two weeks. Another reason is that I prefer continuous deployment and automating PM reviews away so we don't block shipping stuff over somebody's opinion. Sprint deadlines are utterly meaningless from a release point of view if you release multiple times per day. If it's better than what we have then merge and ship it ASAP. If it's obviously broken, obviously fix it now ASAP and don't wait to have a meeting about it in 2 weeks. Do that multiple times per day. Have responsible processes to ensure bad stuff gets caught before it is merged. It works. Shipping early means you catch stuff early and get feedback from real users rather than opinionated PMs and managers detached from reality. Also people stop throwing things over the fence when they are confronted with the consequences of what they did right away. It forces them to think end to end, a quality that I find is often missing with engineers because they are being shielded by layers of management.

Finally, I work with remote developers a lot these days. Anything involving telcos or meetings is downright painful; especially if you add language and cultural barriers. Having full day scrum ceremonial crap is just not something I have a lot of patience for. Doing that over a hangouts meeting is just an exercise in futility.

Kanban of course does not stop you from having regular planning meetings, estimation sessions, retrospectives, etc. All that can still work and if it adds value to your team, by all means keep on doing that. IMHO it makes less sense in distributed teams and those require more asynchronous approaches. I find decoupling what we do from when we do it in a Kanban style to be helpful. If the backlog is getting too short, add more stuff and add it in the priority you want things fixed. Don't allow your team to lose focus (aka. multi task) and set concrete goals. Have breakout meetings when things need clarification. Don't pile on too much work in too much detail until it is ready to get done. Doing CD means you can specify changes in terms of what you have in production and how you want things changed. I find that injects a sense of reality that just stops a lot of BS. from ever happening.

> “ There is another problem with locking your sprints. Say you get a new customer ask, it's a potentially trivial but low priority bug. The developer could fix it in a day. But you follow The Process TM, the task gets added to your team's backlog, maybe gets picked couple sprints down the line and does get fixed. But you lost the opportunity to delight the user!”

Nope. That context switching will cost you more on average than sticking to the process. And most often you’ll be wrong about the time estimate, or the customer will be wrong about actually wanting that feature, etc., making the context switch even worse and more costly.

Out of all of the factors that could potentially cause poor software (along with the "software disenchantment" as discussed in another recent thread), deadlines stand out as the most concrete and immediate driver of the decisions I take that may result in poor software quality. At least it's the one I can consciously acknowledge as influencing me more than once.

I'll have to improve my efficiency at developing software anyway, but this article has given me more confidence that I can pace the promises I give to my clients as to when features will be delivered.

More 4 year to make Mickey Mouse public domain or the law extent the period again.

Reply on news.ycombinator.com