Hacker News Reader

Comment by nickelcitymario | Hacker News Reader

Like any and every business theory, Scrum has one or two core ideas that are awesome, but could be explained adequately in a paragraph or two. This does not sell books and consulting, so it evolved into a field of its own.

The fact that there is so much B.S. in Scrum doesn't mean that it has nothing of value to offer, though.

Here's what I get out of it:

1. Sprints are a better way to organize than Waterfalls. I've experienced this over and over again personally, so I'm sold on this.

2. It's important to stop what you're doing on a regular basis to evaluate progress and problems. This always seems like a waste in the moment, but failure to do so leads to regret down the line.

3. Ship functional products as frequently as possible. This is better than waiting until everything is done, because you can get feedback early and often from the end user.

Okay, so that's 3 ideas. It's better than most.

View on HN

I think you are entirely right that the "sell books and consulting" is a major driver of what Agile became. I got involved in the Agile world circa 2000, but by 2010 or so my main feeling was increasing horror: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...

One of the especially interesting things to me was that at the beginning there were a variety of different methods. People behind them got together in 2001 to figure out what was common, and that's where the Agile Manifesto came from. Of the Agile Manifesto signatories, I think only two of them were Scrum people. These days, though, most people thing that Scrum is Agile and Agile is Scrum.

I see three reasons for that. One, Scrum was the simplest, arguably the lowest common denominator; it had no technical practices. Two, it could be installed in place at existing waterfall companies doing what is effectively mini-waterfall, so there would be little disruption to the hierarchy. And three, it had a "certification" program, where a) anybody wanting a career bump could spend 2 days to get a "Master" certificate without taking any test or proving any competence, and b) any "consultant" wanting easy money could quickly become a Scrum trainer. Basically, Scrum became the Amway of software processes.

If you go by actual behavior, it turns out the highest priority at most companies is not actually to improve, to get better at making things for users. It's instead to make managers and executives feel like something is being done without disturbing the power hierarchy. Low-end Scrum fills that need adequately, letting you move marginally in the direction of agility, apply some new labels, and declare "mission accomplished".

And my point here isn't "Scrum bad", really. There are some great people in the Scrum world. My point is, "Business models shape outcomes, so be careful which you pick."

> "If you go by actual behavior, it turns out the highest priority at most companies is not actually to improve, to get better at making things for users. It's instead to make managers and executives feel like something is being done without disturbing the power hierarchy. Low-end Scrum fills that need adequately, letting you move marginally in the direction of agility, apply some new labels, and declare "mission accomplished"."

I think this is a key insight, and I suspect for many people (myself included), the phrase you use when you say "Low-end Scrum" is just the only type of widespread Scrum we've ever seen practiced anywhere, ever, to the point that it makes us question whether Scrum itself is partly responsible, even in an idealized format.

One of my good friends came up with the idea he called "barriers to anti-quality" to describe this. A good workflow process establishes inviolable norms that act like preventative barriers to doing things in an anti-quality way -- so that you specifically arrange the world such that trying to take a quality shortcut actually costs you more than just doing it in the quality way.

We proposed trying to judge workflow and management practices based on how willing upper management was to give up control of the process in exchange for erecting barriers to anti-quality.

If they wanted to force a team to pivot in some quality-destructive way, just on their whimsical preferences for reprioritizing different tasks according to vague business needs, they could not do it.

Barriers to anti-quality would be like Odysseus pre-committing to be tied to the mast, so then when the Siren song comes, he cannot give in the temptation.

I think a lot of the spirit of early Agile was meant to suggest something like this. Even though the method talks about being flexible and adapting when there are real business needs to adapt to, this gets bastardized when middle and upper management conflate adapting to real business priorities, and begrudgingly paying a cost in terms of quality lost due to pivoting developers, with demanding that people have to adapt to the management flavor of the month, unsubstantiated ideas about business priorities, often stemming from political power struggles, restructurings, quarterly earnings, etc. etc.

What I really wish Scrum or Agile had, was a more unbreakable way to let development teams say no, and decline work or decline pivots. Basically to veto changes made to the backlog, and say that it violates an anti-quality principle for no good reason.

A key failing of modern Agile / Scrum is that it doesn't create any type of stop-increasing-our-entropy-with-unsubstantiated-management-preferences notion of a barrier to anti-quality.

Don't get me wrong, this is hardly a failing that us unique to Agile / Scrum. But I think when people talk about why in principle these methods fail, it's that they lack any mechanism of enforcement for this. As a result, they can be so easily politically subverted to just be a pivoting, bludgeoning tool of misguided or unexplained management choices, that almost always trade-off quality for anything else, and instead of ever looking back to understand when an under-investment in quality actually cost them, they just explain it away as some other team's fault.

In a sense, Agile / Scrum (or 'Low-end Scrum' as you call it), is a tool to create managerial Dutch books. A manager gets to unilaterally disrupt on-going investment in quality without ever being held accountable when that gamble on short-term anti-quality causes losses.

By the end, it makes developers very cranky that they are being jerked around with all this prescriptive and time-wasting overhead of meetings, planning, estimating, etc., only for this political anti-quality nonsense to go uncorrected anyway. If that's going to happen anyway, then what's the supposed value of all that costly Agile-specific overhead?

Would love to hear more about practical examples of "barriers to anti-quality". Would this be things like e.g. "Emplace and enforce documentation & project change control process" -- people may want to make changes - but if they don't go through the change request process - they can't have those changes effected?

One example which I actually used to great effect in a previous team was that the team decides the relative allocation of backlog items between new features, fundamental research, maintenance, and refactoring / architectural design. The product owner, no matter how upper management is breathing down their neck, is not allowed to supersede that allocation.

Once we got approval for this, it meant that product owner had to focus on communicating to us why the business had certain priorities and desired certain pivots, instead of just nuking our desired plan of work and superseding it "from on high."

It meant everyone had to trust us as the team that we wanted the business to succeed! (Who'd have thought!) And we weren't going to act childish and just demand backlog items and sprint work that satisfied our preferences for some fun project or some back-burner idea. That, as grownups, we would actually consider when business priorities made sense to supersede our intended plans, and act accordingly.

After a while, we exercised our privilege a few times to ignore or decline "urgent" work that the product owner relayed to us. Instead we invested in much needed refactoring stories, fleshed out more serious design documentations, researched detailed trade-offs between approaches or third party tools we would need in the near future.

Over about a year's time span, this worked incredibly well. The team felt happier and more productive. There was less tension with the product owner since he would not circumvent our plans and dictate new priorities all the time. And people learned they could trust us to understand the underlying business considerations and deploy our team accordingly.

In our company, this was very much as "un-Agile" as it gets. It was the farthest thing from what "Agile" methods meant in the company as there could be, and in large part we only got permission to work this way because our output was in a very specialized domain area that had really critical requirements for some large business initiatives. Most teams could not have possibly negotiated this arrangement.

Like clockwork though, the director of our business unit was surprisingly fired and replaced with no explanation or warning, and the new director stopped us from doing this at that point. It reverted to the usual thing where sprint planning is just a Wheel of Fortune gameshow where you learn the radically new and unplanned pivots you'll be forced to make for the next two weeks. Like trying to win a 100m race by dancing on a DDR mat at the starting line.

Anyway, long story short, the barrier to anti-quality principle was that it's totally the team's discretion as to what work is admitted into the backlog. If the team ignores legit business considerations at their own peril, to goof around with fun stuff, then the team owns that failure. But no matter what, they are not told by the product owner what goes in the backlog, and it must be a sacred right for the team to tell the product owner "no" when management proposes to add work or change direction in a clearly stupid way.

So like everything in life, it all comes down to management. If management doesn't buy in, it doesn't matter how many processes you put in place, they're the ones that either hold the keys to success or failure.

Except that Scrum added time-wasting extra stuff on top of the situation.

If management doesn't buy in and I can operate my own low-overhead, low-formalism workflow, then so be it. I suffer the badness of management, but I don't also pay time waster costs for a formalized process whose sole purpose of better facilitating productivity is categorically disallowed in the first place.

I personally, as a matter of experience and opinion, would take it further to say that certain properties of Agile / Scrum actually add fuel to the fire and allow management's already bad behavior to manifest in even worse ways. But even if someone doesn't agree with that, I think the point stands: why incur the overhead costs of Agile if the same outcomes happen because of mismanagement anyway?

No, I agree; that's why I've always thought of it as an amplifier for management. If you have good management that buys in, then everyone is more productive. If you have crappy management that doesn't, then they're given a tool to amplify the micromanaging and dispiriting aspects of the job.

Man. ^ This is a brutal revelation right now. Totally just opened my eyes to exactly why things have been going badly at work, despite formal process.

Conway's Law deserves a management corollary.

That's great to hear, and I'm sorry it blew up.

I had a less extreme version that still served our needs. My rule was that a fixed percentage of effort, say 80%, would go to business priorities. The product manager could certainly ask about where the time was going, but they couldn't claim it unilaterally. The rest of it could be ordered and reordered as the product manager thought best. We in turn could ask about that.

This worked well for us, in that it let us keep the code clean and make tooling improvements without having to explain to a non-developer what was going on.

It was also helpful that we were good at expressing technical-sounding things in user terms. E.g., all scaling and reliability work was put into the backlog with titles like, "system handles 1000 simultaneous users" or "system survives web server failure". And a lot of research was covered by the product manager buying information rather than features.

Of course, this was at a startup, where there was no "on high". The product manager was the CEO, so in theory he could have fired us, but mainly we just collaborated on getting shit done.

My experience with scrum at work using VSTS is that you do have the option to mark tasks as closed and fill/dropdown box the reason why. If not on your own, then at a scrum meeting when the story or task comes up. (If you need to discuss whether or not it should be closed.)

I often wonder if it would be easier to start from the top instead of trying to build those barriers from below.

If your business leaders are prone to changing their minds at the drop of a hat, you probably aren't going to actually change that. Putting process in place to make it painful to do that is just going to cause pain for the people who have the decision imposed upon them - the CEO's just going to tell other people to figure out how to make it work.

Is there a way to bottom-up build something focused instead on reducing that pain from those pivots? Just-in-time planning whenever possible, for instance, to minimize wasted time invested in would-be future projects?

The timeframe matters, here, though. Are the wild changes once a quarter? Once a week? Once a year?

Generally the problem is that management wants this idea you call "just in time planning" to happen all the time, and even believes it to be synonymous to Agile. Meanwhile the software team is saying, that's fundamentally at odds with the entire premise of how to write software. Saying you want to produce software that was always planned at the drop of a hat (which is what management wants) is like saying you want to chisel Mount Rushmore but you won't tell whose faces should be on it until right at the end. It's just not a coherent goal. My hope with the barriers to anti-quality thing is that we would take things like this, where regardless of what management feels entitled to, and make it sacred and inviolable that the obviously stupid and incoherent things can be vetoed by the feet-on-the-ground people doing the actual work.

Hypothetically, "agile" is about _allowing_ decisions to be changed at the drop of a hat, without causing pain for implementers. The whole point is accomodating that, that's what "agile" _means_, right? (One can argue this whole goal is impossible, but I don't think it is. If it is, it's a problem for any agile process, not just "low-end scrum." But I think there are _way bigger_ problems with thinking you can plan out all work and schedules a year in advance and stick to it and wind up with anything succesful).

Now, every decision (new or changed) has a cost to be implemented. Some more than others. If you have decision-makers who refuse to recognize that, and think the amount of work they can ask a given staff with a given level of skill to accomplish in a set time period is unbounded -- no process is gonna save you from misery.

Definitely. I think you're well on your way to reinventing what the originators meant by "Agile". The subtitle of the first Extreme Programming book, for example, is "embrace change".

I spend a couple of years at a startup I co-founded; requirements volatility was legitimately very high. We were very experimental, which meant we learned a lot, which meant frequent wild changes. We had a long-term vision, and plenty of theories about the future, but rarely planned more than a couple of weeks out.

This was honestly fine. The product manager (my cofounder) was really good at including us on the necessity of the changes, so it never felt irrational or out of control. Our process was highly tuned for minimum waste. And we were militant about a lot of quality-oriented practices, including unit and acceptance tests, pair programming, paying down technical debt, continuous deployment, frequent releases, and close collaboration with the product manager. That meant that however things changed, we were prepared.

I think if you can do it at a start-up that's shipping every few hours, it definitely works at much lower levels of requirements volatility. To me the real issue at scale is that if agency moves away from the team level and gets concentrated at the executive level, then decisions at least appear irrational, and often are because real-world information becomes ever more removed from the decision makers.

You might want to keep in mind that making cost estimates for proposals is how developers get to influence the planning process. If you skip this, you're giving up your power.

> when you say "Low-end Scrum" is just the only type of widespread Scrum we've ever seen practiced anywhere, ever, to the point that it makes us question whether Scrum itself is partly responsible [...]

Yes, exactly. I know some good people who are dedicated Scrum fans. In their view, the engineering practices are all implied, because if you do have a relatively autonomous team dedicated to continuous improvement, you'll get there. And it works, because they are smart and dedicated people committed to doing the right thing.

So in that sense, the idea of Scrum is a good idea. But Scrum-the-movement and Scrum-the-business have fallen very short of that ideal. So in practice about 98% of Scrum implementations I've seen make me despair for humanity.

> What I really wish Scrum or Agile had, was a more unbreakable way to let development teams say no

One interesting version of that was the Extreme Programming Bill of Rights. One version of it is here (and in XP jargon "customer" is something like "product manager":

http://www.agilenutshell.com/bill_of_rights

It's not perfect, but I think it was a good attempt to make clear that there's a useful balance of power.

I love the phrasing of 'barriers to anti-quality' as a way to bring attention to this particular category of (rampant) problem. It reminds me of that saying 'under extreme pressure we tend not to rise to our ideals, but fall back on what we know.'

I work on 3-12 month projects in an agency, we don't really do Scrum. It's mostly some version of waterfall, and occasionally dual-track agile. Generally the process of determining what to build is squeezed into the cracks. I just recently built about 40% of a prototype for a SaaS in a couple weeks with fractional team input, and then had to lay out the rest of the flows in a week. After that, more needs popped up (I can only imagine the shitshow that development will be). This is the worst case scenario.

In less-worse cases, it seems like this becomes an issue of how to compare a requested change's value to its cost. Without enough data, this can be really hard to actually know well enough to have a grounded position on. It seems like consensus here is to focus on the cost of the change, rather than the potential value.

It also seems like there's little faith in managers. I don't totally know what it's like out there for ya'll though.

My experience is that a good team does a good job. A bad team doesn’t.

I think the focus on methodologies is to get a good result from an uneven team. Companies desperately want to treat programmers like standardized workers that can be mixed and matched as needed. The siren song of the methodology is that maybe it can achieve that goal. I have never seen this work in practice.

There are no quick fixes. People can improve but it takes time, dedication, and support.

I really dislike agile. A product owner asked me once if I get some benefit from them...and no, I don't. Thry get a lot of benefit from me sharing what I'm doing because they can then keep track of it and communicate it to other people, but I don't get any benefit personally.

I want to be the guy who talks to the rest of the employees, implements things and get direct feedback from them. I dont want anyone in the middle of that. It just complicates everything and makes me less connected and known in the company.

I want to be the guy who talks to the rest of the employees, implements things and get direct feedback from them.

How exactly does agile prevent you from talking to your teammates and getting feedback from them?

If I have a full and well-groomed backlog waiting for me at my desk, what's there to talk about? Sure, you can, but the the whole point of the PM is to take that off your shoulders and the parent is saying he doesn't want that lifted from him.

A well-groomed backlog is not working code. So I don't get how that somehow means there's nothing to talk about. If there's an agile MVP, it's "write some working code, get feedback on it, then repeat until everyone's happy within some tolerance." Scrum is designed to increase feedback, but it's a floor, not a ceiling.

Sure, but the PM is there for a reason and the PM has roles. There is overlap with what developers can be doing, but we are trained/encouraged to punt to the PM for such decisions, questions, and suggestions.

I get it, but it's now how I do my best work, proxying questions and discussions through a gatekeeper.

We do agile, but also use Slack and email, and walking across the room. If your organization is actively discouraging communication within the team unless it goes through the PM, that's bizarre. On the other hand, if the PM is acting as a gatekeeper with customers, that's not unusual. The reality is that you can't have everyone on the team talking directly to the customer, for a whole host of good reasons.

Regarding decision-making, that can be a problem---some PMs and product owners are micromanagers, hamstringing their team. But they would be micro managers with or without agile.

IME the PM works best when acting as a shield from random questions or future things or other not-important-this-instant distractions, but shouldn't block direct clarifying questions or feedback from you to a stakeholder. I'd keep them in the loop, but the ones I've worked with haven't wanted to just be a glorified "I deal with the goddamn customers so the engineers don't have to!" middleman for basic things like that.

As a tech lead in a Scrum team, this shield causes more problems than it solves in my experience. We often get questions missing context or find out about a decision too late to provide meaningful input on a technical level.

Team mates are the ones I'm working with all the time. I don't need scrum processes to talk to my team. A good team knows how and when to share information.

> Team mates are the ones I'm working with all the time. I don't need scrum processes to talk to my team. A good team knows how and when to share information.

Scrum processes are just a way to ensure that no stakeholder, including team members, is left out of those talks, left in the dark, or left unheard. Having a clique single-handedly and unilaterally dictating changes without communicating them or listening to any stakeholder doesn't serve anyone's interests. That's how projects fail.

A good team knows how and when to share information.

True. Scrum is a floor, not a ceiling. Nothing prevents the team from going above and beyond the daily standup and the sprint review in terms of communication, and I've never been on a team that didn't communicate a lot more than that. But doing that standup every day ensures that there is at least a minimum of communication, every day.

I'm pretty sure that when he says "the rest of the employees" he means what Scrum calls stakeholders, not members of the development team.

For a long time I felt that way as well. There are days when I still do, and this is from my current perspective as a tech lead who spends ~50% of their time in meetings with other parts of the company.

Sadly once a team scales up past a certain size its not practical to have developers also going out, finding out what people want built, and handling all the feedback on that. You need someone (or a team of someones) gathering the often conflicting requests coming in, and prioritising them against each other.

Having said that, I do believe once a piece of work makes it to a point where people are likely to start working on it the developers who will be doing that work should be involved in the process. They should understand not only what is being requested, but also why its being requested, in my experience that often results in them suggesting alternative solutions that either bet solve the problem, or provide a 90% solution with significantly less cost. It also gives them the context needed when deciding on technical trade-offs involved in different design choices.

I want to be the guy who talks to the rest of the employees, implements things and get direct feedback from them. I dont want anyone in the middle of that.

Part of little-a agile is making changes and adapting -- it's literally in the name. If you have a process in place that prevents you talking to the people you need to talk to, change or delete the process.

"I want to be the guy who talks to the rest of the employees, implements things and get direct feedback from them"

I am the same way. I much prefer talking to users directly and come up with a solution instead of having a middle man. But I have noticed that a lot of devs much prefer somebody filtering things for them. I guess to each his own...

Why haven't you told your manager that you are willing to do your job plus the product owner's job for only your pay plus half the product owner's pay?

Because that just doesn't work. It sounds clever, but in reality, it would get you laughed out of the office. And then the PO finds out that you offered to get him fired, and now the working relationship is ruined.

My experience is that a good team does a good job. A bad team doesn’t.

Of course that's your experience. If you judge teams based on outcomes, then you're going to conclude that teams that succeed are good, and teams that don't must be bad. Unfortunately that offers no guidance in how to become a good team. The goal of software management as a practice is to find patterns of positive behavior that make good teams. Agile, in general, is a collection of those patterns.

The observation is more about people. You take random people from good teams, mix em up and put them in a very different environment, they're still going to do a good job.

What if there is no reasonable and reliable way to become a good team, and the only way to go from a bad team to a good team is to replace most of the team?

How do you know they are coming from "good" teams? And if so, isn't it reasonable to suppose that they might carry over good practices in their new environment?

I don't doubt that there are bad developers---I've definitely encountered them. But software development is a team sport, and it makes no more sense to just "recruit good people and leave them alone" in software than it would to do the same in basketball. Great teams have both talented people and winning strategy.

Because people regularly do "recruit good people and leave them alone". At my first job I was the only developer working on the project and I rarely spoke with my boss. He came back from a month long vacation and I had not noticed he was gone.

That's surprisingly common across the field. Most developers work on small teams doing small but often very long lived projects. Code staying in production for 30 years is surprisingly common outside of SV.

Those people, in those positions, probably don't need much in the way of process. If you really can just sit down with the owner of the business, hash things out with them, and then go build something that does what they need then you're golden. And guess what, you're probably doing agile without even knowing it, because agile is attempting to reproduce that in organisations where you can't just put a single developer and a the CEO in a room and expect that to happen.

They also regularly "recruit good people and leave them alone" and they fail. And then we look on those people as "bad" people. Which was my original point above.

Look at sports... I've seen both cases, players that are good within one system horribly underperforming in others, and players that don't look that great meshing with other castouts and creating a surprisingly good team.

I suspect the same can happen with software developers.

Of course, there are people that will be bad anywhere and top guys that will be great anywhere.

I wrote more in my sibling comment. I generally agree that there aren't any strict guidelines for becoming a good team, but I disagree that replacing most of the team is the solution in any but the most extreme cases. I would venture a guess that, most of the time, there's one person to remove and one person to add: a new leader. If that doesn't solve the problem, potentially looking at developer contributions and seeing if there's someone who isn't pulling their weight and has no interest in learning how to. (I've only encountered these types of people a handful of times in my career. Most of the time it was shoddy leadership)

Then it would be good to figure that out, measure that, and then at least we can put the best people on the most important projects. It's perfectly possible to imagine a world where e.g. developer IQ was the only thing that mattered and the quality of software that got produced was simply proportional to the average IQ of the developers who worked on it, sure.

I don't think that's the world we live in though. I've seen smart, sociable, effective developers fail in misorganised teams, and I've seen teams of ordinary people produce very good software by getting their processes right.

I accept that he's polarizing to some, but Jocko Willinck's book "Extreme Ownership" has some interesting stories on this topic. One of them is at Navy Seal training camp, where there's 6 teams involved in a multi-stage boat race exercise. Half-way through, team 2 is destroying everyone and team 6 is losing pretty badly. So, to teach a leadership lesson, the instructors decide to switch the leaders from teams 2 and 6.

At the start, team 2 continues doing awesome, and while team 6 doesn't win, they improve. Next round Team 2 isn't doing quite as well, but still good. Team 6 has improved a bunch. After a couple of rounds, team 2 is falling apart and team 6 is now winning (or 2nd place, I forget).

My experience, admittedly anecdata, is that great teams perform well when they're in an environment where they're allowed to succeed. I've worked with great teams who did terrible work, and it was pretty much always due to a bad environment (e.g. an owner who would redirect their efforts every few days, so no feature ever got to done-done). I've also seen (but not personally led) teams that I had originally somewhat written off, who under good leadership managed to do some pretty damned impressive work.

And don't get me wrong, I'm not devaluing the potential contribution from a single amazing person, or a team full of amazing people. My clients hire me because I have a (locally) rare set of skills that their team doesn't have, and help them get over hurdles that their team can't (currently) tackle on their own. But if you take a great set of individuals and put them together in a shitty environment with a shitty leader, you're going to get what you get.

My quip on scrum/agile is that it "ensures mediocre results from mediocre teams." Its just a methodology in the end. It can help keep those that tend to have wandering minds focused, but for high performing teams the overhead slows them down.

The assumption of a standard programmer is just never a reality as well. I actually saw a startup try to go full religious agile and only hire "full stack" developers to try to homogenize them so that they would fit the agile mold better. It didn't seem to work very well, people are people and naturally gravitate towards data modeling, or UI, etc.

My quip on scrum/agile is that it "ensures mediocre results from mediocre teams."

And honestly that is all that most companies actually want/need. Most teams are mediocre and most projects only need a mediocre result. Getting something as good as a mediocre results out of your mediocre teams would be a massive improvement for a lot of place.

I would rephrase it from "mediocre results" to "good enough results" - not everything needs to be a work of art. Especially in the startup world, where there's never enough people to go round, there's a huge value in knowing when you've done enough that the project you're working on is no longer the thing that'll deliver the most value.

What agile does when executed right is to fully embrace that, and wrap it in a process that ensures that when you reach that point you're able to finish the current iteration, ship what you've got, and move on leaving things in a good enough state. For me that's the core of agile, and in my experience at least you can't pull that off with a mediocre team. You need a team where at least some members have the experience and the pragmatism to know the difference between delivering a minimum viable product and cutting corners in a way that'll hurt in the future.

When I learned about the concept of "just barely good enough" I was thrilled by it. Lots of people been using the expression "good enough" and to me it feels like a defeatist idea. "They did their best! It's good enough, isn't it?"

http://agilemodeling.com/essays/barelyGoodEnough.html

Given the constraints of the estimate we've made for a story that could really go much longer, except that we've only allocated two days... I think the problem you're hinting at is that different people have different ideas of what the Minimum Viable Product should look like.

Accordingly, on my team where we're all under-trained with respect to Project Management and mostly just making this shit up as we go along, we've identified several other points on the curve.

It's suggested that one should make some nominal efforts to identify, and stop for a sniff along the way to JBGE:

"that's definitely not good enough, try again"

"I'll know it when I see it, but I'm sure that isn't it"

"kinda-sorta good enough"

"that's nearly good enough, except for this one thing..."

Identifying some of these points along the way (and making sure you notice when you've arrived at them) in my opinion is a great way to help ensure you're not actually aiming for points somewhere deep on the wrong side of the curve.

The mediocre team also may have trouble getting to Just Barely Good Enough because of overshooting. But in my experience that's usually not the problem. Nobody is working far past JBGE into the wee hours of the "OMG this is too much, I love it but how long did it actually take?"

If anything, the feature has to be shelved or scrapped, or handed off to someone else, because it's taking too long due to an error in calculation of trajectory. Spending too much time on the wrong things. Incremental progress is the goal, and perfect is the mortal enemy of good.

To be honest, that wouldn't be a problem if more companies were up front about it. But every company out there has this idea that they have to hire "the best".

every company out there has this idea that they have to hire "the best".

Maybe in Silicone Valley, but most companies are happy if they can just find a programmer that knows how to program.

"Agile" is completely independent concept from "full stack" or "homogenized" development.

This is true. With the emphasis that Agile puts on "blockers" though it becomes apparent that the typical model of having a simple person own a component with maybe a backup person or two is problematic. "Only Amy knows that code, and she is working on X which is also critical" is one of the more frequent types of issues that comes up in standups.

To combat this, this individual decided to try to reduce the specialization as much as possible. Everyone would do everything was the thought process, so no one could ever hold anything up.

This was a non-technical founder of a now dead startup, but it died of the much more typical problem of market fit- they did launch, and the site worked. It may be interesting to mention he was also using entirely offshore development teams.

My experience is that a good team does a good job. A bad team doesn’t.

My experience is that sprints interrupt my workflow to such an extent that I can no longer get anything done. In other words, they make a good team bad.

How so? I'd be keen to hear just in case I'm unwittingly inflicting similar pain on my developers.

Perhaps it's just me, but I need long periods uninterrupted to work. It's not just the regular status meetings, but other meetings which break up the day to an extent that I can no longer get that.

Yeah, but that's not a scrum thing. Most days, I have the daily stand up (which occurs right after I get to work), and that's it. The one day where we do all the grooming and retro and demo every two weeks is more or less lost, but that's not much of a problem.

What about code reviews?

Code reviews are very important, so I don't complain about time spent doing them.

Agreed. I've seen a lot of "good practices, methodologies, processes, etc", but none of them even tries to address what is, often times, the most critical issue: a team of good software engineers. Instead of endless meetings and bullet points about how to handle a Jira ticket, maybe focus on how to handle pointers?!. I've really had colleagues who don't know how to program in C and when asked about it said "I don't even want to learn it, we have all this tools and methods", but their title says "C embedded engineer" or thereabout.

I can also see the business point of view in this. You want switchable puzzle pieces. Take this developer from this project and move him to that project. This developer knows X, Y, Z and that project needs X, Y and Z. So it's all nice and well. Only it doesn't seem to work in practice.

Great job distilling the essence of scrum into three bullet points, I wouldn't disagree with those.

Here's the common pitfalls as I see them:

1. Sprints offer flexibility but unless you educate the whole company waterfall + gantt charts are what the business side wants

2. Unless you educate all members of a scrum team on what the retrospective is, and let the developers drive it, you won't be effectively improving processes

3. I've always felt conflicted about the agile "working software over comprehensive documentation value". To me it implies that documentation is easier when the product is working and/or the product might change based on feedback, so you don't want to re-write the docs. To people who don't learn scrum/agile it means "don't write documentation"

That last point is simply working software _over_ documentation, meaning that in a world of finite resources what would you rather have? A pile of requirements and technical design docs _or_ a working product? All teams need to determine the right amount of documentation that maximizes total output of working software, but agility was--in part--a response to analysis paralysis and high overhead of many software implementations of the time.

Not all documentation is equal.

While design documentation tends to be anti-agile (but sometimes useful to avoid wasting more time on miscommunication and mistakes), end user documentation is, for all practical purposes, part of the valuable software product, as it enables end users to do what they want to do.

Unfortunately, being able to neglect end user documentation makes many outfits treat it as a useless luxury, either because they are only judged on software (e.g. passing a test instead of having satisfied users) or because they don't care (e.g. many introverted hacker/artist open source maintainers). It's purely delusional agility, like driving fast by not stopping for fuel.

You're describing having a well defined goal, setting a critical path, and then regularly updating your progress. What we gray beards used to call "project management".

But you have to include 'estimates', which ruins everything (in non-trivial cases, yadda, yadda).

Estimating is my Achilles heel. Always has been. I'm way too optimistic.

My workaround was to do jelly bean estimating. Get guesses from everyone, then use the average. Worked surprisingly well.

We did other things to meet that target date, honor that estimate, of course. But that's a longer story.

> I'm too optimistic

Have you ever thought much about why, though? When I was young, I was super optimistic - and I got burned by all of the surprises (including the underspecified or unstated requirements), so I started to estimate pessimistically based on experience. But then I got stuck in this loop: "How long will this take?" "Probably two weeks" "What? Why two weeks? Why not two days? We only have two days. Say it will only take two days, or tell me everything you're going to be doing on an hour-by-hour basis to justify this high estimate!" So now I've just gotten really good at figuring out how long they want me to say it will take, say it will take that long, and shrug my shoulders when it takes longer. Since everybody else is doing the same thing (as they don't have any choice), it doesn't surprise anybody.

I have thought about it. A lot.

I'm told I work too slow, dive too deep. Now I'm trying to figure out how to do just 80%, "good enough".

When I was a kid, I published shareware. After getting a few 3:00am wake up calls from irate customers, I said "never again". I spent 15 years designing, engineering away technical support calls. It mostly worked. Later on... When my team(s) burned a CD, we rarely had to issue patches, hot fixes.

Customers loved us.

I thought that was the pinnacle of human achievement.

Now, I feel like I'm still fighting battles over things no one cares about any more.

This. Figuring out how to stop at 80% is a skill I'm struggling to learn. And I don't mean "learn to half-ass things"; in the vast majority of cases 80% is more than good enough to get the job done.

Customers may talk about hating bugs, but when you ask them to put their money where their mouth is, 95% of the time they'll buy buggier software with more features.

> Estimating is my Achilles heel. Always has been. I'm way too optimistic.

That's kinda the whole point. Everyone sucks at estimating. Scrum proposes a way to do "what greybeards call project management" in a way that deals with this universal inability.

I'm having trouble articulating this, please forgive:

I didn't explain "the other stuff" for why jelly bean estimating works.

The number doesn't matter. It's the shared hallucination (consensus) for the deadline. It short circuited, resolved the debate about "how long". It secures the team buy-in.

Once we had a deadline, we managed the work to match. Closest analog I can think of is Kickstarter: initial goal and stretch goals. We had the must haves (stuff listed in the draft press release). We also had dozens of nice to haves we'd sneak in, time permitting.

Agile, Scrum, sprint style estimating is too fine grained, the horizon too near, to do effective estimating.

What I've learned in engineering (any form, including software) is take the estimate (acquired in any decent enough way like averaging) and multiply by PI (or 3 if that suits you) to get a reasonable amount. In practice it works more accurately for larger sums than smaller ones so your mileage may vary. I guess it is easier for everyone to guess smaller workloads more accurately.

In my practical experience, I multiply the amount gathered from the previous statement by two to have sufficient negotiation space for people 'higher up the chain' who have no idea about anything of this and simply pull estimates out of thin air or their rear without any proper experience except "previous time it was xxx so it should be the same" (which it almost always is not).

edit: I prefer to have some time left on larger projects to avoid overrunning, even though that might be common practice. Usually though time is cut short due to delivery constraints or selling a product before asking the development and you get the 'wasn't it done yet?' speech.

Here is the "chart" that I use when determining how accurate an engineer's estimate will be. The left column is the hour estimate to look up the volatility from. The right column represents the potential volatility (probability of increasing in time) of the work item based on the hour estimate.

Hour Estimate : Volatility Level

    1 - 5 : Little to no volatility.

    5 - 15 : Small volatility (a day, maybe two).

    15 - 30 : Medium volatility (multiple days).

    30 - 60 : Medium to High volatility (days to weeks).

    60+ : High volatility (multiple weeks).

The reason behind this chart is that when determining what goes into a release there is a tendency to treat all items as static in size. What I mean is that most people will say, "Three 15 hour work items are equal to one 45 hour work item." or "Six 5 hour work items are equal to one 30 hour work item." The worst offender, "Four 15 hour work items are equal to one 60 hour work item."

Additionally, as a work item increases in size the amount of "knowable things" increases as well. Of those "knowable things" there are many known knowns, known unknowns, unknown unknowns (thank you DR for that gem). There's a near 0% chance that a team will get all of the requirements in place up front on a 60 hour case.

If a scoping team is told that they have 240 available hours to work with, then they will pick a combination of items that fit within the 240 hour box. If they have four high priority 60 hour estimate items then they may pick all four for the release. In the event that four 60 hour items are chosen I would place a near 100% guarantee that the release will not go out within the 240 hour time frame. I would even wager that will be up to four weeks late.

Nothing protects against toxic management.

"Evidence Based Scheduling"

https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...

I go one step further: Get guesses from everyone, then use the average multiplied by 3. That lets management talk me down by 1/3rd so they feel they've won something, and I still end up with a little wiggle room.

Everyone tacitly assigns a time value to "points" anyway.

Which is fine. The point (heh) is that the mapping can be adjusted in a feedback loop based on what you actually did in the past.

I've always seen there being a maximum size of a ticket for one sprint. So effectively 8, or, 13, or 21, or whatever, is two weeks, and all the other sizes are some fraction of that, and you end up trying to calibrate how you vote for points based on how long things take.

> You're describing having a well defined goal, setting a critical path, and then regularly updating your progress. What we gray beards used to call "project management".

That's not true at all. The whole point of organizing projects around scrums is precisely that there is no well defined goal nor can one exist. What exists are the client's needs, and those needs do and will change frequently and radically as projects move on.

Therefore, the classical project structure followed by grey beards does not make any sense in software development. What does make sense is to acknowledge that requirements specified by the client do and will change, and it's better to adapt to the client's needs and get everyone involved in the project. Therefore, projects are organized in small bite-size units of work that gradually contribute to implement a product based on the feedback received by all stakeholders, and that these small bite-size units of work are implemented in small time windows. These units of work are short-lived and the amount of effort invested in planning is limited to the goals set forth for that unit of work, which need to be manageable.

"...there is no well defined goal nor can one exist."

Consulting is the pejorative we gray beards used for that activity.

You reminded me of another pithy throwaway line:

Agile didn't improve outcomes, it just reduced the cost of failure, allowing teams to fail many more times with the same budget.

So, agile improved outcomes? I'd love to have 1 big win a year and 11 cheap losses, than 5 expensive losses and then bankrupt before a big win.

> Consulting is the pejorative we gray beards used for that activity.

Those hypothetical gray beards may come up with all the pejorative terms they need, but that only hides the fact that they are entirely oblivious to the reality of running a successful software project, one which actually meets the client's requirements and delivers working products. Therefore, they spend their time coming up with pejorative terms while they insist in wasting their time and effort forcing the proverbial square peg (waterfall) in a round hole (software development projects)

> Agile didn't improve outcomes,

That statement is patently false. No one can make that claim with a straight face in a world where paying customers make it their point to change fundamental requirements on a weekly basis.

In the old timey's waterfall world, a waterfall project that's executed flawlessly is a project that ends up delivering the wrong product that fails to meet the client's basic needs, thus leading to a very unhappy client that may even feel that he has been played.

> it just reduced the cost of failure

A sequence of small failures that converge to the client's needs is a whole lot better than a single unmitigated major failure that's ensured by following Waterfall methodologies.

You and I have very different expectations. Long ago I decided to walk away from work lacking clarity. I’m a dev, not a therapist.

re Outcomes, my data is old, I’d love to proven wrong.

I’ve read (elsewhere) that high(est) functioning orgs like google, facebook, netflix have made great progress. But the rest of us are just banging the rocks together.

PS- PMI & critical path, as I’ve done it, is the opposite of waterfall. One hard earned trick is proper ordering of the work. eg, after a project kickoff, first deliverable is the press release, second deliverable is a demo (even if its entirely faked).

>but that only hides the fact that they are entirely oblivious to the reality of running a successful software project

Wow, amazing that the shoulders of the giants you stand on was never a successful software project.

>In the old timey's waterfall world, a waterfall project that's executed flawlessly is a project that ends up delivering the wrong product that fails to meet the client's basic needs, thus leading to a very unhappy client that may even feel that he has been played.

Waterfall was iterative as well, when the needs changed, so did the project plan. You bought that line, but that's not how it ever was in my experience. You know those version numbers software has? Those are the iterations. We'd cut a release about every 3 months.

Based on your comment, I suspect you've never done anything outside of scrum / agile, so you are comparing it to some mythical way of doing things you heard about (undoubtably from scrum consultants.)

> Wow, amazing that the shoulders of the giants you stand on was never a successful software project.

Those giants you're referring to made a lot of mistakes along the way.

One of those mistakes was blindly trying to force project management practices that were driven by the need to allocate material resources into projects where the only resource that's allocatable is man-hours. In projects whose success is determined by how well are material resources spent, redesigning something midway is something that's entirely unthinkable and can even dictate the project's death. That's not the case in software development projects, where the only resource is the fixed amount of man-hours that's at the project manager's disposal. With the development and adoption of a couple of software engineering practices aimed at preserving the project in a deliverable state, redesigning entire modules is essentially free. Therefore we end up with project management challenges that superficiality may appear to be the same but are actually fundamentally different.

Therefore, different types of constraints result in different optimization problems that lead to different solutions and require different approaches.

> Waterfall was iterative as well, when the needs changed, so did the project plan.

The keyword you've used is "needs". You're assuming that changes are exceptional. They are not. In software projects, requirement changes are the norm, not the exception. Changes aren't needed, they are constant. If your goal is to meet the client's needs then you need to meet the client's needs, and not some line in the sand that has no bearing with the paying customer's goals.

> Based on your comment, I suspect you've never done anything outside of scrum / agile, so you are comparing it to some mythical way of doing things you heard about (undoubtably from scrum consultants.)

You suspected wrong. In fact, I have far more years of experience in waterfall projects in real world engineering projects than in scrum/agile. Waterfall projects make sense when the requirements make sense. Software development projects based on basic software engineering practices don't incorporate some fundamental requirements found in engineering projects, thus their efficiency can and does improve by following adequate practices.

"The whole point of organizing projects around scrums is precisely that there is no well defined goal nor can one exist. What exists are the client's needs, and those needs do and will change frequently and radically as projects move on."

I don't feel this is true at all. The issue is that most clients aren't willing to put in the time or effort to actually see what their needs are.

> I don't feel this is true at all. The issue is that most clients aren't willing to put in the time or effort to actually see what their needs are.

You're assuming that it's realistic to expect that a requirements gathering process is able to precisely define all requirements, that these requirements will and cannot change, and that software architects have perfect information and are able to make flawless choices regarding the design.

None of these assumptions hold even in conventional engineering projects. The only reason why waterfall processes is used in conventional engineering projects is that it's so expensive to fix problems arising from these sources of failure that it's perfectly fine to deliver working but inadequate solutions.

"You're assuming that it's realistic to expect that a requirements gathering process is able to precisely define all requirements"

No, I'm not. I am assuming it's realistic to expect that a company actually put some effort into finding out what they need, and getting us that information so that we can actually plan out a project.

"None of these assumptions hold even in conventional engineering projects."

Part of the reason for that is that companies don't do any of that research. They don't look at what they need; they think about what they want.

One of the biggest reasons people here dislike "Agile" is because management uses it as an excuse not to plan anything, and fly by the seat of their pants every two weeks.

Another observation I have had, it provides much clearer and realistic progress reports to management. You can track the normalized points per day executed by the team to see their overall velocity. You have clear check points about what has been accomplished (which is why a shipped product is critical, it removes any chance for something not to meet DOD)

On the dev side, I have never attended so many meetings. Technical debt being addressed is generally against the feedback loop, unless the PO/Leads of the team make it their first priority. Grooming meetings are a mix of people who care, those who don't, and vague stories. To save time, any new story got sent to one of the devs for a sniff test before grooming, to help ask critical questions early. Typically, the request is reasonable, but may run into a contradiction of the system.

I have also found that, forcing scrum (or waterfall) on a product that is running at a difference release methodology is horrible. Got a customer that needs a new thing built by a specific date? That won't fit, unless the date is flexible and customer stake holders can be deeply involved in the process. Waterfall at least forces the customer to define the project up front and allows for calling out changes to the plan and thus changes to the scope.

As with all the above, this sits on how skilled your project manager/scrum master/product owner are at their jobs. Anti patterns abound.

Sprints are a shit idea and need to die a horrible death. The number of times we've had to break up some functional requirements for no reason other than to meet some stupid concept isn't funny ultimately it's a waste of time in itself

Are you saying you have projects that show 0 progress of any kind after 1 month, but still succeed later? What is happening in that first month?

Laying the foundations. Doesn't look like a house even after you've done it.

Exactly. You need the end-to-end backbone working so that you can start adding end-to-end user-visible behaviors on it, a few at a time.

Research? Experimentation? Foundations?

Pffft. We're talking crud apps here. Has any innovative software ever been developed in "sprints" using "user stories" and "story points" with a "product owner" and a "scrum master"? I doubt it.

Yeah, I've been on a couple of those. It was XP, technically, not scrum, but we did sprints and user stories and story points, and we had a product owner, and we had a guy who was de facto the keeper of the methodology. It worked really well, too.

And no, it wasn't just crud apps. It was a distributed security auditing tool. Yes, it had a database component, but it was much more than that.

It's best to break up user stories into small vertical slices and implement them within a single sprint so that you can get early feedback from the customer. That's a great way to uncover misunderstandings about requirements, even though it might seem less efficient. Optimize for rapid value delivery; a little wasted time is acceptable.

That or having to estimate work based on your sprint. This is a 130 pointed task? Oh we only go up to 8.

I can probably agree that sprints are usually more realistic than old school waterfall (arguably it depends, like all things) but why do we feel the need to perpetually sprint?

A simple kanban board seems to give all the same benefits without the constant pressure and weekly death march to meet the end of sprint commitments.

Because a simple kanban doesn't have any scope/goal, it's just a disorganized TODO list. A sprint/milestone is a way to set goals for the next 2-4 weeks. At the end of the iteration, you redefine new goals according to what would bring the most values to the clients at this new moment T.

If the sprint is not finished, well it's not a big deal. You move the tasks back to the backlog, or in the next sprint, or delete them if it doesn't make sense anymore. And you decrease the number of tasks in the next sprint. You shouldn't feel a constant pressure to finish sprints if the velocity of your team is taken in account correctly.

"Because a simple kanban doesn't have any scope/goal, it's just a disorganized TODO list."

Not everything has to be encapsulated in the TODO list. I mean, a Jira backlog is also just a TODO list.

To be fair, the simple kanban doesn't stop you from creating a scope/goal, you need to generate your TODO list somewhere, which isn't any different from sprints, but it's less rigid when it comes to the deadlines.

More to the point is, setting sprint goals is defacto pressure which many people and orginsations are poor at handling, and we can largely avoid it by not having people doing or managing the work care about end of sprints, but just that the tasks are scoped and are of a reasonble size.

The business benefit of having a longer planning horizon is that it enables making some commitments outside the agile team. In the real world sometimes that's necessary. A sprint cadence also helps ensure that the team does real retrospectives rather than putting those off. But certainly Kanban can work as well or better in some environments. Pick what works for the circumstances.

I definitely feel that once something has gone into "maintenance mode", where most of the work of making the project exist is done, then Kanban can be much better for organizing work.

Sure maintenance engineering is usually a good fit for Kanban. If a high priority defect report comes in you don't want to have to wait two weeks to include it in the next sprint plan. Just move it to the top of the backlog and fix it tomorrow.

For larger products with multiple teams working on the same code base it can even work to have a mix. The feature development teams follow Scrum (or something similar) and the maintenance teams follow Kanban.

It's worth noting that Scrum does not have sprint commitments: https://www.scrum.org/resources/commitment-vs-forecast

I'm probably oversimplifying, but this looks like 3 different ways of saying "get feedback often" :)

That's... actually pretty accurate? Dang.

So it becomes this?

* Get feedback often from the business owners

* Get feedback often from the developers

* Get feedback often from the users

And act on that feedback.

I argue that's actually a misconception of many scrum practitioners. It's not really "get feedback often", it's "get feedback early". "Often" is just a bad approximation of that. The difference is most obvious with retrospectives - if you just do them often they feel like a burden. What you want is to get early opportunities to improve .... but that's not necessarily through a very frequent retrospective! You have to let things crystallize a bit, so that people can observe the effect of process changes & give accurate+reasonable feedback on it.

> This does not sell books and consulting, so it evolved into a field of its own.

It's funny how it turned into UML

UML comes out of a different culture of development, that of the Rational software crew. They had their own much more heavyweight process than the Agile folks: RUP, or the Rational Unified Process: https://en.wikipedia.org/wiki/Rational_Software#UML_and_RUP

There was a little overlap; in particular, Martin Fowler wrote an intentionally thin book called "UML Distilled". But the UML world was generally heavy on both up-front architecture and formal, heavyweight design process. The Agile world at the time was energetically opposed to that, thinking that the main value of UML was in having a consistent language for us to all sketch on whiteboards together.

Too bad people didn't bother reading the Unified Process book, which clearly specifies the use of iterative and incremental development and the creation of just enough UML diagrams to communicate and document a system design. It even argues that working software should be used to describe an architecture in the Elaboration phase rather than a 10000 words document. It can actually be considered as one of the first agile methodologies. RUP and other flavors of UP just added tools and guides aimed to facilitate the implementation of UP, but it was misunderstood that every template and section must be filled out instead of actually thinking what to use depending on the project. Unfortunately the whole thing was ultimately bastardized by the non-technical template zombies that thought software development is like building a skyscraper.

The CASE motivated tooling got in the way.

My go to joke: RationalRose is like an 800lb angry gorilla sitting between you and your work.

Alistair Cockburn did a great post mortem about that era.

Characterizing people as non-linear, first-order components in software development [1999] http://alistair.cockburn.us/Characterizing+people+as+non-lin...

During that time, I strongly preferred lo-fi paper prototyping for UI. And I'd scratch out UML diagrams. Worked fine for intra-team. But it just didn't fly with management. We had to make everything more formal and pretty.

Oh well.

It’s funny looking back at what we used to call software development methodologies. Booch and Rumbaugh were primarily focused on modeling. Any talk of process meant a dynamic description of the program.

Jacobson was a little more method focused. You can see how his introduction of use cases started the idea of talking to your customer about what they need.

I remember seeing Rose and thinking this will make a huge improvement only to be repeatedly confused and frustrated that it seemed to make everything take longer.

Finally! I get share my Booch story:

At OOPLSA 98, Booch convened the first society of software architects (or some such).

Having read all his writing (ditto Yourdon, Brooks, Rumblaugh, etc), I was super excited.

During the Q&A I finally got to ask “What is software architecture?”

Booch replied “Software architecture is what software architects do.”

Bubble popped.

I gave up seeking the council of the priesthood and blazed my own path.

Wow. In some ways, that's the worst possible answer. In my view, software architecture is what every developer contributes to, which means we have to be thoughtful about how we collaborate on it.

Every time I see a shop with designated Software Architects who hog the architecture work, it's a fucking mess. Because then Software Architects don't get rewarded in their careers for making software. They get rewarded for writing white papers and coming up with "brilliant" architectures and impressing executives who know less than them about technology. Which is mostly inversely correlated with actually making it easier for developers to get real work done.

I totally agree that it was taken over by zombies, and also agree it had some good ideas.

But I don't think it was quite as incremental and iterative as all that. One, it was more a "process framework" than an actual process, so iteration length could be whatever the grand poobahs wanted. They were entirely fine with multi-month iterations. Two, as the "RUP hump" chart shows, it was always pretty waterfall-ish. And three, whatever the words in the book, the RUP adopters were generally large-company shops, and he who pays the piper calls the tune.

I definitely remember the RUP people coming to the early Agile conferences and being very suspicious of our wild ways. So I'm not sure even its originators would have called it an Agile process.

Thanks for your misc comments in this thread.

This particular comment, the "UML Distilled" reminder, triggered a few hindsight notions.

Much like XML and J2EE tainted Java, I'm now having trouble distinguishing between UML and OOAD. Though I preferred Fusion to UML, I'm now wondering if any OOAD based methodology had a chance.

"...thinking that the main value of UML was in having a consistent language..."

Guilty. For a long time, I very much wanted executable models, roundtripping. I got over it. Now I place more weight on concision, DSLs.

But here's the new (to me) notion:

Maybe UMLs (and RUP) greatest benefit was the idea of having a list of questions to ask. You know, a methodology.

Writing this out, it sounds stupid, so please forgive my stumbling.

But two things stand out from the 90s, which I'm missing terribly today.

I miss Joel Spolsky's 12 steps to better code. What's today's equivalent? I haven't found it.

I miss strategy, strategic thinking. With all this agile stuff, I have no idea what anyone's talking about. Yes, UML & RUP were terrible. But at least we were trying to bring order to chaos. With agile, it feels like we forfeited, that we should just accept the chaos.

I don't think the gist of successfully developing software has changed that much since the early 90s, at least in consultancy: You work in iterations where the team and client develop, refine and prioritize the features of the system until the system is good enough to be considered finished. There is an initial vision and strategy that needs to be developed in the first iterations. The UP implementations describe ways on how to do this, though the agile community has come up with more creative frameworks: design sprint, product vision box, customer service map, etc. The strategy is reviewed each iteration with the re-priorization of the features. For this it's important to have revisions on the documents created on the first iterations.

The challenge as I see it is that clients usually don't buy into this and they believe that developing software is like making sandwiches or turning buttons on and off when in fact is a discovery process of abstract things. They then demand a fixed budget, fixed time and fixed milestones for a fixed scope ("You are the expert, you should know how much time this takes."). To sell the project the salespeople from software companies deliver this, those who don't are considered incompetent. In this conditions agile methodologies are prone to fail but not for the methodology, but because most of the time those sold promises are done with incomplete information or tailored to be optimistic to sell the project. Metrics from agile methodologies show from the beginning that the project won't meet the expectations and then chaos ensues: they stop collaborating with the client to hide the reality of the project, "managers" pressure the team to do more in each iteration, they stop doing the disciplines altogether, iteration and daily reviews are treated as long meetings where people commiting errors are shamed and scolded and frictions start to emerge between all persons involved. This is where I believe the whole notion of "agile sucks" comes from.

Much agreement. Been there, been done like that.

What’s more waterfall than the sales team throwing a poorly baked contract over the wall to the dev team and telling them “make it work”?

I think "accepting the chaos" is fair. We've accepted it because it works - or rather, Scrum is about doing the minimum amount of "bringing order" necessary to move forward, and no more. Which makes sense - people like the idea of tidiness, but you shouldn't put more effort into organisation than you're getting out of it.

> 2. It's important to stop what you're doing on a regular basis to evaluate progress and problems. This always seems like a waste in the moment, but failure to do so leads to regret down the line.

This point is of general usefulness and applies to many areas of our lives, not just software development.

Preach

"This does not sell books and consulting, so it evolved into a field of its own."

Exactly. This is why it's stupid when managers get all hyped up about <present software development trend/> and decide everything needs to be shaken up based on what they learned at a seminar or heard from some consultant/"coach".

For the most part, those 3 points seem like easily understandable ideas, and have been used in some form by other industries, such as manufacturing, for a long time before they were presented for software development. Replace "PM" or "Scrum Master" with "Foreman" and you can get the picture of what activities are involved with a worker managing other workers and the work that gets produced.

There is so much over-complication with the current system of methodologies and nebulous acronyms that accompany it[0]. There's SAFe, Agile, Scrum, Scrum of Scums (recursive scrum??), RAD, LeSS, etc. It gets confusing and messy very quickly. Instead of focusing on doing the work, it's easy for managers to get swept up in the semantics of these frameworks and end up hurting the work by over-thinking and over-applying the principles on the workers. Scrum methodologies are guidelines that can be tweaked to fit the situation, and don't have to implemented if they don't make sense. Instead of thinking "Does this make sense?", the manager/business side tends to think "Well, it worked for X, so it must work for me!". Scrum is an editable template that should help eliminate wasted effort and mindless work, and if that's not the case then it isn't being done right.

[0]https://en.wikipedia.org/wiki/Scrum_(software_development)

My mental model was that ideally it is a greedy algorithm for job shop scheduling (dev time) with dt = two weeks and a horizon as long as your backlog. The biggest and most common failure mode I saw was having a bad product owner (it can be a single point of failure, devs have a team to help them out).

I completely agree. I don't get why people try to strictly follow the methodology (and then complain about it) instead of just taking the core of it and apply what work for them and their company.

I've a SCRUM master certification and still, I follow only what's good for us.

It's totally fine for me to add a task in a current sprint. If you care about finishing the sprint, you just move a task of same points back in the backlog.

It's also totally fine to allow the technical lead to add his own tasks to the sprint. You can do 80/20 for instance (business/technical values). It's not going to kill your business and will make your developers happy (SCRUM is about people, not process, right?).

Problem is, that's a recipe for management ignoring all the stuff that gives developers power and agency, and taking all the micromanaging things.

My best experience with Scrum was with a team that was willing to completely overhaul our weekly process if we needed to. We finally settled on a structure pretty different from the general recommendations, the most significant being we had no daily standups. They felt kind of useless, one guy on the team particularly hated them and pushed for us to do a sprint with no standups. So we tried it out once and kept it that way permanently, turns out our team really didn't benefit from them.

When the rest of the company saw our productivity, they moved every other team over to Scrum, hired a Scrum coach, etc... And it didn't work because they were extremely strict about the "process" and any changes to the process were like arguing against the bible. They even made our team begin doing daily standups every morning again.

I'm not certified or anything but the entire point of Scrum to me is that you frequently re-evaluate your situation and team and adapt to that. Everything else is just a good baseline to start with.

>1. Sprints are a better way to organize than Waterfalls. I've experienced this over and over again personally, so I'm sold on this.

There are so many people who say, Agile or Scrum, respectively, do not need specs, the specs are just incremented with every sprint as the product itself, leading to software projects with undefined goals and overlong project time

I feel like the books and consulting came from the fact that Scrum, while it can be explained in a paragraph or two, is very hard to reconcile with the idea that what the business wants is for you to be able to guarantee that you'll be "done" (and define exactly what that means) by a particular date in the future.

The scrum manifesto is not overly verbose. It explains the value and concepts in a manor that does not lend itself to misrepresentation.

There is definatly a lot of companies peddling hype around scrum but it has a very lean and strong core which is in no way pulling in that direction.

What's the scrum manifesto?

Paraphrasing: The Agile Methodology is to argue about The Agile Methodology.

Source: Certified Scrum Master training. Three times. Still have no idea what these charlatans are talking about.

Which is a higher distinction, Scrum Master, Web Master, or Dungeon Master?

Dungeon Master. They're the only ones that can do something as simple as place an apple on a table, and tear apart your entire team.

Maybe they meant the Agile Manifesto? It’s short and hopefully not controversial: http://agilemanifesto.org

But... my eyes. They burn. Thank heavens for developer tools and the ability to delete backgrounds.

Jokes aside, good call linking that. Super short, super influential, super relevant to the conversation at hand.

You also missed the trade off's for increased speed compared to traditional wf projects.

One of the problems with the Scrum model is its used for every thing - just like they used to try and do every project with WF and IS9000/BS5750

Are you suggesting Waterfall projects are faster? I don't think you are, but that's how I read your sentence.

Also I don't know what IS9000 or BS5750 are but they sound like they had amazing naming committees.

Oh bum I meant ISO9000 BS5750 is the Uk equivalent (and I worked at my first job on a joint project with the BSI on BS5750)

But and its a big but some times you need the rigor a properly done waterfall process provides Air traffic control for example.

You seriously don't understand the trade offs you make with "agile" vs waterfall? I have done both very successfully.

Nope, don't understand the trade offs if you're suggesting scrum is slower than waterfall. That just doesn't line up with my experience. "Results may vary," though.

The issue I've had with every other project management approach is the illusion of structure. I'm not an expert on all or any approach. But my experience is that everyone wants me to be able to project, since absolute certainty, what my team can get done in 12, 24, 36 months. And I can't. No one can. It's all bull shiitake.

Scrum does not solve this problem. But it's honest about it.

It says: "We can't tell you what could be done in 2 years. But we can probably tell you fairly accurately what we can do in the next 2 weeks. And if you let us proceed in these honest 2 week intervals, we'll probably get way more done than if we all agreed to the collective insanity of the waterfall lie."

If you've done waterfall successfully, meaning you delivered a project on time, on budget, and on scope based on your original estimates for all three, then congratulations, you're a mythical beast.

I, for one, cannot do that. I've failed every time. Every team I've worked with has failed every time. I've never seen it work.

> Nope, don't understand the trade offs if you're suggesting scrum is slower than waterfall. That just doesn't line up with my experience. "Results may vary," though.

The idea is that if you have a well defined problem domain and you know exactly what the software is that you have to deliver, you don't need the short cycles, and all the meetings and pokering and whatnot would just be a waste of time. You know what you need to make, so you can just continue working.

I suppose that makes sense "if you have a well defined problem domain and you know exactly what the software is that you have to deliver".

I don't have a lot of experience in such areas though, and i can't help but wonder how often that's the case. If my problem domain was well defined, the problem would probably already be solved by someone and I'd use their code. Why reinvent? Am I being too simplistic?

> If my problem domain was well defined, the problem would probably already be solved by someone and I'd use their code.

Not necessarily. Look for example at the team that made the code for the central computers of the Space Shuttle. They knew exactly what their code had to do, how fast, etc et.

I don't think he was saying that waterfall is faster; I've never heard anyone say that. But WF is more rigorous and probably better suited for safety-critical software like avionics; the trade-off is that it lacks speed and flexibility.

I'd like to add priority next to progress and problems in the second item. Being able to pull things in and out of planning is one of the biggest advantages over waterfall.

You’re absolutely right - and that’s the e tire core point of Scrum: to promote agile methodology (which can be summed up as you just did)

However the reason there is “bs” (by which I assume you mean the process/meetings/etc) is that most teams suck at being agile. Scrum provides a more rigid framework so that a team can develop agile skills and then, when ready, move beyond scrum.

Most people think Story Points or terminology like Standups comes from Scrum too - they don’t! They’re from XP.

Scrum is exceedingly simple and suffers from being over complicated and dismissed by those who don’t understand it,

The blog post above is no exception.

Reply on news.ycombinator.com