Agile Is Dead, Long Live Continuous Delivery
gradle.org
gradle.org
Both agile and waterfall methods attempt to give you a pattern by which you can predict when software will be delivered. That's the main business value. The suits don't care how the devs operate, provided they can get things on time.
Waterfall tried to do this by trying to understand the problem as fully as possible, to liken it as much to previously solved problems, and to involve gurus to say "that kind of problem will take X amount of time to resolve", and thereby create milestones and delivery dates.
Agile instead said "look, we don't know enough at the start of a project to do that. Let's instead keep track of everything we want to do. Let's try and estimate each set of tasks (stories), individually. And then let's rank them in priority. We can measure how good our estimates are, we can modify them, we can generate more data and determine what we'll have at a milestone, and then can either push back the due date, or at least recognize that we won't be able to ship the entire feature set at that time.
Continuous delivery can be done with either one of those (it almost never is in waterfall, but it -could- be). But by itself it offers a business nothing for planning purposes. All it does is allow an immediacy, a "as soon as it's done it's out in front of the users". This may or may not be a good thing to the business, but it doesn't solve the basic issue that business people want to know when they can expect a given set of features to be live.
The trouble with waterfall is that it tries to predict beyond the scope of a sprint, which just isn't valid. Project estimates are asymmetrical curves (likely poisson?) and you can't add them up and expect them to cancel out: http://www.sketchdeck.com/blog/why-people-are-bad-at-estimat...
The problem is like recursive state estimation, except when you break it down just to estimate and add it back up you aren't actually taking any new measurements.
On most projects it's usually just better to estimate based on relative size to your last project, start delivering continuously, and do a forecast of completion (as opposed to an estimate):
http://www.agildata.com/keep-it-lean-you-arent-ready-for-scr...
(edit: spelling)
I've experienced one -or- the other of those, rarely both. The only times I've experienced both were due to product owners who wanted to be managers, to bring 'leadership', and so insisted on wasting my time with meetings that didn't actually lead to well defined stories.
If I've had a product owner who wasn't a waste of space, who met with the customer(s), actually understood what they needed, and then met with us to help define a story, then while that story might take a good 10-20 minutes to fully flesh out with good acceptance criteria, it almost never changed during the sprint. We might, on the demo and/or release of the feature, realize modifications were in order, but that wasn't because we did things incorrectly at first; we did them correctly, with something releasable, and useful, and that allowed us to learn something that helped us to refine it.
However, more often I've had product owners who are wastes of space. They'll slap a story together, maybe with acceptance criteria of a line or two, maybe nothing, and then hand it off to the devs to go do. We roll our eyes, make some assumptions for all the missing details, do something, demo it to the customer, and get "That's not what I wanted! Change it!"
The former is infinitely preferable. Even if it's a highly complex feature, that takes a while to get understanding and consensus on, it almost never changes once we do and the devs, customer, and product owner are all on the same page. The problem is it requires a product owner who doesn't suck, and the reality is most of them suck.
As a product manager who hasn't spoken to any actual users, I'd like the following features to impress my boss.
As a product manager who overheard a Sr Manager muttering something...
We recently lost two or three bigger costumers because of stuff like this.. No one talked to the actual users.. The program was full of functions, workflows and solving problems no user wanted to get solved.
They just never used the programs only for situations like : we have (another new useless feature, please test)
The story is the last possible moment of decision making before the developers go and develop something. Obviously it may need further refinement at a later date once you have learned something. But, by definition, you now know -everything that can be known before development starts-, because development is about to start. As such, anything, -anything- that remains unspecified, is, again, by definition, undefined. It may be worth explaining that if failure conditions remain undefined, then you will ignore them for convenience, because whatever you decide to do will almost assuredly be wrong. If the product owners want -any- say in what it does in the event of something going wrong, and don't want it to be a surprise, and require even more work, then now is the time to say something.
Scrum is not trictly bijective with agile.
The concept of a sprint creates an entirely arbitrary deadline with an arbitrary box of work.
(I have a dog in this fight: XP with Lean trimmings, since I work at Pivotal and learnt it in Labs)
This assertion is nonsense. There's nothing magical about a couple of weeks such that it forms a boundary outwith lie impossible predictions. The validity of predictions depend entirely on the understanding of the problem domain and the complexity of the solution space.
The single biggest benefit from agile in theory, IMO, is controlling risk by getting the customer in front of the software sooner, so it can be iterated based on feedback. The primary risk being controlled is building the wrong thing. But you wouldn't develop e.g. an autonomous driving subsystem for a car that way.
Agile (scrum, specifically) in practice is too often used simply to chop large tasks into bite-size stories to be fed on a conveyer belt to a team of more or less replaceable programming cogs; the sprint scope keeps blinkers on everybody so they don't look too far in the future, they just keep munching through stories.
And when agile is used in this way, not only can it be demoralizing, but also extremely inefficient: a focus on user stories typically encourages building small features that involve narrow vertical slices through the stack of an application. That's hard to parallelize effectively - related stories will affect the same bits of code and cause conflicts. If you can bundle a bunch of related features together based on how they are likely to be implemented, you can slice them up horizontally, and implement the different layers separately, using things like APIs and data models at the boundaries. This paralellizes quite well at the team level.
It's still not great for software design. It's still a very blinkered approach; you're not going to design an application-specific framework that makes implementing features easy. It doesn't allow any space for experienced developers who have foresight, and relies on refactoring to create reusable domain-specific abstractions. But refactoring isn't a user story, and a team munching on stories isn't in a good position to think holistically about a problem.
Three months into an agile project and everyone already knows there's a problem.
In a waterfall project you know you are behind schedule.
In an agile project you haven't planned two years ahead and so you simply adjust future expectations.
You can say that again.
On the other hand, if you have a bunch of inexperienced, mediocre developers (myself included) then "bite-size stories being fed on a conveyer belt" is probably a good way to get productivity out of them. It's certainly a lot better than "you have 3 months to build this enormous system based on this 1 paragraph brief" which is pretty common.
Different people working on similar vertical slices through the system leads to slightly different parallel implementations and probably some duplication of helper logic. Refactoring won't get scheduled and it'll become technical debt in the way all duplicated logic is: not a big deal to start with, but an increasing source of bugs and features fixed / implemented in one place but not another.
Inexperienced developers won't be disciplined in keeping their abstractions separate; they tend to intermingle their abstractions so that the boundary is fuzzy. Specifically, they lack layering discipline. Instead of libA using libB which uses libC, they'll pass bits of libA into libB and return bits of libC. When implementing a complex algorithm that joins libA to libB, they'll write code that zips the two together within its convolutions, rather than creating adapters for libA and libB so that the algorithm follows naturally. And they'll model the domain, but nothing much more abstract, and write convoluted procedural algorithms in VerbingClasses (new ThingDoer(x).doThing(y)), possibly with interfaces for mockability.
And to be frank, many line of business applications can cope with this. The developers are cheaper and easier to find, and IT was always a cost centre anyway. It's no way to live if you love code, though.
And I'm not going to complain, because I benefit from the current system.
I'm sure there is an element of truth to this, but I doubt it's as decisive as you're suggesting in real life.
If you've got a good team of expert developers, people who are both individually skillful at producing useful code but also good team players and able to co-operate, I'd say you have a decent chance of them self-organising. I'd omit the "whatever the methodology", because these are exactly the kinds of teams who don't want or need some consultant's pet methodology limiting their options.
Of course, there is more to building useful real world software projects than just producing a good design and implementation. Even the most technically brilliant team also needs a clear goal to be effective in practice. Capturing the requirements and turning them into actionable specifications is a significant challenge in its own right, one which requires a very different skill set that even exceptional developers won't necessarily have.
I suggest that one of the big differences between a highly skilled and experienced team and a team with more modest capabilities is that the former will immediately recognise the need for clear specs and try to do something about the lack of them. The kinds of development processes and methodologies we're talking about today are designed in part to shield the latter from the same responsibility, but consequently they also rely on having very good people to do that work instead (and by corollary tend to fail hard if the communication with customers and resulting planning work aren't up to standard for whatever reason).
I wonder if this is an extension of 'programming by poking'[1]. You replace serious thought and planning with a piecemeal, try-it-and-see-what happens process.
It sounds like your product owner isn't doing his job. What specifically does it mean for requirements to change? If the specifications were unclear or incomplete, well you should have held out for clear and complete specifications. If you can't do that you have an organizational problem. But did the customer change their mind about what they want? Not likely, but possible I guess. The only thing left is that the owner didn't really understand the customer's needs.
The real question is test driven development / continual-integration worth doing? CI isn't too controversial, but for TDD there is no clear answer and it really depends on the domain and what language you work in etc. etc.
Product owners create stories. These are titled things like "As a (type of user), I want to be able to X". The point of the title is to determine who this actually benefits. Then, they attempt to define it with acceptance criteria. These are a list of "what does it mean to solve this need". Ideally, it implies a set of tasks, and gives a decent starting point for QA to start testing. This is stuff like "When the user clicks X, the system shall Y" and "Should the system fail to do Y, it will instead (failure mode), and (inform the user? Stay silent? Whatever). Sometimes the product owner needs help from the devs to determine this.
The devs will then add tasks to the story. The story should be able to be completed in one sprint; the tasks are, indeed, much faster. They're tracked only insofar as to see progress towards the story's completion, but they're not nearly as important as the story itself. A story with half of its tasks complete is not done; the feature is not implemented, it's not ready to go out. When all the tasks are done, the story is handed off to QA to vet; at that point the story is done, and it can be shipped.
The dev team is only ever committing to what can be done in a sprint. They should have an idea of how many story points they can handle in a sprint, such that they can work with the product owner to determine the stories they'll work on in that sprint.
When the estimates start lining up with what is actually achieved (that is, the team has a velocity of, say, 40 story points per sprint. And they're completing 40 story points per sprint), the product owner can start planning around it. "We have four sprints until the business wants the next milestone. As such, I have assigned 160 points worth of stories to try to get in for that". And that's reasonable. And then, if anything emergent comes up, or new stories take priority (a 'pivot', if you will), the product owner knows they can't manage it; they either need to replace a currently existing story with that emergent/newly prioritized story, or, they need to slip the schedule.
And that can actually work. It requires honesty, transparency, and a desire to actually get shit done, but it can work. The problem is oftentimes people or cultures value CYA, politics, and 'leadership' over getting shit done, and all of those make agile little more than a scapegoat for why everything is on fire.
And it's only one way of being agile. Kanban, for instance, still has stories and tasks as described, and it still offers velocity, though measuring it slightly differently, but it isn't concerned about the sprint boundaries.
If estimating is what you want to learn, James Shore has some good posts on the topic:
http://www.jamesshore.com/Blog/Agile-and-Predictability.html
When you state that a team should know how many Story Points it can handle, it would make much more sense to see Story Points as some measure of time. If you've completed last few Sprints 38 to 42 Story Points, with 2 full-time devs, than one could state that a single developer on average can finish 20 Story Points in Sprint. And if a Sprint is 2 weeks (10 days), that would mean maybe 2 Story Points per day for each dev.
Yet, in every business environment I've been in, the SCRUM fanatics always state that "No, story points are a measure of complexity". In practice, to me, this makes no sense. At least not if one wants to use historic Story Points to estimate how much work can be completed in the next Sprint.
That said, there's a good reason to say it measures the complexity, and not the time. You want to keep distance between your estimates and time, or else the business people are likely to come to you and say "You said this will take 6 hours, so I expect it tomorrow". Nevermind that it requires another task to be done first, that it's blocked on getting something from another department, that you only ever committed to delivering it at the end of the sprint, and that you underestimated how long it would take anyway (but it doesn't matter because you overestimated something else so it comes out in the wash), -you- said it would only take 6 hours!
For ease of use, let's say that a story point represents 8 hours of actual work. That gives us 168/8 points a week, or 21 points. Of course, we lose 7 of those to sleep, 4 to the weekend and another 5 to the evenings during the week, leaving us with 5 points per week. You probably lose a point to lunch, coffee, bathroom breaks and mingling with coworkers and another point to meetings and interruptions.
That leaves you with 3 points out of 21 potential points in a week, and you aren't getting 3/5 done per day. Using story points instead of hours tries to get people to treat the sprint as a unit instead of time as a unit, since time can be split up to a very fine degree. Just because you work 40 hours a week doesn't mean you can complete 20 2 hour tasks.
Good waterfall project managers do that too. You can't bend reality to fit a gantt chart.
Call me an agile zealot fanboy or whatever, but that doesn't feel to me like progress (and, inter alia, more or less the opposite of what Dave T was complaining about in his own agile is dead thing).
I would also add that, in my completely personal and anecdotal experience, they are not one-fits-all methods: in general I find Agile much more suited to developing a "product", while Waterfall makes more sense when the aim is to deliver a "project".
Agile grew as a bulwark against the bad old ways of the "enterprise" trenches that led to the original "software crisis". There was a time when big software projects would not deliver, at all, as often as they would succeed. And even when they succeeded they often delivered the wrong thing. Daily builds, continuous integration, using the software itself as the source of truth (instead of elaborately negotiated specs), and focusing on iteration, these are how you can extract productivity from any team, and how you can keep on target and deliver something of value. It's not necessarily the best way, but it's one reasonably reliable way to get stuff done.
Nothing about waterfall is obviously bad. It's how most engineering projects are done. Heck, it's how most projects generally in the world are done.
There's a reason it took a long time to move away from waterfall. It solves obvious problems and meshes with how the rest of corporations typically operate. In fact, with software delivery methods which were dominant back then, agile probably would have completely failed.
---
In the end, I ended up reading the full piece. One doesn't get much more oblivious and buzzwordy than this:
> We are now more squarely in the age of Microservices, Mobile first, Polyglot, post-Java JVM languages, GitHub, Docker and the emergence of a world being eaten by Software.
How is this piece on the top of Hacker News? This piece reads like it was written by a marketing consultant with no idea what these terms actually mean or how software is actually built.
Gradle provides only one language, i.e. Apache Groovy on the JVM, for specifying build configuration info. Before talking about "Polyglot, post-Java JVM languages", perhaps Gradleware needs to eat its own dogfood and provide an API so users can write their build configs in Jython or JRuby if they want. Be polyglot before preaching it.
Yeah, let's replace one consulting fad (the 4 or 5th I've seen in my career, I entered when "Waterfall" was still in vogue, then XP, then Agile, some variations of each too) with another.
How about this methodology: http://programming-motherfucker.com/
Is this not true? I'm too young to remember what came before agile.
However, there was a waterfall-like process that was widely used. After getting the reqs and design in place, you started coding and you mostly coded. You would compile as you went along and made sure that your code worked on the handful of tests you put together as you went along. When the app was mostly coded, you then started more serious testing: writing more elaborate tests and verifying functionality. This step often highlighted buried errors that you then spent time debugging, writing additional tests, and so on.
So, the biggest innovation in Agile was the latter's orientation towards developer testing concurrently with coding. Which, of course, has had many other ramifications.
While the old way of doing things might sound klunky by today's standards, when done right it actually worked better than its reputation. Because devs hated spending hours in the debugger, they tended to code very carefully. The concept of "let's code this and run it through some tests to validate it" was unknown. Rather you coded so that you were pretty much sure that what you were writing worked properly if it compiled without error.
Yes it was.
> Think about it, how could you possibly code an entire app and only then debug it?
Which is why programmers hate waterfall so much, but you're absolutely wrong to think this isn't exactly what was being attempted. Time and time again, management in an attempt to cut the cost of programming time, thought the way to do it was to first build specification for everything and try and prototype the entire app up front, storyboard every screen, build out mountains of specs because naturally they think that's how you build things, with detailed blueprints so all the decisions have already been made. They couldn't be more wrong, but it's certainly the most natural way to think if you aren't a programmer and don't know better.
> So, the biggest innovation in Agile was the latter's orientation towards developer testing concurrently with coding. Which, of course, has had many other ramifications.
Agile in general had nothing to do with testing, the big change in agile was removing the big design up front, the specs and meetings and months spent planning something before being developed. Some agile methods like extreme programming certainly had testing as a big part of their process, but what differentiated agile from waterfall was introducing iterative programming where work was done in short week or two cycles and then delivered whereas waterfall wastes enormous time trying to nail down details that simply ended up being wrong come programming time.
tldr; agile is not about testing, it's about iterative development in short cycles with little planning and always has been and that's what made it different from waterfall; no "Big Design Up Front".
This is correct.
The first paper to describe a stepwise model was by Royce in 1970 [1]. The model he is describing is hypothetical and does not use the term Waterfall.
The first use of the word "waterfall" (including the quotation marks) is from 1976 [2] and specifically refers to [1], the hypothetical model. In [2], the writers specifically state that "so few" projects fit this scheme.
If you were programming in the 80s and early 90s, you would know that no one in programming ever referred to a "waterfall" model.
Even Kent Beck's seminal book on XP, written in 2000, describes many failures of software development in those days, but he does not once use the word Waterfall.
So, in summary: the paper that supposedly describes it describes a hypothetical system; the paper that first uses the term mentions how little the model is used; the term wasn't used by people during the era it was supposedly most popular; and the folks who initiated the new generation of software dev don't refer to the model.
I think it's safe to say that it was not a thing. Or, to be more accurate, if it was a thing, it was never the thing it became until the Agile consultants used is as a strawman.
[1] http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/wate...
but it absolutely was very common to use the process that it implies.
many man-years of gathering requirements, building lists of features, interrogating stakeholders.
people knew they would never get a second chance to get what they wanted, so they would throw every feature in that they could think of.
Then the developers would build it. more man years building. mostly they tested as they went along, but....different features and areas of the app were often build in isolation, so the most you could say was that you had tested what you had built.
the stakeholders were consulted with screenshots, and sometimes the app, but the Big Fear was new features and additional requirements (which would cause massive contractual headaches) so any contact was VERY carefully controlled.
Also it was assumed every project was massively valuable IP, so detailed designs and features were held very close to the chest..again, any contact was VERY carefully controlled.
Eventually, after months or years of effort the various parts of the app would be "merged". omg the cluster fuck that would occur, the compromises, and arguments, the agonizing...
..then more testing..
Once that was completed the brilliant new creation would be seriously unveiled to the users to cheers and relief.
Finally, the contractual battles would begin as the paying company realised they had asked for entirely the wrong thing, and the development company tried desperately to cover their ass and make a profit.
The single biggest problem that Agile fixes isn't the testing, its the disconnect between what the client wanted and what the developers actually deliver.
I ran projects that ran through a VERY traditional waterfall process as recently as 2010.
There were also several books on the matter, and also revised waterfall models. The very term was not always used early on (though around the nineties it did), but the schemes were the same.
Lots of public projects, including some I've been involved, were designed and managed in such a way.
There were no changing requirements until the things were delivered, which could take 1-2 full years. The design phase resulted in monstrous 400 or 800-page documents covering every aspect, and to apply for such a software tender you needed to write those in excruciating detail (I did that too for several projects).
The waterfall model, with the name and all, was also taught at our university (early/mid nineties). XP wasn't even mentioned back then at our level, though we discovered it on our own, reading Fowler, Beck et al in the late nineties.
There's some tinfoil conspiracy theory thrown around that Waterfall was never a thing etc, and it was just used to push Agile etc. I wonder what its adherents did in the 80s and 90s, but surely not programming in large enterprise/public/military etc software projects.
(That Waterfall cannot work is another thing altogether -- nobody did it 100%, as nobody does Agile 100% today).
Waterfall still isn't dead and will never die because it's how people think by default, plan it all out, then build it. That's how buildings are made and thus it becomes the default methodology of all new managers who don't know what they're doing.
Much like most "scrum" teams don't really do by-the-book Scrum, it was very rare to do by-the-book waterfall.
If you defined waterfall to mean
- get every requirement written up in absolute detail before you do any design
- finish the design in great details before you write any code
- if the design process identifies problems with the requirements (missing/ambiguous/etc), then stop the design work and go back to requirements phase
- write all the code before you start testing.
- if during development you find an issue with the design, stop development and go back to design
etc
Then I never saw any project that worked that way.
But it was quite common to have a requirements gathering exercise, write up a document that covered the requirements, get that "signed off", then do a multi-stage design phase (usually we'd start that before requirements were signed off, since the probability of the requirements being rejected in their entirety was pretty low), sign that off, and then move into development to implement that design. And then test the whole thing at the end once it was "done".
If at any stage we found missing requirements, that would get raised as a scope change. If, during development, we found that the design was broken in some way, we'd have a design change (which was usually lightweight and was a couple of emails saying "we can't do X, so we're doing Y, OK?")
The thing to remember though is that the 'pure' version of the process (in which you never return to the previous level - hence the name 'waterfall') was regarded as an ideal, not as something achievable. Nobody expected that there would be zero bugs found in testing that would require more coding work (why test otherwise?), but it was regarded as a kind moral failing, to be met with a lot of hand-wringing about why we can't be like real engineers, who never make mistakes &c. And the same applied to revisiting the design or requirements phases.
The next level of idealism held that even if avoiding returning to a previous stage was impossible, you could at least limit it to the immediately preceding stage. As Royce pointed out in the first paper to provide a convenient diagram of the established software lifecycle to cite, this never happened in practice either.
Over time the necessity of doubling back became less of a moral failing and was incorporated into the teaching of the process, without any acknowledgement that the entire concept was fatally flawed. It was also succeeded by several iterative but still waterfall-like processes (starting with Royce's paper and eventually followed by Spiral and RUP). Thus there are many folks who still believe that the pre-Agile process (they may not recognise that there was more than one) was just fine, and that the pejorative 'waterfall' was applied to something that never really existed. In a certain sense that's true, but it gives a completely misleading picture of the history.
Agile's contribution was the idea that any or all of the phases could be happening simultaneously - and, more importantly, that was OK. Plenty of people had been ignoring the orthodox methodologies and working that way, of course, but don't believe anyone who tells you that was an orthodox idea before Agile.
"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software." (Emphasis mine)
Continuous delivery has been an integral part of agile methods since the very beginning.
just use your brain. agile is only a problem when people follow it religiously instead of intelligently imo.
its just one tool in the box, not the ultimate methodology.
But feel compelled to add: Agile is only a problem when teams pretend to follow it but actually don't.
Some of the points in the Agile manifesto are just arrogant, unjustified statements of fact or vague and useless comments.
Some people hate face-to-face interaction... and what the hell is a 'regular interval'... and sometimes a clear vision and direction from above achieves the very greatest of results that the team without it would have failed to produce.
Its not useless, but its not without fault either.
i prefer to think that a master craftsman is one who can take any tools and materials and use them to produce good work... if not exceptional.
If you are forced to use an inferior tool like Agile, then sure, you want to make the best of it and be a craftsman who can still succeed.
That is an endorsement of being adaptable and self-reliant as an engineer, not an endorsement of Agile.
And if a tool is bad like Agile, it's useful to point it out and slowly steer the bureaucracy that feeds it into bastardizing whatever the next tool is, but hopefully inching forward to a better global state as well.
Doing awesome things with primitive tools is a parlor trick. Grandstanding.
The craftsman blames themself for using the wrong tool for the job, not the tool.
Agile is actually hard to get to grips with. It requires lots of discipline. So did the pre-agile methods that worked best. And it gets half-done for the same reason that the pre-agile methods got half-done.
It's hard. Hard to do, hard to learn, tempting to stop.
Agile process without agile culture objectively isn't True Agile. It's worse than everything it sets out to solve with the alternatives.
Putting aside that particular nitpick, it's easy to cargo-cult the practices. I sometimes use the analogy of the introduction of lean manufacturing in the US. At first what was introduced were the tools: boards, line-stoppages etc. But tools are just tools, by themselves they're not enough.
The uncomfortable, expensive and difficult truth in our industry is that people, process and tooling are not substitutable. You need the best of all three that you can get.
The agile manifesto has plenty of problems that should be obvious with some thinking. Some of the statements are outright ludicrous and naive as much as others are valuable insights (but even then, only if you haven't thought about the problem very much and tried to approach it in reality).
I shipped software very quickly before learning much about agile at all... I ship it even faster afterwards, but not because I agree with it all... in fact I am convinced I am better off for ignoring the more harmful aspects and selectively choosing the ones that have a measurable benefit.
Are we talking about the same document? What are the problems and harmful aspects that you're referring to? (Please quote the relevant part) This comment feels either a bit like a content-free middlebrow dismissal, or we're referring to different documents.
From the very first.
> Individuals and interactions over processes and tools
Have you seen this in practice and experienced how people work day-to-day? Leave them to their own devices and it varies a lot by personality. Some people need micro-management for instance... there is also thing that process is very important, and this point is self contradictory given that a lot of the agile manifest essentially describes process or a strategy for creating one... and as for ever relegating the importance of good tools.... ick.
?
(p.s. thanks for calling it out, its good to take these things seriously imo. its shows passion and care. :) )
That sentence is important for interpreting the statements.
But that is the point of the sentence. A process does not care about individual variation. What the manifesto is saying is exactly what you are saying.
> Individuals and interactions over processes and tools
needs to be read in conjunction with:
> while there is value in the items on the right, we value the items on the left more
Which means I can interpret it a way that is true, but not that interesting.
That is, I believe that a great developer with crappy tools will produce better software than a crappy developer with great tools
But I'm not sure what that really tells me other than "hire great people".
I guess, to some extent, it means "if the process says I should do X, but the team thinks it's a bad idea, then I should listen to them", but even that's is not a necessary conclusion from the statement. In the non-software world, you only need to spend a little bit of time looking at workplace deaths to see that many of them are caused by teams that decided to ignore safety protocols.
Ultimately the only thing I really take from the agile manifesto is that doing the exact opposite is bad, but so what?
I don't want to ever work in development team that:
- Thinks that "following a process" trumps "investing in good people"
- Prioritizes writing documentation for software rather than making it work
- Has watertight contracts but ultimately disappoints the customer
- Sticks to "the plan" even when it's obvious that the circumstances have changed.
But I didn't need a manifesto to tell me that.
For example, let's say that we did some project planning a month back, which says that we should start on a certain new project tomorrow. Everyone on the team is grumbling and no longer feels like that's a wise expenditure of resources. What should we do? Stick to our goal and follow the process, or get people together, work it out, and set new goals? Agile is saying, especially when you consider other bullets like "Responding to change over following a plan", that people subscribing to the Agile Manifesto value getting individuals together, collaborating, and responding to change more than following their plan.
Apparently the author hasn't really read the agile manifesto, because nowhere does it talk about scrum or sprints or xp or any of those things.
The agile manifesto is essentially a few sentences, that very eloquently say
"Don't be a dick and understand that the product people or customers don't really know what it's supposed to do either."
Much like the bible took Jesus's essential point "Don't be a dick." and wrapped a terrible mess of a system around it, the various agile methodologies have done the same, but the essential point of the manifesto is as relevant today as it was when it was invented.
(Note: I'm not actually religious, the Jesus thing was just a comparison)
http://programming-motherfucker.com/
Customer: errr.. what methodology are you following?
Developer: none of that agile garbage I can tell you...toxic stuff. I am programming.
Agile never assumes project death is a reality. It philosophically implies that you can always measure and iterate your way to the next phase... but that's just not true. No one measures and iterates their way to a dead state on purpose and yet, projects end up there all the time.
So how are the projects getting there and what is Agile doing about it? And don't give me the Agile vs. "Agile" argument. Assume a flawless execution of agile principles with completely accurate measurements and successful iterations took place when answering that question.
To me, agile just assumes infalsifiability because it refuses to acknowledge, let alone, prepare for project death. And worse, according you agile, you can even measure and iterate your way out of the grave.
The operation of a projects has more in common with biology than immortality. Treating projects like immortals flies in the face of the reality of project lifecycles and it brings very bad mental states to problem solving. It makes far more sense to treat a project in lifecycle phases (birth, child, youth, adult, mature, elderly, dead) than assuming a singular monolithic philosophy that completely ignores the arc of a project.
A: "The code base is too unwieldy, no one understands it any more"
B: "Oh, just measure and iterate"
A: "But that just creates more unwieldy code!"
Agile makes sense as a subphase of a project lifecycle for when that lifecycle is spinning its wheels in the mud, not as a end-all-be-all philosophy.
TL;DR: Agile should used if a project stalls, not as a guiding philosophy.
In fact, regular deliveries are by many considered the single most important indicator of a working Agile process.
It'd be nice if people could just layout concrete development / release plans and not assume everyone has the same interpretation for the buzz word that happens to be in vogue for a particular management team.
Oh, you didn't read all those development methodology books?!
I'd sort of like to speak plainly and get programming done, personally.
- Agile is Dead because people abused it, so we should move on from the term. - Instead use the term Continuous Delivery. By the way this term doesn't mean anything, use whatever system you want. - Lots of buzzwords.
We will proclaim agile is dead without realising we constantly adapt our code to changing needs of our users and their representatives who are very deeply involved in the whole process and who give us feedback as we constantly release new functionality.
I was just having this thought last night. "eXtreme Programming" and Scrum seem to either exist in a clunky bureaucratic mutant form or have evolved into companies that use CI and prioritize really good communication. (Yes, people in the late 90's and early 00's did sometimes write it like that.)
>Individuals and interactions over processes and tools
>Working software over comprehensive documentation
>Customer collaboration over contract negotiation
>Responding to change over following a plan
The process we've designed and use daily at Gridium most definitely adheres to those principles. We've tested and abandoned the idea of "sprints" and weekly estimation cycles. We have no idea what our "velocity" is.
Yet we collaborate with customers nearly constantly. We are highly responsive to changes in the business. We integrate continuously. We deploy very often. All ideas that I think where present when we all fell in love with the idea of "agile".
* Edit: Shame on me for not reading the article, because I basically summarized it :) Heap your downvotes on me HN!
I also have given up on fixed sprints for my (not anymore) scrum team. We have a prioritized backlog and whatever is next gets done. More like Kanban. As long as people behave like adults and do their work efficiently it works pretty well. We still have a weekly velocity and that number is surprisingly reliable.
That makes absolutely no sense because the sprint always finishes, on a scale far basis.
The only way to make a sprint not finish is to submit code that cannot be safely deployed, and that is a bug in the code, nothing to do with changes in plan.
But as you pointed out, daily scrums can essentially devolve into what is essentially a policing of developers to make sure they are staying the course and are reminded daily of their deadlines.
It's actually just an unusually unfriendly version of waterfall. Why? Cause at least in waterfall, the business users are forced into something unfair and impossible as well. It's impossible for developers to accurately estimate completion dates, but it's also impossible for business units to accurately and fully spec out a software application. "You didn't meet your estimate"... "Yeah, well, you changed your mind".
It's a nasty business, but there you go. Best to avoid the entire thing and work on projects that are very high value but aren't deadline dependent, where the value of a developer is measured by working software at reasonable intervals rather than daily discussions of sprints, stories, and deadlines.
Actually, in some ways, that sounds more like what agile is supposed to be than all this scrum/velocity/stories stuff that isn't in the manifesto in the first place.
Or do it like anyone who wants to build a business that lasts longer than 2 years: use stable, battle-proven software only and don't run after every hipster trend (remember EmberJS and friends). Before you use any software component, evaluate its support lifecycles and how well it is being maintained - especially with the trend of "modularization", where stuff like a string pad function is a separate module, every additional dependency you pull is a potential security issue.
Oh, and don't make the cardinal error of putting your node_modules/vendor folders into .gitignore. THAT is build hell once someone unpublishes a popular module...
-- The Who
The other side of the coin is that defending agile falls into the No True Scotsman trap too easily: Since people don't practice agile, failed projects aren't agile. Well, of course.
The real problem is how do you get the people who could be doing agile to be motivated to access the benefits of agile, and how do you get the borderline cases to acknowledge their limitations and address them.
It's not bad, but I still don't really like scrums.
I think our process (which is fairly informal) would fall apart if different people tried it.
1. What's the next task/bug/story we should be working on?
2. Build & test it.
3. Ship it.
4. Rinse & repeat.
We'd have meetings periodically to define new sets of tasks and figure out priorities, and/or to reflect on things that hadn't gone well. But these weren't necessarily scheduled on a regular basis. We did them as needed.
I then swithed major to mechanical engineering, and there "toyota production system" is the thing. Agile and lean are just taking the underlying idea bit further. The original idea is to eliminate intermediate storages. This happens by making every production cell to order shit from previous production cells via kanban card. Previous idea was to just optimize volumes of every cell while giving zero fucks who, when and why the product was needed. But even inside the realm of mechanical engineering this sometimes is plain stupid. For example the workflow of large castings doesn't follow rythm of manufacturing cells. It follows the rythm of curing molds, heating metal and waiting the metal to cool down. What is "Agile" in software seems to follow TPS somewhat. You do one unit of work, you cash that in and start on second one. Id guess it's just coincidence that short compiling times and intermediate storage elimination happened during same timeframe and that resulted one term for both "agile".
I have very limited experience on coding, but neither of the previous seem to fix any underlying hard problems in coding. When I code, I struggle with interfaces, specs, debugging and data flow. The only thing that comes close to coding is design of mechanical parts. But that has lot less degrees of freedom. And there seems to be pretty much zero management fads applied to mechanical design anymore. The only thing everybody is saying is that "try to keep open mind in the beginning of process. And spend some serious time defining your problem, because that's what affects total costs most". The rest is just going by gut feeling.
We have ISO and ASME, but still interfaces are some of the most tricky subjects around in mechanical design. I'd expect the next meaningfull paradigm for coding to focus on interface design, as it's the one that requires communication between human beings. And it sets limits to what individual coders can achieve.
"Never stand between craftsman and his professional pride" Quote by Kalevi Aaltonen, but I guess he got it from somewhere.
The source code is about 6000 Java files + 3200 groovy files besides many others. To me it appears excessive and over designed software. I will be very reluctant to use a build software which needs 300+ pages user guide.
TBH it is entirely possible that I just do not work on such complex software projects which needs tools like these.
If your company has trouble developing software, the problem has nothing to do with development processes. The problem is always management.
Upon reflection this seems like a proto-agile methodology, and because it doesn't have its own set of buzzwords and extra processes it seems like it could be a good reactionary (as a reaction against full Agile) methodology to use.
Manifesto for Agile Software Development
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
I think this manifesto is wise. Like Zen koans, contemplating and discussing it provides an understanding about what Agile is. Fortunately it's not cryptic, although the meaning behind the statements might become clearer with consideration and experience. "Individuals and interactions" is saying, focus on getting people together and talking and working things out, not following a predefined process or using complex tracking tools. "Customer collaboration over contract negotiation" is written, I assume, in the context of contract work, but it applies generally to the relationship between the software team and stakeholders: collaborate with your stakeholder directly, rather than trying to negotiate some kind of specification (contract) up front. The Twelve Principles of Agile Software are also worth a read: http://agilemanifesto.org/principles.html
These principles are what Agile is about -- not some particular methodology. Indeed, blindly following a methodology like Scrum would violate the Agile Manifesto ("individuals and interactions over process"). One of the things that I like about the Agile Manifesto and Principles is that they're not prescribing a particular method; they are instead providing some principles to consider while thinking about a situation. I've always liked Amazon's Leadership Principles for the same reason: https://www.amazon.jobs/principles - insight comes from thinking about what it means to embody a principle in a given situation; or what it means to value one principle over another, in what situations, and why. It's not prescriptive nor a method, but it's a starting point for discussion, and a way of talking about what we value as individuals, a team, or a company.
This should be about contracts between software development firms and their customers, where the development firm gives a quote to complete a project based on very detailed and legally binding specifications - which is a big problem because you can't know every detail before the project even starts. Hence a big part of the "Agile" philosophy.
If no one's doing Agile, how could it be dead, since it was never alive? Maybe we should start trying to actually "do" Agile, to see if it might work, since no one's ever tried it yet, according to this blog.
Sure, for our clients and internal developers, devops makes CI happen. But as far as the devops themselves are concerned, Agile hasn't gone anywhere.
sigh
I can see how Docker is relevant, Microservices are arguably relevant, maybe even GitHub, but that still leaves 4 out of 7 items that have nothing to do with continuous delivery.
I think the author is confusing eulogize for euthanize, which amuses me.