Why don’t software development methodologies work? (2014)
typicalprogrammer.com
typicalprogrammer.com
1. The paper methodology that everybody completely ignores - this is Scrum or waterfall or rational or whatever they call it
2. The management ideology that management actually insists on and bases your pay on - the ticketing system (nowadays it's JIRA, but there's always been something similar). In theory it's driven by the paper methodology but in practice it's somewhat random.
3. The support methodology that everybody follows - drop everything and work on this emergency because Somebody Important mentioned it. This is what your reputation is built around, and is always independent of the other two methodologies.
In my company at the time, they had a million rules and processes, but everybody had to drop everything because the son of an influential client could not get toy cars (!) into the new enterprise retail procurement system because of some incorrect logistical data. Normally you would get a ticket, analyze etc.
But he was the loudest/most influential, so everything else was unimportant. No Jira, nothing, just one angry email. Fix it for him alone, NOW! Billion-dollar company, thousands of clients but a dude with 20 toy cars won.
The issue was resolved (we removed the validation rules that were implemented before because "this is the filter for incorrect data, and we want to clear our data by restricting them") and everything went back to normal. Jiras, process and so on.
Well, until review time - then, "why didn't you make any progress on your Jira tickets?"
I always found that tickets are protection. When you're being asked to "drop everything for X", my first thought is to create a ticket for my own protection. At the least I can track the fact that I had to spend N hours on this thing X and I can refer back to it. At best I can document the stupidity of the whole thing w/ the ticket contents.
If you're in a place like my sibling where they ban creating a ticket yourself and assigning yourself: run. That's so stupid for so many reason that it's not even funny any longer
I can give you my answer: Because the ticketing system is painfully slow to log in to, create the ticket, move it to the proper status, etc. Single sign on can make things worse too. The whole process can add up to easily a minute, and not a productive minute of the actual filing (I would be fine with that) but a minute of poking at things.
This does not make you wrong. If you want to say there's some laziness there, yeah, maybe. But it's not just laziness, there's a psychological torture aspect to it all too that is, if not "objectively" true, then at least not entirely my imagination either.
I really, really, really wish Atlassian would basically drop everything for a quarter or two and work on the speed of their products. There shouldn't be a fundamental reason why their stuff is so slow.
Login processes can suck. Absolutely. But like you say, single sign-on means you should at max be logging in once per day if not even less.
So just for giggles I timed various things in our instance for you. I have a logged in single sign-on session already.
Opening the Jira instance to my default dashboard until I could conceivably click on the "Create" button: 3.5s
Click the "Create" button until I could start typing into the summary field (while the rest still loads): 1.1s
Now as for filling out what you need, that obviously depends on your company and they can make it a pain, agreed. If we're talking about the "someone tells you drop everything now and to work on X" I doubt it takes more than 30 seconds to fill in even one of the "wild" issue creation screens. I mean you literally have to type something like "Exec B made me work on X" into both the summary and the description field, maybe choose an issue type and select the first value of any dropdown required fields and type "a" into any required input fields.
Actual ticket creation duration shouldn't take much longer either and even if, who cares? By that time I'm already back in my IDE.
I'm not who you're asking, but I have timed this at multiple workplaces. The most egregious example took 32 minutes end-to-end (for many reasons: Jira was behind a VPN, which was extremely slow to login and even slower to actually use, the SSO system only allowed a session to persist for an hour, the server actually running Jira was underspec'd, the ticket template was incredibly verbose, etc, etc, etc). I am excluding the time it took to type the ticket in the first place (I wrote up the text beforehand and copy/pasted it into the fields) and including the time it takes to sign in to SSO, log into the VPN, browse to the ticketing system, etc. Essentially, I assume that it's the start of a work day and the only thing I've done is boot up my machine. Excluding the 32-minute workplace, I've got an average of ~9.5 minutes (only one was < 60 seconds).
While many workplaces have good systems, don't assume that every workplace does. There are many out there who accept/mandate potato-quality tools.
I asked my parent if he timed it, because he mentioned "a minute". Lots of things feel longer to us than they actually are. If you're waiting for something to open or react to a click or other input even half a second can feel long and we're bound to say things like "whoa that took like 30 seconds, WTF!". But when you time it, it actually takes way less.
E.g. opening an issue directly from a new tab in Jira takes about 5 seconds for me in my quick test for example. Yes, that's absolutely a "long" time for what it ultimately needs to display and do. But it's also nowhere near a minute and if all you gotta do is to open a ticket someone sent you because you're supposed to "work on ASAP", these 5 seconds don't matter. You'll wait about the same time to run your unit tests after making the required change. You'll wait way longer to run the entire test suite etc.
It took about 15 seconds to load the ticket page, and another 5 seconds to process the unsubscribe. I just tried again now, and it took "just" 3 or so seconds to load the ticket page.
Their p50 time is already unacceptable, and their p90 time is completely absurd.
We've known what acceptable delays are since 1968[1]! Why is it so hard to build software that meets these requirements? Our computers are only 20 million times faster than back then...
At the same time, why would they? Your employer bought it anyway. And I guess the execs won't notice these HN discussions ...
[1]: https://linear.app/
It is annoying to click on something and wait 4-10s for the page/board to load or the login redirect dance to be completed.
So, what happens is that you click, and you go to do something else like reading another page in the meantime.
And then you completely forgot what you were about to do and when you remember you are already logged out or you have to refresh the board again, and it is slow again, so you go do something else in the meantime, etc...
And so on you lose a lot of time with frustration.
I think many developers have a vague understanding of how companies work and how their role fits into the company at large. I know a lot of developers who want to focus only on solving technical problems, want to be given the freedom to make technical problems, and want to have the other problems solved by everyone else.
If you’re focusing on technical problems, a ticketing system is a burden imposed on you by management. If your priority is the success of your entire team rather than the success of your own technical decisions, then you’d understand that visibility of your work, your thought process, how you justify decisions, is all critical knowledge that a ticketing system serves to record and disseminate.
I can’t count the number of times where I found a reference to a bug in a piece of code like:
// Release gadget early to avoid starving froopy pipeline,
// see bug #12345.
In other words, do their job and not do everybody else's job for them? Or conversely, get paid everybody else's salary since we're already doing all of their jobs?
If I’m interpreting your comment correctly, there’s a lot wrong with what you’re saying. First, the “job” that computer programmers do is much more than just writing code and designing systems. Computer programmers are expected to prioritize work, advocate for work that they think is important, gather requirements from stakeholders, make decisions about who holds various responsibilities, document and communicate the process to team members, and disseminate knowledge about how systems work to others. Just for starters.
Second,
> Or conversely, get paid everybody else's salary since we're already doing all of their jobs?
1. Computer programmers are paid generously, at least where I live. One of the reasons we are paid generously is because computer programmers are very efficient at getting work done that may otherwise need many people.
2. It’s not somebody else’s job to follow you around and document what you’re doing and justify it. It never was. It was always your job.
“Keep your head down and just do good work” is a kind of hyper-individualistic fantasy. It’s the fantasy where you’re some kind of powerhouse getting work done in your own world, and you do not need to advocate for what you feel is important. That’s what I’m arguing against. The technical part of our jobs is only one part of it, and you can’t make a good team out of several highly skilled programmers if they only have technical skills.
Fortunately, now, we're SAFe.
Were you unlucky enough that they actually had checks in place or was it the typical "these systems are 'integrated' but not really"? If no checks, then to hell with the remedy ticket. I'm not putting any. If it only checks for not-empty field, I'll fill in junk. If they do check existence, I'll find an unlucky Remedy ticket that gets stuck with all my Jira tickets. If they check that the Remedy ticket is actually still open, I'll somehow find one of those Remedy tickets that are forgotten and hopefully will never be found and cleaned up (unlikely anyway) and that one gets stuck with all my Jira tickets. If only one Jira ticket can get associated with each Remedy ticket (which is again stupid for so many reasons ...) I'll use subtasks on one special "junkyard" Jira ticket.
I'm sure there are more cases you can think of and where we can find workarounds that still allow me to publicly document what I worked on.
What I generally want is someplace to cheaply mark an action item to create a ticket, so I can jot down a quick idea, then come back to write the heavy weight ticket from that later.
If I'm aiming to jot something down, it's because im focused on a work item, and don't want to fully context switch away from that item and into the user story and background details for the new ticket. It's not that it's hard to write the ticket, but it's hard to resume the work you were doing before you were prompted to open a ticket
FWIW: I've consulted at places in the past where developers couldn't create tickets, only update them. It was most definitely a way to attempt to keep them under control and harder to advocate for themselves.
For example, one way to classify process changes is when they make problems obvious. E.g., a release-early, release-often approach makes problems visible early on. And I've had great luck with them on teams that are actually trying to succeed, because faster feedback means faster learning. But most organizations reject that because actual success doesn't matter nearly as much as managerial appearance of success. Many managers thus hate fast-feedback processes because they would rather hide negative information, even if that makes long-term success much less likely. But they like the appearance of success that comes with using the latest buzzwords. This results in the common pattern of the manager who talks Agile but whose projects are in effect waterfall.
One book really helpful to me here looks at the default ideology of current management: https://www.amazon.com/Confronting-Managerialism-Business-Ec...
All of those are true for some markets, but it's no way to create new products because it breaks core feedback loops. Companies that don't have a way to improve will eventually get replaced by those that do. And as you say, they might then become the next company forced to slow their pace by their customers. But I am hopeful that even enterprise companies will learn to move faster.
Feature flags and great communication with customers and empathy means you can have frequent releases without annoying the customer.
It should be strictly better than slow releases. You can emulate a quarterly release of 1000 changes at once if you want that. Or you could say release fixes and small feature continuously and big features are timed with the customer/training/support functions. They might even be beta/optin at first.
Getting the code out fast always and then providing ways to avoid dumping confusing features on people seems better to me.
That doesn't mean methodology #1 is useless. Agile/scrum really did have an impact on the industry, and the proof of that is how hard it is to find people who still advocate for the kind of waterfall methodology that was once accepted by pretty much everyone.
The annoying thing about development methodologies is meeting people who don't understand that the map is not the territory and insist that a perfect application of a methodology is absolutely necessary. They drive everyone else crazy with buzzwords and process. Most people are fine with a sort-of agile process that takes reality and business value into account.
I wrote more about that a long time ago, and still stand by it: https://williampietri.com/writing/2011/agiles-second-chasm-a...
And, yeah, I think a lot of devs would agree that the traditional waterfall is not as good a process as agile. You could say that it's an antiprocess, whose basic idea is that the old inflexible waterfall with its overly structured teams is inefficient. Yes, small, tight-knit teams working closely together and iterating quickly is much more productive, creative and rewarding way to work.
... although, in my humble opinion, the one thing that agile really needs to stress more (that is key in the waterfall) is requirements. A (couple of) dedicated rounds of requirements where you talk with users (and stakeholders) saves a ton of dev time - agile does have requirements, but at least in my experience, devs seems to gloss over it.
Back in the day, our CEO's neighbor mentioned that our site didn't work well on Microsoft's WebTV. WebTV appeared on our list of supported platforms.
(Of course those are deeply in managerial self-pleasing and probably would make a hello world project cost a million dollars, still...)
Leads often don’t know how to lead, scrum hides this.
Management is supposed to be understanding the organizational needs of completing such projects i.e. making sure the right personnel are on the team, on the right teams, that the right teams exist, and that individuals are capable (technically, as a matter of attitude, incentives, conflicts of interest).
Management often doesn’t know how to manage and scrum hides this.
Product is supposed to be coming up with the projects. It gets more complicated as there are “technical” products where the PMs are maybe the leads of Engineering teams, and other ones, but overall, someone is supposed to be responsible for defining what to do next as a discrete goal.
Product often doesn’t know how to do this and scrum hides this.
The thing everyone gets wrong with agile / scrum is that you just keep hacking away until “something” gets done. Without getting into how “agile isn’t scrum” and really just meaning “the death march of ill-specification and low accountability that often gets described as scrum or agile”, you’re supposed to actually start and stop stuff. You can start, fail, and restart. But you need to complete stuff, then look at what was planned versus what was delivered. You need to make specific people responsible for specific things getting done. If people can’t handle their assignments, they need to be given different assignments or removed.
This all sounds really simple, but at dysfunctional orgs, it gets way off track. Being somewhere where nobody is responsible for anything is practically a right of passage in this industry.
People tend to gravitate towards Scrum because they know it, but it's usually implemented because a process is deemed to be required, not that specific goals are trying to be met. By starting with the goals, I can at least start from a position of agreement, and develop a Scrum or other process that can provide some measure of success for stakeholders while meeting the needs of the engineers. It's also a forcing function for me to say that if you want X from the Engineering team, we need Y from you.
I'm also very upfront that a hill I'll die on is that velocities and story points are great planning tools and signals around process health, but they're never goals nor are they tools for judgement and reviews.
Otherwise, I dunno, my best managers had their own way of waxing philosophical about leadership and my role, the product, company, etc to make me more autonomous.
https://www.infoq.com/presentations/netflix-five-level-owner...
> My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls “conceptual integrity.” This doesn’t arise from any particular methodology, and can happen in the absence of anything resembling a process.
Some processes try to break conceptual integrity. Micro services are one but Scrum as practiced usually tries to stick people into a blind valley where the horizon is months out rather than years. That leads to bad minmaxing behaviors, and/or covert channels being used to keep the wheels on. Scrum theater is especially grating when people who make the sausage have to listen to zealous managers brag about how the process is the source of all of their success. Since they don’t know where the success comes from they don’t see it leave the building to be appreciated somewhere else.
The second time it was my circus, my clowns, someone had divided the backend and front end people into different groups and it was a shitshow. Until I/we discovered that diagrams of user interactions were less than useless for preventing impedance mismatches between the tiers. What you needed were data flow analyses. Particularly for complex data entry workflows. C comes from B, B comes from A. Rework dropped like a rock once I incorporated this into kickoff meetings.
Worst is, this has nothing to do with short feedback loops and frequent releases. It all has to do with perception and how the method is used.
It's a mixed bag. One thing refactoring is good for is getting past friction introduced by people who don't like a good idea. Early in my career I got a lot of "that will take too long" for features in the High Cost, High Value quadrant of the feature graph. A couple months of picking away at the accidental complexity and you can come back with a <6 week estimate instead of 4 months, and your proposal gets more consideration. I'm not saying 4 months is a reasonable estimate, I get why that causes concern, but refactoring can be so much more effective when you have long-term goals in mind. When you have a list of 'target of opportunity' features. When you can romance an epic into the code, instead of forcing it in.
I still get that pushback of course, but I'm more used to it and also sometimes don't ask questions I don't want the answer to. It's all subterfuge and puppet mastery. If I wanted to be an accountant or a lobbyist I wouldn't have enrolled in a CS degree at a semi-respectable institution.
The settings :
Giant industrial corporation, giant projects with other giant corps as clients.
Process process process. SAFe Scrum.
Somehow I ended up in a team that started to care. Like, we, small individual team, had a agenda and a goal. We saw path to success for this mess of a giant project.
And we started to shadow work our way thought it.
At first myself and another senior guy. As a hobbit.
But it became prevalent for the team as a whole to do “prep work” on top of our tickets.
We self organize, self train.
Why? So we can move faster and actually deliver.
Our normal ticket we’re adjacent work. It’s not like we started 30 repo on the side.
Finally our scrum master caught on that. She went onboard.
We repurposed tickets like nobody’s business. Acceptance criteria was a mere indication, because the QA folks were on board as well.
Not to less us merge crap, QA is QA and you don’t go to prod with a buggy industrial system. That shit is dangerous:
but they knew where we were going and were fine with changing the AC if they see progress, and if the black market scrum master say we’re gonna have a black market ticket to fix it.
We delivered. It fucking worked. My favorite part ? We were the “bad” team. The team without a cool project or a decent backlog. The cleanup crew.
How about if everybody can fit (the outline of) the entire system in their head? XP says that everyone needs to understand the cocktail-napkin-level view of the system, and everyone needs to be able to reproduce it when asked. That helps conceptual integrity - everyone knows the vision of the system.
Personally I don't like answering the same questions over and over. I got into programming so I could 'solve' repetitive work, not so I could get paid more to to it faster. And I've seen to many cases of unapproachable=out of touch so that's not an option for me. The only way to scale being the person to ask hard questions is if you make sure people can self-serve on easy and intermediate ones.
And yet it is I would say very rare to see a good team scale their vision up. AOL was able to onboard new programmers and share the vision, but we never got a different geographic site converted, not really.
One tried things with documentation and two person Webex meetings, and code reviews that focus not on being tidy, but here’s how we understand this problem, these are the reasons to do something, this use case is an example of such and such a concept. (Just a final closing comment, be tidy you won’t be happy otherwise.). It can gradually build up over time, but rarely causes teams to catch fire.
Why don’t software development methodologies work? (2014) - https://news.ycombinator.com/item?id=15885171 - Dec 2017 (184 comments)
Why don’t software development methodologies work? (2014) - https://news.ycombinator.com/item?id=15476638 - Oct 2017 (1 comment)
Why don’t software development methodologies work? - https://news.ycombinator.com/item?id=7211341 - Feb 2014 (197 comments)
> But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results.
What matters is delivering value to the organization, and doing so in a cost-effective, sustainable way. Making plans and then meeting them is incidental to this, and quite often opposed to it.
Hey, I just invented eXtreme Budgeting. If you buy my book, attend my conference talks, and hire my firm the coach your organization through implementing it, you'll be among the A-tier companies that have decided to take budgeting seriously by practicing XB!
Who wants to go in on it?
"Double-entry bookkeeping is just pair programming for accounting" -me
No, what _should_ matter is delivering value to the organization. But that's impossible to measure, so what actually matters is what they can measure, which is "on time and within budget".
What you said you would do is based on old information!
If adjustments are needed every week, then the root problem lies deeper.
However, I think the failure of methodologies can be ascribed to treating SW development as a sort of reproducible, factory process. Two major misconceptions that come out of that:
1. Assuming that the progress on SW development is linear, and the sum of software is exactly the sum of its parts. In reality, SW development process is more like an S-curve, where you plan a lot in the beginning (so coding comes slower), then you speed up because most of the issues are clear and the research is done, and then you slow down back again as you're trying to fine-tune the whole result. This inherent non-linearity is really difficult for the "bean counters" to deal with.
2. Assuming that the people should operate as workers on assembly line, that is, the architects designs, programmer codes, QA tests, support does fixes.. In reality, it's more like a workshop operation - ideally people live with each feature for the whole lifecycle.
What always amazes me, though. Big pop art productions like computer games or movies.. they often run on a schedule, and yet are as creative or diverse as SW development. So it seems like there perhaps should be a methodology.
> Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results.
Asking a methodology to always meet requirements on time and within budget is asking for human perfection. You might as well expect it to deliver world peace and end hunger as we know it as well. So of course if you set a ludicrously unattainable goal, development methodologies all fail. In every project, unforeseen obstacles will always come up, which usually means some combination of scope-cutting or delaying or (occasionally) spending more.
But when teams are having major problems in delivering working code at all or at a reasonable pace, that's where different methodologies can absolutely improve the situation, depending on what's going wrong and what the desired product is. So obviously they do work in terms of adding value, when applied appropriately.
Let's not throw the baby out with the bathwater here.
I'm not quite sure what you're getting at here, because that's absolutely, positively 100% what the people who are recommending methodologies insist they're for and absolutely, positively 100% what the people who adopt them want them for. The ENTIRETY of software developer management is focused - beginning and ending - on meeting requirements on time and within budget. That's the ONLY thing they care about and their ONLY purpose for existing.
Sure, that's what management would wish in an ideal world. But obviously that's not what these methodologies are even marketed as -- e.g. the entire purpose of agile/scrum is to break into sprints that have deliverables as goals but not as guarantees, in order to de-risk larger failures by exposing smaller failures earlier, to be able to then adjust requirements/timeline as you go. Heck, it's so important that they insist on estimating tasks in terms of abstract points instead of units of time.
Methodologies attempt to give more accurate, less wishful-thinking, estimates of timelines/resources. And they're often quite successful at that with things like planning poker etc. But all they do is improve, they don't give guarantees.
It has taken over by managers to just be used for pressuring software engineers, that's why the name had to change so much.
"Agile teams define success as delivering value, not conforming to a plan. In fact, truly Agile teams look for opportunities to increase value by changing their plans." (p.7)
So there's at least one person recommending methodologies who absolutely, positively, 100% does not agree with you.
The whole chapter is free online if you want to get out of your bubble: https://www.jamesshore.com/v2/books/aoad2/what_is_agile
They talk about a triangle of (scope, quality, cost) where you can only pick 2. As a business you can choose what trade off you want to make, but no one’s promising that you can have a rocket ship by next week with 2 engineers if you just organize your tickets correctly.
I tend to think of a working methodology as one that produces software that meets the customer's need in a way they are willing to pay for that is sufficient to fund the organization that builds and operates the product.
But even that definition can be poked apart because it only accounts for self-sustaining companies, and many software startups do not have an intent to be that.
Take scrum vs. for example.
The focus on small changes, repetitive action, reflective insights, and short feedback cycles is absolutely awesome for software maintenance. Use cases like the python 2 migration, Database upgrades, human language translations, or incremental architectural rework are perfectly suited for scrum.
You can develop features as well, as long as they are fairly isolated. If you want to add a device driver to the Linux kernel, scrum would probably work very well.
But scrum totally breaks down for features that are not independent of each other. Using scrum can easily lead to two features deployed to production that logically cannot coexist under certain circumstances. And no, testing doesn't prevent that. Another problem are features that are just very large by nature. They tend to break the repetitive cycle, because they simply don't repeat.
Finally, bugs are often problematic if they cannot be planned into a future sprint because they are urgent. Scrum simply doesn't assume that anything could happen in the middle of the sprint. It's an alien concept.
That doesn't make scrum a bad methodology per se, it's just important to consider the use cases and potential exceptions. And to chose another methodology once the exceptions become the norm.
Breaking down features is a skill that not many learn because, unless it's trivial, they believe it to be impossible to begin with.
Good god is it valuable though.
I'm with you on the rest of scrum. It's an awful methodology.
I experienced this once, when feature A that was about 3 years old prevented feature B that was a out 1 year old from functioning correctly. The error lead to accumulation of garbage in a DB and was spotted when said DB ran out of disk space. It was incredible difficult to convey the message to management that the bug was at a meta level because both features worked as specified. They really struggled with the concept. It simply didn't fit their view on reality that features aren't completely independent.
Meanwhile the engineer mindset says to give us good engineers, not numbers and good things will happen under any methodolgy.
The problem is that nobody really knows what practices are best, anyway. It's mostly just following the herd and gold-plating.
A story I typically trot out at this point is some code that was written in Fortran. I passed a meeting room one day, where some consulting dude was talking about "yeah, we could just put that in xml."
The thing about Fortran is that it can read arrays as a one-line statement. It's native to the language. You put data in XML, then you need to link in libraries and parse the whole shebang.
Programs need to be revised periodically. That means a bunch of guys who know what they're doing (not necessarily programmers) have to sit down and go through the tedious business of figuring out a new spec and how things need to change. I don't there's any shortcut to this. XML won't magic the problem away.
- people who care
- people who make a good faith effort every day
- people who consider the big picture and don't relegate themselves into a comfortable little box
When everyone agrees on what is being built, tools and how the product will be used, methodology is nothing but a framework to simplify the non-coding part of building software. All of the methodologies will work b. So if your methodology isn't working... it's a deeper problem.
Software engineering has accepted as its charter "How to program if you cannot"
Put steel ingot into furnace. Wait until hot. Remove to anvil and hit 50 times. Put back in furnace until hot. Remove to anvil and hit 50 more times. Repeat until steel forms a blade. Then quench.
This methodology is fine for producing a basic blade blank. But clearly it's not enough to simply give someone a method to make a blade. They have to actually do the job well, which they can only learn by doing. The method describes the process, but the method is not what produces the blade. If the blade turns out crap, the blacksmith is at fault, not the method.
My theory goes that since the general management / MBA mindset didn’t evolve to do higher level things with the time this would free up for management, so they kept turning the screws until management as a job was also a requirement for things to work and then it all starts to deviate from the core tenants until it’s just Agile In Name Only or what have you
Agile OGs will tell you Scrum is too-often a way of bureaucratizing Agile and making is safe (SAFe?) for managers who demand predictability. If you don't let a small-a agile approach emerge in a way that is appropriate to your project, in your organizational setting, you just end up with a different flavor of bureaucracy.
When planning and budgeting there is a lot to be said for traditional project management methods. They are more predictive than a spreadsheet full of task estimates. But if your team doesn't know why running a project off a Gantt chart fails so often, they won't know which parts of agility are most important to their project.
You just have to be good at knowing what the problem patterns look like is the only trick to this, and not let something get out of hand when there's a ready-made tool available to fix the issue.
What's particularly ironic about the methodology obsession is that if a perfect methodology came along that could reduce software development into cookie-cutter assembly-line work then, while programmers would be reduced to minimum wage janitors, the managers that insist that WE'RE the problem because we can't do complex things in the blink of an eye would be reduced to spreadsheets.
Many companies and teams share similar structures and characteristics, but every single one has differences (even at the same company). Just because something worked for some company and team (even ones which are similar to yours) does not mean it will work for you.
There have been so many times where a group I've been involved with has rushed headlong into a specific methodology or tool without having a decent grasp on the team dynamics and goals, and then spent precious hours configuring Jira or arguing about how many hours equals a scrum point. Lord help all of you who get that one person who fancies themselves an expert on a specific methodology or tool.
What you need are people who are experts at understanding the business and the dynamics of team they're throwing at it.
No one actually knows that FrAgile, SCUM, or TeeDeeDee are universally better than just having a bunch of programmers self-organize. There's no well controlled studies that show anything other than a weak association between hip methodologies and measures like productivity or profit, and no one has demonstrated actual cause and effect. That probably won't happen because these are purely human systems, and good luck controlling for that.
It's fine to consider methodologies, but there's no reason to believe that the people who invented or tout said methodologies should be blindly trusted. They discovered something that they believe worked for them, and that's it. They're not smarter than you. Only use aspects of methodologies that you would use or that you have seen work, and don't do anything that seems stupid or that you would dread.
Folks who have bought into the vision of the project and are motivated to "do it right, make it good" have a much higher chance of a successful product than a team who are attending the religious ceremonies and cargo-culting. These methodologies are like painted-lines and highway-signs; they can be great guidelines, but ultimately the driver has to want to avoid getting into an accident and aware enough to realize blindly following the lines/signs won't achieve that on its own.
Edit: *I use "nerd" here as a term of affection, not derogatorily. I proudly call myself a nerd. :)
You make it too complicated, no one can modify it without tons of domain knowledge, you make it too abstract and flexible and the abstraction leaks and the flexibility means a ton of extra time spent.
You can really only lock devs in a room if you have very very clear product expectations and a longer term vision that allows people to add flexibility where it's needed for future projects. This simply won't happen with most companies because that's HARD.
Geek: a person who is knowledgeable about and obsessively interested in a particular subject, especially one that is technical or of specialist or niche interest. "a computer geek"
I'm a proud geek, but after bullying as a kid calling me a nerd is fighting words. A geek nowadays socially ranks above the people that tried to stand above others by calling them nerds, turning labels into strength.
But I will say that agile (real agile, not agile-in-name-only) demands quite a bit of discipline and maturity from your people. If you don't have the people for it, it won't work.
1) your team's code must have a clear, sane, documented interface/API
2) all teams should have a homogenous intake process for requests for change
Everything else becomes a matter of either priority or performance. These areas are where changes have real impact, so it's better to focus the conversation there.
> agile demands quite a bit of discipline
I'd say small-a-agile requires more discipline from managers rather than staff/line employees. Namely, discipline to refrain from imposing significant process overhead on their teams so they have a better feeling of control and predictability. Manager-driven, observability-oriented process changes seldom give managers more actual control. Instead, managers should embrace the uncertainty that comes with software development, demand only to see real progress on working code in short intervals (no milestones, delivery dates, etc.), and set ambitious goals for their teams only in these intervals.
The goal is to find a methodology that brings ease. So you can stop putting effort into the organization and meta work and instead put more effort into the work that actually brings value.
Huh? The entire Agile Manifesto is about how programmers should just self-organize as they see fit and highlights some key areas that developers need to be consider in the absence of a manager who would traditionally coordinate things, such as "talk to the business people".
* Applying the principles of the Agile Manifesto, or
* Top-down semi-ritualistic application of a canned methodology, usually some variant of either SAFe or Scrum (but rarely, in the latter case, much like the version in The Scrum Guide, but instead something that played telephone through many layers of consultants, distant managers, and the direct manager of the team, is poorly documented, and changes arbitrarily via management fiat based on outside influences without team input.)
The problem is right there, most people are not organized.
To do coding for sure, to scope things, evaluate, prioritize etc ... not so sure.
If truly no one is able to get a team organized I’d say treat it as an exception rather than a rule and figure out what is hold the team back from being more organized
Team organization isn't the same thing as managing a product. Self organization is about teams having autonomy to organize and plan units of work and drive the teams forward. This isn't some magical vacuum where the business has no input on what those things are, and you very likely should have some distinct product management that engineers aren't in charge of.
the only justification for a thing is if it works. If they regularly do, as you just acknowledged they do multiple times, they're of value.
> and no one has demonstrated actual cause and effect
If repeated success S correlates with repeated use of methodology M, that's a damn strong sign of causation, and if it isn't, I'd still use it.
Actually your whole post is so relentlessly negative and petty (SCUM? FrAgile?) I won't give it credence. I personally use for my own projects TDD and plenty of comments in my code and it works (or correlates if you prefer...) very well with good results.
Have a sense of humor. If TDD works for you, continue using it. I've done TDD before and sometimes still do.
Sometimes people are negative for a reason. I've seen enough iterations of these methodologies that I don't think they have any special value. Should a cattle ranch be more "agile?" Maybe, but honestly, I doubt that would end well. That's an extreme example, but even different software products may not necessitate the same workflows that these methodologies are intended to create.
The point is there isn't necessarily anything in particular about any of these loosely related methodologies that makes them special or effective for everything. In fact, they fail to live up to their promise quite often because they're treated as if they've been proven to be more effective than hypothetical alternatives, which simply isn't true.
> the only justification for a thing is if it works. If they regularly do, as you just acknowledged they do multiple times, they're of value.
Yes. However, I've personally never been with an organization that actually attempted to measure whether a methodology was working or not. My guess is, the vast majority of the time, methodologies are a magic talisman to ward off the bad C-level spirits and bring good fortune to middle management, and sometimes make it look like more work is being done than is actually.
I'm sure those organizations exist, but if they are actually willing to do such honest self assessment, they're probably capable of arriving at the same place without needing a pre-built methodology to start with.
the problem is that several different methodologies "worked multiple times" in practice, over the time, often in the same company, and every time the managers of the era sold it as "a great innovation" and abandoned it for the new trendy fad few years later or because the new manager had different ideas and brought a different "culture" (scrum masters that charged thousands euro/day COUGH COUGH).
See many Google engineers that considered agile nonsense (for Google), no more than 3-4 years ago.
So basically it doesn't matter which one you chose.
No matter what the methodology, if the rules for your organization's implementation of it are written by bureaucrats, the result won't be developer productivity.
Even if everyone buys into a process the process is only as good as the people in it.
A process doesn't guarantee results - it guarantees consistency. Sometimes that means you get consistently bad results. But the good thing about having an actual process is that it's easier to change it to see if it works if it isn't delivering.
The latter is what most process people forget - the process is a living thing, not a stone tablet of commandments.
But really, "Conceptual Integrity" is the key. When everyone is working towards a goal and cranking away it's magic, because things seem to happen on their own. They don't, though.
,,Individuals and interactions over processes and tools''
Manage everything with a prioritised list of things to get done. Always work on what is #1 priority. Don’t interrupt work that has already started.
I have used it for small personal projects and when managing large teams of developers. It’s simple. It works.
Could you please provide more details, or some relevant paper or book references on how this works? I'm currently looking for a lean, sufficiently efficient solution for my Oberon+ runtime (see https://github.com/rochus-keller/Oberon ); I evaluated different solutions, one e.g. based on MPS, but I would prefer something which is lean and as simple as possible; so hints are welcome.
When two services talk to each other, you must define a clear contract first. We can name it "specification".
But in real world, bad task management will break the law !
For some reason the number of bugs did go up the more testers we had ..
Software development isn't like most orgs, where the boss can do the job of their underling (even in most engineering disciplines this isn't true). This disrupts the normal MBA expectation of power dynamic and replaceability.
The power dynamic disruption is due to a communication barrier. It is probably mathematically impossible to encode all the issues and complexities in whatever IT system smorgasborg some IT person's day to day exists at to a box-and-arrows spreadsheet manager.
I almost suspect this is like many theory of computation proofs where complexity is either extremely high even in seemingly trivial circumstances, or exists in mathematical impossibility.
Anyway, that means the methodology exists not only at a push-and-pull fault line of power and control, it exists at a communication barrier that is fundamentally impossible to overcome.
The power dynamic disruption is such that: who manages the company, the managers, or the automated IT systems that do 90% of the management work, silently and in ways the human managers can't even fathom? Yet every modern company will only survive in the markets, even with cartel/monopoly advantages, with extensive IT systems automation. Even if they maintain position, the profit demands from shareholders will doom a management group if they can't implement IT.
But of course MBAs and management schools still don't get this. Accounting is needed by all MBAs, but IT is for "MIS people", it's for grunt work / low level MBAs.
Now add this power dynamic to heavy demand for labor, and a hatred of MBAs to pay labor good wages, possibly more than THEY make. They won't pay Developers per the real power dynamic. So:
- there's always turnover, since Devs make money by job hopping, the MBAs won't give massive amounts of money to a Dev except in rare places like Silicon Valley
- or they outsource, and there's a 12 hour delay and a language barrier on top of the normal communication barrier
So aside from the Mangement<->Labor comm barrier, there is either a System Documentation comm shortfall, so labor turnover / outsourcing destroys that, which eventually undermines productivity of a project long term, so the methodology, which is SUPPOSED to protect that long term, but has no provisions to do so, fails.
ANother dynamic is that the methodology doesn't really address disagreement. Methodologies can only address devleopment where people agree, but if there are technical, approach, economic/scope, requirements, or other differences in opinions between different groups that paralyze things, and the Management (because of the comm barrier) cannot decide or even realize they need to decide, things drag. The "methodology" fails, but software development methodologies don't have dispute resolution in them.
Methodologies are largely about managing, resolving, and/or moving forward in the face of disagreeement. Some may be unrealistic in how they do that (e.g., assuming consensus is always attainable).
> software development methodologies don't have dispute resolution in them.
I know of no software development methodology that does not address dispute resolution (usually, with different mechanisms for different kinds of disputes.)