- Agile is about flexibility, but it can* come with a much bigger price tag if you don't do most of your analysis/design up front for big/technical projects. *Sometimes not always. More on this below.
- It's great having conversations with product owners and business representatives at various times as you progress to other features, but you can't have multiple developers going back and forth for this (as you loose continuity) and if you choose your lead developer or solution architect (if you have one), then they'll spend more time on admin than on actually designing and building things. Therefore it's good to have a business analyst or dev manager help do most of this, with the option for your tech guys to get some face time with product owners if they want/need it.
- I found that none of the business analysts charged with leading/facilitating the Agile requirement gathering really knew how to write good epics / user stories where I worked, and this can make life difficult / waste lots of time during refinement sessions and during development.
- The biggest challenge I found with Agile previously is how to incorporate design tasks (more on this below).
- Some of the business analysts, dev managers, project managers, portfolio managers that I worked with thought that once all the epics and stories were defined we'd just be able to build everything with zero design. Er, no. The lack of a definite 'design phase' like in waterfall doesn't mean no design is required, it just means that your design execution is more flexible and incremental if need be. You still should go through the design process for your separate features (as they get prioritised) because a) Like TDD for units of code, it forces you to think about the gotchas/edge cases/touch points/cross cutting concerns BEFORE you implement something. b) You have the design to refer to in follow up discussions with product owners / BAs / dev managers and your development team which could consist of several developers.
- By design above I mean a design document/wiki page that covers conceptual, logical, and physical aspects of the solution that will be built. The design might also include and overview of discounted designs initially considered but then not chosen. You want your conceptual/high level design section to be just that, with all the really tech stuff in your logical design section e.g. most your UML stuff and other tech points. But back on Agile, by design I ALSO mean Spikes and Proof of Concepts that you sometimes need to do to validate design assumptions.
- Even with an 'Agile' approach being dictated from above, you might still decide / realise that you need a waterfall approach of sorts, at least initially for the analysis and design (or enough of the design to de-risk things). For example, in massive or extremely technical projects if you don't consider the full requirements or full design up front, then you can have these incredible costs of re-work. E.g. If features 1 and 2 need to be re-written because their implementation is not compatible with features 5 and 6 which were always needed, but were just further down the backlog and not considered in the initial design work.
- This leads me to what I see as the the delayed cost and increased cost of Agile, for large projects with set business requirements. By that I mean again, if you have a large/very technical project with clear defined goals (example in next point), then you probably want to do your full design up front or at least 80%-90% of it. Definitely all of your conceptual design and most of your logical/detailed design and some physical design. Why? Well you don't have to, you can be fully Agile and do just in time / incremental design as you build new features, but the business and your tech leaders need to be told that this can come as a BIG cost down the line. As mentioned in my previous point, you could deliver certain features first but then discover you need to substantially re-write them at a later point because the design didn't factor in those features 5 and 6, that turn out to be just as important. If your business and tech owners aren't aware of this risk when they are sold on the Agile approach then I guarantee they'll be fairly pissed with you when you say you need to re-implement features 1 and 2 and they need to swallow the cost/delay of doing so.
- By large projects with set business requirements, think an entire ecommerce platform that is currently live but total spaghetti code that the business knows needs to be replaced due to slow / costly changes, or other technical risks. E.g. You know you have a search feature, product page feature, category page feature, basket feature, checkout feature, authentication/membership feature, fulfilment feature that ALL need completion.
- So how do you do Agile and Solution Design for large/complicated projects? I like this approach:
a) Your BA defines the high level features in the user story language ('as a', 'I want', 'so that') if the features at a high level lend themselves to this format, but if not, some general high level description of the features.
b) Your BA then starts building the backlog by defining the actual User Stories for certain features, in whichever priority they are likely to be built/released in. They interact mostly with the product owner, but ideally bring them along to at least some of the refinement sessions with the dev team.
c) You raise a technical story for Your Solution Architect / Tech Lead to do First Draft of the Conceptual/High Level design for feature A or features A-E if you want to de-risk things a bit more. You can have another story for First Draft of Logical/Detailed design too.
d) Your SA / Tech Lead then starts on the design under the relevant stories, but also raises Spikes, Proof of Concepts, and other composite design stories within your technical backlog, to cover off questions/unknowns that need to be answered for enough of the solution to be done. This is where you can get your dev team involved in the design process, by ticking off some of those technical stories. For example, you might as part of your design need to consider Capacity and Performance but decide to divvy up the work, and get the devs to flesh out that part of the design. Or you might decide a Spike is needed to determine if X is needed or not. Or you might want to prove a facet of your solution with a bit of POC work.
e) As both b-d progress above, your team will be able to start estimating some of the backlog being produced by the BA. You'll reach a point where you have both enough of the business requirements gathered, documented as User Stories or Technical Stories, and estimated.
f) Then you can start building the initial / highest priority features, while the BA and SA continue to progress with b) and c) respectively. You can do this because they've had a head start and your team has enough backlog & design to get started on development, while the BA and SA then try and keep ahead of the curve with the remaining backlog and design work.
- Part of the problem is that even with Agile development, certain businesses will want an up front estimate for a large project for costing purposes and budget sign off. The key to avoiding issues is honesty. Rather than giving some finger in the air estimate, you need to explain that until the precise requirements are known, solution devised, and backlog estimated, you can't give a precise estimate. Try to ask them for at least some of the concrete requirements and for a week or two for your SA / Tech Lead to give an initial estimate. Let them know that early on at best you can probably give only a +/-30% or +/-40% estimate.
- Most of the points above focus on Agile with big projects, but if you have a well established product / platform, and are making smaller feature changes gradually over time, then Agile preparation and development is much simpler too.
- So again, finally, honesty, honesty, honesty, is a big part of success. Your BA's, PM's, Dev Managers, SA's, Tech Leads, Dev's, all need to learn to be comfortable saying: 'I don't know the answer/estimate to that right now, but if you can give me x information and y days then I can get you an answer, or bring you much closer to having an answer.' and 'Because we're taking this approach of developing and delivering features A and B sooner with an Agile approach, it means that we might need to re-work some of the implementation at a later time when we consider and design features C, D and E.' and (as can happen with waterfall).. 'Unfortunately we underestimated X, and need Y amount of time to complete it. We've learnt Z, and this will help us avoid this situation in future.'. Good luck!