Most of these look like MBA problems.
That said, maybe more experienced coders should be doing MBAs (which isn't just about attending classes but about networking, acquiring a wider shallower knowledge base etc) so the dynamics of tech debt are clear to people who are in charge of making debt decisions.
Many times I've seen long time estimates for something, because C level was requesting "X feature". But when you sit down down with C level, and understand what they want to achieve with X feature, many times it is possible to sit down with tech and figure a way to get what was wanted without the feature, generally by adapting some other request or feature.
The problem is that most managers are looking to suck up to C level guys and 'deliver'. And many C level guys won't stand the insubordination of someone telling them 'maybe that thing you guys thought of isn't the best way to achieve what you want', no matter how much you work on the techniques from How to Win friends and influence people. Some guys are just dictators, and some are suck ups, and when the two meet, as they often do, it means tech will work 10x what is needed to get where the company needs to be, because the focus will be on what someone requested, not what we are looking to get at.
Also, many developers get stuck in the mindset of places like that, where they assume they have no voice and everything they say will be used against them. So you ask them as a manager how they would achieve X and they just say "tell me what you want in the spec sheet"... people get weirdly conditioned when in negative re-enforcement environments.
In the best functioning development organizations I have experienced or seen, nobody should be asking for "X feature", they should be explaining what their actual needs are, what they need to get out of the software and why. And how important it is to them, or even what their "budget" for it is.
And a technical designer (who could be a developer who has some design skills perhaps by experience, or a designer who understands the tech) _designs a solution_.
This requires someone(s) on the "development" side who can do needs/requirements analysis leading design (ie, "UX"). And it requires the stakeholders to trust that the capacity is there, and that it will turn all right if they don't try to micromanage the feature -- not just "all right", it'll be SO MUCH BETTER.
This works so well when the organization staffs itself to make it possible and the people in charge allow it to happen, that it seems kind of insane that so few organizations do so.
They solicit needs from customers, support conversations, etc. and translate those into future development. They combine those needs into major product directions and balance them with an internal compass for where the companyw wants the product to be (i.e., what jobs they want to solve for which users.)
They are technical people with an eye for how something should be built, working closely with designers (visual, UX, etc.)
Excellent companies figure out how to deploy these types of thinkers -- with appropriate mandate -- throughout the organization.
If they are supposed to be technical, then we hired wrongly.
I can imagine a non-technical person could end up in this role and succeed, but someone with a technical background and some business acumen will have a much higher probability of success.
I once worked with a company that had that attitude toward accounting/compliance - so much more work and all because the primary holders were not in agreement, so they kicked the corporate structure can down the road.
Same goes for CS, etc.
I guess the only solution is a holistic approach, but even then sometimes someone will have to bear the brunt of being the area where people will 'will handle it', at some point a CEO will decide that is what is needed and it will be needed, even in a perfect organization.
This is one reason why tech firms win. Their senior management are made of [former] engineers who are much more aligned with the workers around what implementations make sense.
The skill includes working with the stakeholders to develop actionable needs (not just "make our business more efficient"; the obvious place to go from there is, okay, what do we know about where the greatest bottlenecks/inefficiencies are now, if we don't know enough what can we do to learn?).
Also you don't just take needs and come back with a finished feature, you iterate with feedback from the stakeholders. The first iteration might just be a textual description of the feature, to make sure it makes sense to the manager/stakeholder.
It's a skill, and it involves building trust.
But translating needs to features that will succesfully meet those needs is not a skill most managers/stakeholders have either, is exactly why it doesn't work to just get detailed micro-managed implementation directions from managers/stakeholders. (Or customers!).
Other times, it is a problem with the existing developers. Choosing not to learn/follow industry best practices. Creating an over-engineered solution instead of a simple one. Skimping on automated testing and spending more time on manual testing instead. Making their colleagues wait an inordinate amount of time for a simple question, code-review or approval. There are many things developers can do that would slow down their team's productivity.
I don't think tribal finger-pointing is very productive. It really does vary case-by-case.
Why do we, as an industry, expect to be able to run a project to schedule when we often give it the most cursory of glances in the planning stage? I work for a very large integrator and we don’t plan at all. We jump in and start while we’re ‘planning’. It’s absurd.
He and I have long said that if the people who built [hospitals|bridges|aeroplanes] behaved the way we did, the world would look very different [1]. Now, thanks to my recent PMP-by-proxy, I’m starting to understand.
[0]: Project Management Professional. One of the main accreditations for project managers, I think PRINCE2 is the other one.
[1]: I’ve seen the project schedule for a railway line extension. “You really plan a concrete truck coming three weeks from now down to the half-hour?”, I asked my friend. She does.
Yuck. I think the agile approach might be better here. It's better to expose your plan to reality ASAP and adapt along the way.
Except, that's not how agile really works. How agile works is "bosses" have an hour planning meeting, and then act as though they have a veeeeery robust plan (like you talk about) and get upset if things do go according to their quick one-hour plan. So much for adapting along the way.
And anyone wonders why we're in this situation? We dug this hole, kids. (And to be clear, I'm guilty. I'm not trying to be better than anyone. I am not! I'm just attempting to explain it.)
It was a way to bring some sense of order to what was chaos: new requests coming in all the time, work being dropped partway through to deal with the new requests, little progress on any front.
I think agile works really well in this kind of context by bringing in a sense of discipline and keeping new work at bay until the beginning of the next cycle[1]. It can also help to keep teams delivery focussed in other contexts but, key point, does not obviate the need for additional planning outside the sprint/iteration framework.
The problem really occurs when people treat agile as sufficient on its own, or as the one project management tool to rule them all. Total nightmare, especially with multiple teams involved.
[1] How workable this is in practice is open to question: I can tell you it's not always a great approach in an environment where client projects typically last a few days and on time delivery has been known to depend on a critical bug fix for a legacy codebase. Especially problematic when requests come in either during the launch phase, because there's usually a fairly hard window, or during the reporting phase, because there's a delivery deadline looming.
With agile, building houses still somewhat works because people will do that planning in their heads. So, knowledge will end up _only_ in people’s heads, not even in everyone’s heads, and will deteriorate there, even if those people do not leave the project.
So, by the time it’s time to do a large-scale update to the house (add a room, update ventilation to modern standards), nobody knows where the ventilation ducts run, why one of them is so much larger, that there’s asbestos in the ceiling, etc.
And of course, some people try to use agile not for building houses, but for building apartment blocks.
You ain’t wrong about how most businesses implement “agile,” though. :)
Learning and applying "how to plan a project" has pretty universal career relevance. I suggest 30-40 hours of study time, and it's around ~$600 to buy the book, take the exam, etc.
Reasons for the delay:
1) Proxy is a stand-alone application. Needs it's own deployment and build configuration 2) Main app was running on Node 6.x. High time to upgrade to 10.x as 6.x LTS is running out. 3) Upgrade to 10.x breaks some modules we depend on (Google Cloud datastore) and that module has been deprecated in Node 8.x+.. time to refactor that.... 4) Main app is in a mono-repo with backend systems still running node 6.x Don't want to spend the time upgrading everything to node 10, so need to split Main app into it's own repo, decoupling from existing mono-repo codebase 5) Since we are upgrading main app to Node 10, Best to remove dependencies on old definitely-typed typescript typings (upgrade to npm @types definitions). And as such refactor to use latest version of all modules.... 6) Security: New Default Proxy should only allow the Main app. code a solution for this using Keypairs.... 7) Docs, sample code for users, announcement mail.
thankfully i'm currently on step 7. Usually I expect to be off on my estimates by a factor of 4x. This is excessive :)
It does make me wonder though if there is a trend of 'hiding' necessary maintenance work in larger tasks - if so that seems to me to be indicative of a larger problem around not being given/making the time to do those jobs as their own tasks.
Or to look at it another way, if no stories are maintenance stories, then all stories are maintenance stories.
But yeah, if it needs t be done, do it.
Take whatever time I think it should take. Then double that number and go up one time unit.
Exp: I think it should take me 3 hours this afternoon to do this -> 6 days.
I think it should take me 2 days to to this -> 4 weeks.
I think it should take me 1 week to do this -> 2 months.
I think this should take us one month to accomplish -> 2 years.
Would you rather deliver a product with finished surfaces or cut corners? Proper design and good overall test coverage, or zero budget for maintenance?
It also helps to understand that management will ultimately decide that we need to do the project in half the time, and with these extra features you didn't ever plan on building.
My budgeting strategy is to always assume that for whatever scope you have planned, 50% of the complexity that you will need to deliver in order to reach the finish line, remains unknown at any given stage of planning. Compensate upward if there still remain known unknowns.
My approach to the same project would be different if I know that I get to walk away when the project is over, versus if I know I'll be the one to maintain the project when this phase is over and it goes into production. I work for internal clients lately, and I can't honestly say it's my way or the highway, but if you ask for an estimate, I'm going to give you an estimate.
If you tell me the deadline then we're having a different conversation entirely. (And that's OK! Deadlines are better than estimates in my book, especially if the consequence of missing the deadline is some unfinished garbage goes to production, or doesn't ship at all as we had to move on to the next thing. If you asked me for an estimate and told me "that's too much, do it in less" then you didn't really want an estimate, did you? Just tell me the deadline and I'll deal with it, in that case, let's not play games.)
Here's another approach: tell me how much time you want to spend on this project, then I'll put a pen to paper and tell you what part of the scope we can deliver given the resources you've allocated. Don't like that very much? OK, you write the plan and I'll say "yes sir." But that's not why you pay me, is it?
Estimates are a function of scope, time, and cost. If you tell me to do it in less time, I'll do it in less time, but it's going to come at the expense of one of either feature scope or quality. This is not a negotiating tactic, it's just a statement of fact. If you tell me I have to shave off a bunch of time without sacrificing anything, then we're not having an honest conversation and we're not going to have a good time. I'm sorry you had a bad experience, mate.
At least in my case, as a public facing SaaS I don't think running Node 6.x past it's LTS window is a valid choice, so better to bite the bullet now. If I went about doing the minimum work, it just adds more code/complexity that needs to be refactored when upgrading Node.
On one project there were CERT advisories out, and we had deferred upgrades due to breaking changes in those upgrades. All of a sudden we had to deal with the upgrade and a security issue at the same time. It was ugly. After the second time, we started putting at least one upgrade story per month on the work queue. Sometimes we let the engineer pick what they wanted to upgrade (just upgrade something!). Other times we picked the engineer for the work that needed to be done.
And that seems 100% true.
You do not have to do everything at one time. You can easily to one thing, and then right after that, literally right after, do another.
You could have gotten the proxy up and going, and then upgraded node.
It's not about "doing the minimum work" it's about understanding the problem space, what can be split up, and how to attack the surface area efficiently.
You're also introducing other problems that can occur by doing multiple things at the same time.
Generally speaking I advocate my developers to be really good at separating problem spaces and attacking accordingly. It's really hard to take on multiple issues at one time and actually have high confidence in not breaking many systems along the way.
That's nothing to do with adding a new feature, and should be planned accordingly. It's unfortunately very common for developers to hide maintenance tasks in other work, but clearly not ideal.
I update Java dependencies maybe once a year; it's usually painless and incompatibilities are highlighted by the type system. If you let your JS dependences get stale by a few months, you're looking at hours or days of work. And that doesn't count the massive shifts in build systems and frameworks that seem to happen annually.
Maybe in the last year they've settled down, maybe not. I learned my lesson and don't take dependencies on them anymore. Much easier.
Meanwhile, I see non-software people putting their complete trust in people with less than half my experience, and getting burned.
If estimations are based on FTE then you can factor in meetings, holidays, and even other projects that might suck up resources ("resources" in this case being your engineers work time) into your delivery date. You can detect very early on if your deadlines are slipping (assuming your team are logging their hours against tickers or a timesheet) and you can also make informed judgements ahead of time about whether you need more resources (were that's an option / the work can be scaled across more engineers).
Obviously this is harder if you're tackling a significantly larger "green field" project - you might have to start making some educated guesses in those instances. But most of the time you should have some idea about the work involved.
Now, you can try and do effort estimates using hypothetical people. But, at that point you have already given up the possibility of an accurate answer.
This is one of the many reasons why I disagree with giving projects to specific individuals. Not only do you end up silos of knowledge (which is a risk if that engineer should leave / get fired / die in a bus accident) but you also make it harder for yourself to make estimates for the reasons you've described. Sure there will always be a variance from person to person but you stand a greater chance of that averaging out if you discourage engineers "owning" code bases.
> Further, interruptions don’t just cost time, they also reduce productivity around them.
You missed my point regarding meetings. I'm saying if you estimate a project based on FTEs (different methodologies and frameworks will have different terms but they usually amount to the same concept) then time spent in meetings becomes a modifier you can easily adjust for, rather than a hidden time sink that you can't account for.
> Now, you can try and do effort estimates using hypothetical people. But, at that point you have already given up the possibility of an accurate answer.
You don't need to make the estimate on hypothetical people. You just need a system of tracking and reporting the hours people spend in a working day. For example in JIRA you can log time against a ticket and you can create generic tickets for meetings. Therefore after a week / sprint / arbitrary point in time you can view where your engineers have spent their time and if it's below the allocated time for that project you can then either:
- enforce a new policy (eg are all the engineers going to every meeting when just one or two representatives would suffice? Are some meetings just duplicates or redundant? etc),
- inform project managers that there will be a deadline slippage due to resource constraints
- hire more resource to compensate
- or all of the above
I agree estimates are never going to be 100% accurate but that doesn't mean there aren't better ways to estimate than just applying guesswork based on your current teams circumstance.
The entire dev team walks out, and yes this actually happens surprisingly often and could happen to you’re team tomorrow. Now, what happens to all your estimates?
Well clearly anything short term is now worthless. You can build a new team and use older effort estimates as a guide, but with different skill sets and a massive learning curve they in no way translate into FTE hours.
Effort estimates can survive those kind of transitions. But, time outside of a functional team is not meaningful.
The estimate is still the same because, at risk of repeating myself, you estimate on FTE and not wild guesses at a project end date. Thus you then go back to your project managers following the steps I outlined in my previous post.
Granted in the most extreme of situations you would need to factor in some upskilling time - maybe even get the team to re-groom the tickets (if you're following the agile methodology) but your method of making wild guesses wouldn't put you in any better a position should your edge case example happen. So you're not exactly winning any arguments by raising this point.
When running projects, estimations are based on the required work to do, not the team itself. Teams can fluctuate (as you keep pointing out) where as the work required should be closer to a constant. Thus any feature creeping that happens - as often does happen in software projects - gets captured and costed before so budget holders aren't surprised by hidden escalations in costs. Delivery date is then derived from totalling up the required work.
If you have a hard end date for delivery then it's up to management (eg yourself as a hiring manager, the companies board and the projects PM), to decide if you reduce the complexity of the project, do a staged release, hire more staff or even argue if the requested delivery date can be extended.
The above is how software projects should be run when they're managed properly. Other places might do things a little more adhoc but in my professional experience that almost always ends up being a worse way of managing a project, budgets and teams.
I understand you’re desire for an FTE to be a meaningful measure, but imperially that’s not true. As I said several times effort estimates can be used, but after deciding to add staff progress slows down for a while. So, using effort estimates you would reduce progress over the next 3 months before expecting a longer term increase in rate of accomplishment.
Brooks law applies to late projects, not new ones. If you have good estimates you can determine if you need resources early on.
All a lot does depend on the size of the project. If it's something that will take 12 months or longer than extra hires definitely wouldn't have that affect. Depending on the hire and the work required, you can shorten than time frame significantly too.
However I do agree that hiring isn't a silver bullet. This is why I suggested "hiring" as one of many outcomes that can be considered rather than the preferred outcome in all situations.
> I understand you’re desire for an FTE to be a meaningful measure, but imperially that’s not true. As I said several times effort estimates can be used, but after deciding to add staff progress slows down for a while. So, using effort estimates you would reduce progress over the next 3 months before expecting a longer term increase in rate of accomplishment.
This is where another piece in the jigsaw comes into play - an employee might not be 1 FTE. They might be part time, might have leave booked, might have commitments on other teams (effectively part time from your perspective) or might be a new hire so need upskilling time. Those are just a few common examples - I'm sure you can think of others.
This is why I keep reiterating my point about calculating your figures based on work required from FTEs. By talking about new hires as you are, you're again thinking about "teams" rather than "work required". When you look at work required then you can use your team as a variable in the calculation and instantly estimations become easier.
I'm undoubtedly explaining the process poorly but I do strongly recommend you read some books or articles online about managing projects and teams using (for example) agile methodology - even if you've already worked in places that employ scrum (again, for example). You could potentially really improve how you estimate work which in turn will improve how you manage your team. From personal experience, I've been a manager for a number of years and have found my skills really improved as I've adopted those lessons too.
It’s mostly bullshit. This team has historical reasons for their bloated schedules, but at root, they’ve simply been coddled, and never forced to justify their behavior.
I’ve been on both sides of the table now. Developers like to gripe and moan about “unrealistic” schedules, but without aggressive pushback from management, a huge number of programmers will simply never ship.
Note: I'm a dev and I can tell you I very much dislike working with prima donnas.
Yes, but not the way I think he’s thinking (and maybe not the way you’re thinking either). If you were actually in a room with him and said, “ok, let’s just walk in there right now, fire these clowns and replace them with somebody better” he’d immediately hold up his hand and say, “but I can’t find anybody better…”. So yes, it’s probably a hiring problem but the problem is that he’s putting unreasonable expectations on the people he’s hiring.
If it weren't for schedules I'd have no defense at all for 'wasting time' on something that saves each of my teammates an hour of pain per week. (I've had a lot of shitty managers. Stuff like this shouldn't need a defense).
Well,this is basically lack of trust between you and your devs. Or lack of common understanding of priorities. I'm not sure if you're a project mgr or a client, but clearly some expectations are misaligned. Perhaps you don't see their barriers or they don't agree with your priorities
The existing contract doesn't work, so some mediation is due to close that distrust before any damage is done. After all it's the team that you currently consider yours.
It’s pretty amusing how many definitive responses I’m getting from devs who are diagnosing my problem without any knowledge. It’s also telling how different they are.
Like I said, there are reasons, and I’m aware of them. I’m also fully aware of the technical barriers.
The devs are padding their schedule, and think they’re being clever about it. They are not. Some of it does boil down to trust (i.e. bad history with previous managers), but a lot of it is just that they don’t want to do the thing because it’s less fun than other things, and nobody has ever held them to account in the past. A sufficiently lazy developer will find endless Legitimate Technical Justifications for not doing something he doesn’t want to do.
I’m not trying to generalize to all developers here, but I do know the people and problem I’m working with.
Also, if they give you non-padded estimate, that is the one they will occasionally miss, will typical manager of your company force them to work weekends and evenings or otherwise punish them? Or will typical be ok with it and accept it as risk factor?
I am asking because the one manager that complained about paddings was the one that took pride in manipulating people into weekends - including by claiming they promised it. I do tight estimates when I know I can be late and generous when I assume it will be law or don't trust management.
To answer your question: no. I don’t do any of that. Also, this situation has nothing to do with aggressive schedules.
That particular manager was not worst.
Ate they having bloted estimates or not shipping within estimates? Why there are such large features anyway? Why is manager not a ten member but instead someone who don't belong?