When everything is important but nothing is getting done
sharedphysics.com
sharedphysics.com
* Technical staff in an organization feel they have no authority or respect within the organization. This can happen when companies become too sales-driven.
* Senior technical staff feel they have no ability to delegate, either because of the previous point or because they don't trust the technical ability of less senior staff.
* Those in an organization who do have authority to delegate lack sufficient understanding of priorities or lack technical knowledge.
These things highlight the classic balance between technical knowledge and managerial skills. If you don't have people that bring both to the table, you're in for trouble. Not sure if I'd go so far as to say this is what it's all about. However, reading over the author's advice, I thought to myself, "None of this could have happened at some places I've worked." You can come up with a good plan, but if no one will listen to you or take your advice, it's worthless.
The author probably was that person who brought what was needed to the table. As such, perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through. Also, having the right people from top to bottom. My feeling sometimes is that effective teams occur almost randomly. This might be because organizations rarely have the patience or the resources to assemble them deliberately.
Particularly: "perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through.". This is very true. It was a combination of having tenacity to grind through a lot of politics and repetitive conversations and doubts that everyone had, while also addressing extremely real technical and process issues. Previous leaders often brought one or the other to the team: they either knew what needed to be done technically but couldn't push a plan through, or were politically savvy enough to get changes implemented but they just ended up moving deckchairs around. When I stepped in, I inherited a team that did this twice previously and everyone had zero trust and learned helplessness. I wrote this up and tried to emphasize the slowness of the process and the fact that it wasn't a "one simple trick" type approach because that was the case.
Or: "If no one will listen to you or take your advice, it's worthless." This is true. We had a few really brilliant developers cursed with a cassandra complex: they correctly identified the problems but given their way of communicating, they actually fostered resistance to the changes. Communication of the plan and building consensus is as important as the plan itself, if not moreso. Every plan will need to be adjusted based on reality and context, but as long as you build consensus and goodwill, you'll be able to have that flexibility.
With regards to your point about technical staff, we were an enterprise sales organization where decision making was concentrated in the finance and sales departments. It was extremely sales driven. The reason we were able to push this through was precisely because the company had lost faith in product and engineering to accomplish anything after a series of poor launches and chronically missed deadlines. I actually think the problem is that if engineering leadership did have more trust, they would have pushed back on this. It didn't look good to "give up and try to just do one thing at a time" as we did. For most organizations, that's painfully radical. But in the words of Munger/Buffet: "Don't just do something, sit there!"
I know I'm not the only one who is backing away slowly from the concept of tech debt, and I'm not the only one trying to drag other people with me.
Tech debt has turned into hippie-dippie bullshit in the minds of many people. It was an attempt at enlightened self interest by appealing to the bean counters, but if you're talking to bean counters you've already lost.
The developers want easy things to be easy so they don't have to spend half their time slogging through code that does everything the wrong way, half their time explaining why everything takes so long (heading toward infinity), and half their time implementing new functionality. And yes that's 3/2 which is why we have people doing 60 hour work weeks.
A long time ago I saw a speaker claim that it's not tech debt, it's wear and tear. I don't know if I agree entirely with that characterization, but I definitely think that it's less wrong.
A lot of what people call tech debt is just working code that they don’t understand (maybe because all of the people that did understand it are now gone). Or it’s complex, which may just be because it’s dealing with a complex issue and accumulated lots of bug fixes over time. Or tech debt is just nitpicky design choices which may very well be not-great but aren’t actually causing any problems. None of that is worth investing in beyond a fix-as-you-go, in my opinion, due to there usually being problems to solve with much greater business impact.
On the other hand, you have tech debt like: a complex set of dependencies/services with poor interfaces that makes what should be simple changes very complex and risky. Something implemented so inefficiently that it’s slow enough to impact UX or wastes a lot of computing resources ($$$). Or code that is just so, so bad that making any change is like walking in a minefield - and while a lot of engineers are way too quick to put code in this bucket, I do think it’s possible (eg a complete ball of mud application with tons of global state, very poorly applied concurrency patterns, extreme lack of parameterization in favor of copy-paste).
That said, I have often seen “tech debt” used as a rationalization to work on some nitpick with no business impact and without a clear improvement. In my opinion, this is partially because working on such tech debt can be very easy: it doesn’t require understanding more complex business requirements, it might be very easy to test with existing integration tests, it has little risk to production, and it doesn’t have task/planning dependencies with other people.
Secondary point: there is "near-always" positive business value; near-always; and its actually impossible to predict if a given tech debt mitigation project will result in negative business value. You can only know in retrospect.
An example you give and label as bad: some crusty old system, outdated language, legacy, everyone who knows it is gone. But, it works.
Mitigation: rewrite it in something more modern that everyone on the team knows. Do that, and out of thin air your organization just minted a handful of new Experts on the system. They're forced to dive into the old code. Document the new stuff. Projects like this can be pretty fun!
Counter-point: The old system works. Why change it? We just need engineers to dive in, learn it, keep the old crusty libraries and frameworks up to date, lets hire and plan for that. And people fuckin wonder why engineering retention is so bad! "No one who works here knows how it works" Huh I wonder why they left! Let's pass all our problems onto the new generation; they don't get to own anything, their job is to fix shit and bolt on new functionality at a tenth the rate they feel should be possible, enjoy being an indefinite renter of the system until you get as fed up as the generation prior, move companies, and do it again.
Business value? You're providing cool projects to your employees. That's it. We'll make up reasons it benefits customers. "We'll be in a better position to scale and reduce API latencies" bs. "Feature velocity will be higher" maybe. Throw out some more tactile reasons like it'll ease hiring because when people hear PHP they run for the hills. Yeah that's more correct, but the biggest reason is expert retention. Have you got experts? If yes: don't rewrite. If no: maybe rewrite. Let engineers rewrite, they feel ownership and responsibility over the system, they probably stay longer. That's it.
"Complexity" is similar. Its complex because the business problem is complex. Sure! Engineers just need to dive in and learn it. When? Is the organization willing to literally block off time for the engineers to do this? Lol. Has anyone ever worked at an org that did this? It could work! But what they may block off time for is: rewriting it to make it less complex. The end result probably won't be less complex. But you'll have gained an SME! And the organization can feel good perceiving the problem area as "now less complex" (whereas in reality all you did was train up an SME via way of a refactor).
When does it go wrong? Obviously; if a legacy system works, or solves a complex problem, and the rewrite doesn't quite capture what it does. The old code is right there; that's the spec. If the new doesn't match the old spec, it probably comes down to someone not fully understanding the old code. Guess what: that was gonna cause a problem in the future. You just expedited and uncovered a potential problem; but in a context where it can usually be remediated significantly quicker because you're building off a new system with tons of experts, rather than an old system no one likes working in. That's business value!
And sure; there are much rarer cases where the entire project goes way over time, the new system sucks, gets caught in development hell, people leave mid-project... that sucks. It can happen. But I'll reiterate: its literally impossible to know with any reasonable certainty that this will happen before taking it on. This means its an extremely poor reason to not to do it.
To be clear: I am not asserting that addressing tech debt should always be a priority. I'm asserting that there is near-always positive business value, and that People are extremely bad at estimating what that positive business value will be because it doesn't have a dollar-sign attached to it like CS and Sales orgs have.
Well in that case the real debt is the lack of documentation or clarity in the code. If the code is not written in a way that someone other than the original author can easily understand it, that's a problem. It also indicates gaps in the process of transferring ownership. There should be a team of owners for each piece of code that brings new members up to speed when people leave. Knowledge mustn't leave the company when people leave.
For example: "If we refactor this code and migrate to a new server, we'll be able to deploy in 15 minutes instead of 6 hours, and we'll be able to ship features faster to customers with lower risk. This means that we'll get back ~90h of engineering time per week (6 hours x 5 people x 3 teams), and our defect rate will go down by 50% (We see 10% of customers churn due to preventable defects).. Therefore, there is $702,000 savings in engineering efficiency (more time available) and would decrease churn by 10%, resulting in [X] more value retained."
Once our engineering team started thinking about the business case for why tech debt should be prioritized, they started to make really compelling arguments for why something needed to be worked on, and we could evaluate it against explicit business/feature requests. This value rolled up into the form of decreasing costs or increasing revenue.
Again: resolving tech debt has value, and the engineering leadership needs to work on figuring out what that value is and how it stacks up against all of the other valuable work that the company wants to do to grow. Engineering leaders who don't understand this often struggle at the executive level and ultimately fail the teams they represent.
(I also suspect that not all refactoring work is equal, and that most refactoring work don't result in any positive engineering or business improvements.)
- 1: Keep asking "why is this valuable" until you get to the money questions. It's often a few layers deep. Not being able to get at that (missing an ability to quantify) was in itself something that engineering leadership needed to work on. After all -- how can you prioritize between two things if you can't figure out what value they create?
- 2: Most engineering teams know that they might be able to save some time by refactoring. We figured out the average hourly cost of engineering effort and used that as a multiplier for time saved by a new project's implementation. You can take it a step further and subtract the cost of working on a project from the time it ultimately saves you to remove "dumb" refactors that take longer to deliver than time they save.
- 3: If you can't get to a tech debt's value, that's a red flag as to if it actually creates value. This was true for features and business requests as well. We also said no to features that "felt good" but ultimately created zero value (some redesigns included). "Rewriting something in a new language" because a new sr manager with strong preferences joined the team was a big culprit of these sorts of projects.
- 4: We also often asked if a tech debt project needed to be done independently of other work, or if it could be included as part of a new feature that we were working on. An example was that one team pitched rewriting a microservice because it had no API. On the other side of the company, we had a project to get rid of that feature entirely and replace it with a different functionality. If tech debt projects were handled entirely within the domain of engineering, we would have spent weeks rewriting something only to throw it away a month later. Hence the need for bubbling tech debt up to the single stack rank, and thus treating tech debt projects the same as any other project we were working on (with appropriate visibility, buy-in, and evaluation of value/urgency).
Often this is one layer too far though. Business relationships aren't 100% about business, or we wouldn't put an adjective in front of 'relationship'. We'd just call it 'business'. If you know engineers are trying to put together a proposal to take on a project that might not be immediately obvious to the business, we already know all we need to know here. Developers don't go out of their way to put together proposals for frippery, and most of them don't put together proposals for clear business cases either. They put together proposals because something is pissing them off, one way or another.
If you say no, after they've stuck their necks out, then you've damaged a relationship with people who are not comfortable being clear or direct about these sorts of problems. People who you are paying to more clever than you. Which means when the consequences come, you won't see them coming, be it by apathy or malice.
We aren't allowed to say, "if you don't give us time to fix some of this stuff I'm going to quit, or worse," and so everyone tries to sugar coat the real reasons with things that are objectively true but specious motivations at best.
As far as I'm concerned, the original notion of tech debt was intended to be about borrowing on good will, asking people to do something they don't really want to do, but with the idea of an eventual end. That kind of sacrifice can often be quite healthy, even transformative. But what happened instead was advanced game theory. How long can we stay on the verge of social and/or technical bankruptcy without ever crossing over? And every new kind of 'check' they learn to float is a whole new world of delaying tactics.
Getting a divorce is very expensive, but you don't avoid divorce because it's expensive. If you're smart, you don't even mention that it's expensive, because if you weren't having an argument before, you are certainly having one now.
[1] https://www.thriftbooks.com/w/the-principles-of-product-deve...
We have lots of old code in production, some 25 years old. Much of it written during the startup crunch, with little thought to extensibility and so on.
But it works.
So we have a rule to only cleanup/refactor code when we have a specific case which requires us to change it. If it requires a bit more than a minor change, we'll do a cleanup, refactor or rewrite depending on the circumstance.
The exception is when a major hacking event unfolds in the media. Then, suddenly everyone is attentive.
I suspect it is the same with militaries, except the USA and a few other countries. It is easy for politicians to cut military budgets and spend the money on civilians, and some intellectuals will even praise them as prioritizing peace, until suddenly tanks adorned with white letters roll over Ukrainian borders. Then, countries panic because their underfinanced military force is inadequate to the threat, which just became very real. (I am looking at you, Germany.)
1. If you don't properly evaluate the work and be selective with what you work on, actual engineering time will be allocated to too many projects. People spin their wheels working on potentially insignificant things, which is a morale killer. So thinking about the business cases was a way of reducing work, not adding work.
2. I tried to emphasize that thinking about business cases is something product and engineering management/leadership need to own, not necessarily IC engineers. ICs can add ideas to the backlog and identify opportunity, and it is the job of PMs and EMs to evaluate those ideas and sequence them for being worked on, relative to other projects. The feedback loop that is important to ICs is the impact of their work in terms of user metrics and goals the project was supposed to accomplish. I don't want to get too heartfelt about it (because Capitalism, Yay!), but there's a difference between churning out features into a black hole and knowing how your deliverables affect the people for whom you build things, and letting that continue to inform your work.
Like financial debt, you pay interest on tech debt, so you do want to strategically pay it down when the stars align, maybe you've recently put out some new features, and you can buy the political capital with customers (and by extension, management).
Debt isn't a bad analogy but an appreciation for the nuance would be nice, I don't commonly see that.
Explaining the benefits of paying down particlar pieces of technical debt is important, but another important piece of the puzzle is talking in advance with stakeholders about when, where and how much technical debt you want to take on to launch a product. When the existence of technical debt isn't talked about until it needs to be repaid and thus comes as a surprise, the conversation becomes a lot harder. Additionally many of the decisions about how much technical debt to take on to launch something can be improved by bringing stakeholders directly into the conversation, at least on a general level.
When taking on tech debt you often get a lot of pillow talk. We have great reasons to do the wrong thing for the right reasons, but that is all forgotten after the crisis is over. Either immediately for those really bad at it, or when the next crisis hits.
I do agree though that when neither transparency nor awareness are good, there will be several crises in the middle that are triggered by the tech debt. I’ve seen that play out a couple of times, typically when I’ve joined a project in progress, and still building trust before I can start pointing out brutal facts.
Some developers are blind to different forms of tech debt. And as long as there is dissenting opinion the business will doubt any information that suggests they can’t have their cake and eat it too. I’ve known a handful of jaded engineers at least who, instead of being afraid or awed by titles, end up treating managers like willful children. It… does not yield consistently good results, but then again I don’t know anyone or any process that does. I’ve seen worse, and I’ve seen better.
Huh, what are you even talking about? What victim am I blaming for what?
I am sharing an important part of the technical debt management process that I don't see mentioned as frequently. Planning out where and when your organization should take on technical debt seems as important to me as planning out where and when your organization should pay down that debt.
Treating people like children is rarely a good approach and to me indicates someone who is either lazy or unwilling to take the effort to communicate well (edit: including when working with actual children).
If an organization can't resolve dissent effectively, that seems like a deeper underlying organizational disfunction that will cause all kinds of problems.
The truth is, the line for what is technical debt is kinda fuzzy and a there is a lot of other stuff that isn't really technical debt, but rather process or organizational debt.
Mostly what I've seen of "tech debt" has not been this kind of deliberate decision making of tradeoffs. It's more like:
1. Non-tech founders want to start a tech company.
2. Seeing the high salaries of developers they opt to hire their college roommate or relative.
3. Friend/relative is horribly inexperienced and builds a pile of crap.
4. Said pile of crap nevertheless gets market traction.
5. Customers complain of endless bugs & sales want long list of features.
6. Friend/relative is promoted to CTO. Time to hire experienced devs!
7. Experienced devs say "this is a pile of crap, throw it away and start over".
8. Founders tell devs no, you have to fix bugs and add features. Hop to it.
9. Developers have no leeway or political capital to fix the underlying problems, do what they can, and eventually leave.
So it's more akin to an 18 year old with a gambling habit who has just discovered payday loans.
If you can’t get your coworkers on board, then they can make you look bad faster than the project will reach an intervention. If you can, you can stop having abstract conversations with managers about being Good and just start telling them that this is how long things take now.
Relentless refactoring is a set of skills that are interesting but difficult enough that people lose the plot. It’s really about amortizing all difficult work across all of the work, not just the difficult bits. This is actually a strategy from real-time systems, where maximum reaction time matters, and something that XP knew and Scrum does not. A thing I’ve noticed is that there’s often a lot of gaming by management once the engineers start to say “no” but haven’t mastered yet. The bargaining is expensive in time, energy, and good will (once people realize they’re trying to trick us into doing something we said was hard by going at it sideways).
When you add a day, two days, or week to all estimates, then the ratio between the smallest and largest stories decreases. The incentive to waste time trying to trick people into saying yes to something that violates the project triangle is drastically reduced. They won’t like it, but they weren’t liking anything else we tried to tell them so that’s not really a new situation. What’s new is that they see we’re serious and they don’t like it.
We have an endless parade of commentary on HN about what and how much should be in commits and pull requests and what have you, but the thing I've always found credulous about these ideas is how they survive contact with a code base where a "small" feature is going to have a very wide-ranging pattern of changes necessary to support it.
It doesn't take much before their are features you functionally can't implement, because there's no possible way to build a neat set of commits for them and get that through reviewers - particularly if there's a lot of activity, because the immediate effect is usually that you'll break a lot of other minor patches with the changes (or they keep breaking you).
I'm not sure though that I've seen anyone actually prioritize fixing technical debt in these terms though.
The fact that these guys cannot build a migration button smells like major technical debt. These guys aren't stupid since they have built an incredible tool. Somewhere along the line, their product became fragmented enough, where it would be too costly to build a feature that would lower the barrier to entry for their enterprise service. We're talking about high 5 figure to a low 6 figure yearly contract. They're telling us we need to go click for an hour to do the migration.
So if you don't think this is technical debt, I'm curious as to your interpretation of the situation.
Some software tasks are tedious and developers try to avoid doing them. But externalizing them to your customers is not a great strategy.
And at that point....
....I don't know that simply calling it something else is going to help.
Yes I'm a bit jaded, grab a drink, next one's on me
And re-writing from scratch isn't refactoring, but that seems to be the most frequent usage of term these days.
It sounds like you agree with the idea in general, but just don't like the current metaphor. Can you expand on what about this alternate metaphor fits the phenomenon better?
Maybe you made one module very generic but now it's cumbersome to add features to. Maybe you didn't add enough flexibility in another place which resulted in a buildup of suboptimal code. Maybe extreme concurrency was planned for but never needed and now the person has left and no one else understands how that part of the system works.
So now, over time, this emergent phenomenon that some call code rot causes your beautiful, elegant greenfield project to turn into a monster that no one wants to touch. It's hard to modify, easy to break, and takes forever to work on. It's legacy code.
Just swapping tooling shouldn't be considered tech debt if you can actually do it without breaking income.
There should be someone in the company, with visibility across the entire organization who says "yes" and "no" and explains why.
I've worked in places like this and the clarity is amazing and very motivating. The goals from the top down were 3 things. Everyone in the org could understand and apply it to their own work. Anything else was deprioritized. There was no confusion or swirl.
Lots of people piss on management as useless, but when done well, it can contribute a lot in terms of work-life balance, employees feeling motivated and organizational cohesion.
Done poorly it can actually make things worse.
Literally every group/dept/team has a list. Someone/process has to arrange that into a single list for the group that builds. If you have more than one group that builds stuff, then more output lists and more complex the transformation from input lists to output lists.
Guess this is a simplistic summary of what PMs and Eng managers do.
It's not actually easy, for a number of reasons. One is organizational. If your boss isn't doing the same then then it's going to be harder for you to do it. If you're responsible to multiple people who don't have any common chain of command then it's going to be really hard. And it involves saying "no" and standing your ground.
On the flipside, if only high-urgency projects get done, you end up becoming a 500+ employee IT service provider without a proper Identity and Access Management System (IAM), without a Privileged Access Management system (PAM), a haphazard, ad-hoc secrets store and so on.
That's all fine and good for a 10-person company, you can get by like that when you're 100, above 200 maybe it starts to get icky...
Yes, I understand that if really nothing progresses, you have to prioritize ruthlessly, but you cannot keep doing only high urgency projects forever.
Unless the management agrees in letter & spirit on prioritisation this is a disaster.
When everything is a priority the message from top is "We want best of both worlds" i.e. we want productivity, quality and less complaining.
Now you have prioritized in seq: A B C D... The team who has C or D to deliver will have there deliverables pushed by say 6 months. Guess who will be pissed-off with this?
But the team accountable for D will say that this is what was agreed and signed on but the top still had those flawed expectations back of their mind.
There is also cross-team impact. The product manager who was championing project D now has his/her appraisal in question. He/She will point fingers to the engineering and complain to his/her boss.
In other words: Every one in the chain of command has vested interest. Unless they are aligned to large extent its hard to get out of this situation.
That said, the reorg allowed for more effective parallelization by making teams more independent, and thereby avoiding the "I'm sitting here and unable to get anything done" problem.
But yes, if people are not in consensus, you'll have sabotage. The Lippit-Knoster model for managing complex change does a great job highlighting this.
I particularly liked breaking down “priority” into the intersection of urgency and impact.
If your organization has lots of politics you might be willing to "adjust" urgency of your projects by setting tighter than normal timelines or boost projected value such that your project gets chosen.
More simple low-mid-hi model doesn't give that detail so people understand that it's just a guess, not exact calculation.
Regarding politics, the author mentioned aligning incentives in another comment. It seems generally necessary regardless of the method used.
If you only add features, you gain rigidity and fragility. To go fast you must go well.
Teams that balance maintainability against features have a competitive advantage and will beat you every time.
Going "fast" immediately slows you down. Right then and there. You think you're going fast but you're just fooling yourself. The DevOps Handbook goes into this - you need to spend 20% of the teams time on clean code and maintainability.
I feel we need to embrace the fact software is a fluid monster and we must work at it dynamically: cleaning up when it s unbearably crappy, adding to it when clients and users depend on it, and a balance between both when clients can wait a bit or when the dirt hasn't settled yet.
Then you d have a team that delivers rather than state that we must lose the client because his need overlap our 20% "moving classes around more logical namespaces" clean code.
I mean, the fact that the 20% figure is only a rough guideline is so obvious it doesn't need to be stated.
Which sounds like something that happened zero times ever. The problem is rarely that there's too much tech debt work, it's that client requirements take over dev work right up until the product goes up in flames (at which point the client you meticulously built features for disappears along with the rest).
Since you're rejecting handbooks, how do you define "unbearably crappy?" Perhaps it's when the devs have already started looking for new jobs? And then, if it's unbearably crappy and you start cleaning up and a client comes with requirements, do you drop the cleanup work?
Here’s a question: why weren’t they devoting the bulk of their time to it? I’ve been in sessions like this as an engineer and they end up being incredibly productive and informative for everyone as to what the priorities actually are. These aren’t things that engineers are supposed to be doing better than product managers.
Why do product orgs _constantly_ fall on their faces when it comes to prioritization? My guess is that the job is highly politics centric and PMs have a hard time getting out of the way of their own aspirations or ego to let the progress take precedence.
And no this isn’t trite PM bashing. I would like organizations to re-evaluate the incentive structures of product owners and make them look more like tech stack performance metrics (literally, as in how much is our software getting out of the way of our customers usage and allowing them to be “productive”, whatever that term may mean for the company). Also, give product owners some credit for backing off on new features to make headway for tech debt pay down.
I think a common cause is simply fear. There is a lot at stake when you have 50+ employees. Maybe those one or two markets are not big enough to support what we need any more so instead of razor focus and discipline which helps, we try and build everything for everyone and a lot of it is simply wasted effort. We also start having teams so instead of a design decision being taken by the entire management (and can get shouted down), it just gets done because "the design team" regardless of the value it is actually adding or not.
In extremis, we see the FANG companies who have so many engineering staff, we can only hypothesize about what most of these people even do for 40 hours per week on massive salaries.
If you actually have some kind of customer in mind, and you have ideas of problems that you're solving for them, then this can be valuable feedback even if it's a lie. Digging into the feature might expose you to a new customer problem that weren't aware of. You can talk to other customers and prospects and understand if this is a problem they have as well. You can take that information and extrapolate it into a real set of use cases, think about the size of the market of "customers who have this problem," and decide if it's something that makes sense to tackle with your product. Then you can design features to solve the real problem.
If you just have a feature factory then all you'll do is build the feature as the prospect specified. You'll go back, they'll still say no, and now you have no idea how to market this feature or if it would even be useful for anyone else or it's just built around the customer's internal process.
This is what baffles/wonders me about the state of the art in building companies. It seems so anecdotal and ad-hoc. Case studies and good advice, while they exist, are from "previous generations" and "we do it differently" because "culture". It feels like society has to continuously reinvent solutions to managing people and projects rather than building upon proven and effective strategies. I guess, perhaps, I am betraying my own lack of knowledge of this domain. Maybe there are tried and true tactics and I've just never experienced them. Related, it really feels like there should be a sub-field of "sociology" set on improving the efficiency of groups of humans using objective methodology that yields consistent and reliable results...
Yes there is. Its called organisational behaviour. Most b-schools have some faculty specialising in it. Cornell,I think, has a whole department in the school of Industrial and Labor Relations. They’re pretty good.
None of them have good theory. It's probably not possible.
"High", "Medium", "Low" are not a priority, they are useless.
A priority is an ordering of tasks. Once you free yourself from trying to assign labels to the priority then you free stakeholders from worrying about putting things "very high" in case they tread on toes and you enable an honest conversation about where it belongs in the list of tasks or projects that need finishing.
After all, no-one ever marks things as "low" priority because they understand it'll never get done, and no-one ever marks things as "high" priority because they worry about the politics of doing so.
The result is you get "medium", "Just above medium", "Just above just above medium" and "JAJAM+" in your JIRA. (True story! And a shout out to anyone recognising JAJAM+).
I don’t think having 40 people work on one thing at once is good at all. It’s like solving scaling by just getting a faster machine. Eventually you may not be able to purchase a machine fast enough to do everything you need to do… And you’re going to have a ton of deadweight.
I hate “agile” as practiced but, in theory, it solves this. If you scope out a project well, you should have a general sense for what cross-team dependencies exist and be able to put it on the other team’s plate. You need a decent amount of “trust” but ideally the other team will actually appropriately prioritize it, ie if it takes only two days of work to unblock a project with big business impact, they won’t push back that their Epic Rewrite v3 is too important and maybe they’ll get to it in 6 months.
The biggest problem I’ve seen is with bucketing all projects in long planning cycles (3 months, 6 months, 12 months, etc.). Every cross team dependency then takes about one planning cycle to be accomplished, and unexpected additional work/dependencies become nightmares. Also, while “ownership” and specialization are good to an extent, fixing a particular engineer to a specific domain over long intervals almost always causes problems because X% of engineers either aren’t skilled enough to be able to handle owning a domain, leave the team, get delayed, etc. If everybody is fully booked for a long time, there isn’t slack to help them, and it suddenly becomes a Big Deal to assign someone off a domain (because the person struggles for months and messes up a big project, rather than identifying they are messing up after weeks and course-correcting by putting them on something they can be more productive working on), and every project with more than a few people gets delayed because there’s no way to fix one or two engineers on the critical path dropping the ball.
What both this article and agile touch on is the benefit of having a flexible planning structure and not over-committing or trying to plan too far in advance. IMO, that is the key to getting shit done. I think you can do that without taking it to the extreme of having only one project in-flight though.
oof. too real.
Sounds like you have no strategy at all.
I'd bet that almost every company with 40 engineers has 300+ projects. Every company I ever worked at tried to do waaaayyyyy too much at once. It might be the single biggest problem in business. Steve Jobs is lauded for his design skills, but I'd guess it was the focus thing which made the difference. Warren Buffett similar. Bill Gates too. They have all commented on it, and it rings true to anyone who has spent time in a few businesses.
But to be fair, even if they have a 10:1 ratio of non devs that's still what 300 projects across < 500 people?
That's way too much.
I get pissy at product when they try to give us more than one goal per sprint.
If someone tried to give us multiple projects I'd laugh at them and ask them why they're cancelling the active one.
He was listing every project the company was planning to do. If you go through your backlog you'll find over 100, without fail.
And, yes, it's way too much. Which was the entire point.
> This routinely caused the team’s focus to change to whatever was the problem of the week (or to solve for whomever was yelling loudest).
My mind immediately goes to a lack of leadership and strategy. Why do we have pirates going around from team to team yelling, inventing new feature requests?
A lot of what I can read here sounds like someone (goopthink) finally took ownership and seized authority from a leadership team that had rotated out twice and had mostly given up.
> we had cycled through two previous heads of product and one head of engineering
> In fact, I volunteered for the role because no one wanted to take on a project
> The problem is that you need a conflict resolution and effort coordination mechanism
> Figuring out the impact and likelihood was something that a product management org would normally devote the bulk of its time towards. Not having that information was one of the failures of process we were dealing with
> adjusting the stack rank continued to be a deeply political process even after it was created
> “you’re a team of 40 people… surely you can do more than one thing at a time?”
> There was no way to scale those conversations. These 1:1s also allowed us to get ahead of influential senior managers who could have been saboteurs if they didn’t buy in.
This whole thing screams that there is no acting leadership in the sense of building a team focused on one goal. Allowing many managers to go around and create new business lines, not holding them accountable to drive outcomes but rather just be busy with some new idea. The issue your engineers have is identical to the one going on outside of engineering: no unified mission, nobody willing to say "no" to "some new idea that could totally make us some money and we should just try it and see".
It is as if the executive allowed a rabble of random managers to do whatever they wanted like gladiators in a combat arena. It's like CV driven development: "hey let's use kafka and http2 for our internal CRUD app" but instead it is like "hey let's start a new division of the business that has an entirely different workflow but exists in the same industry vertical".
The leadership team is absent from every point of this story and things could only improve when they got out of the way or were engineered out with pre-messaging in 1:1s.
Good that OP could improve all of this (which in itself is a huge accomplishment, something like 60% of change management efforts fail) but honestly until that L-team is rotated out entirely the problem isn"t solved, those pirates will go back on their habits that created a dysfunctional org full of "yeah but I have a cool idea" leaders. These leaders tend to want all of the decisions and none of the details, making everyone live in their mess. OP dove head-first into the details and this is why they could fix things.