This is the managerialist dream. To replace employees' judgement and competence with a process and management methodology. That way everyone is just a replaceable cog in their beautiful machine.
It never works.
It never works.
The thing that strikes me is that engaged and experienced employees deal with rafts of problems silently, using their networks to create and co-ordinate responses without executive intervention. In fact executive intervention is actively avoided because there isn't enough time in the exec's diaries to communicate and debate the issues properly so inevitably such intervention will be via over simplified solutions that often make things worse. The problem with this is that execs do not see what is happening. If they did they would be impressed and pleased with what is going on, but uneasy about the risks and their irrelevance. What this then leads to are management culls where value is lost because the problems that are being solved are not acknowledged in the company before the cull. After the cull this all emerges as a huge and nasty surprise that the company now lacks the capability to deal with.
The lesson : not everything (in fact not much) about a business is on the balance sheet, and in the corporate world the dead have no voices.
Approbation: approval, or praise.
"A company..."
"...is like an enormous clock."
"...is like an enormous cl-- yes, precisely!"
In reality they don't try to maximize employee productivity, they try to maximize employee fungibility.
If all the company is trying to do is survive then I'd say it definitely works.
(1) what does it mean to optimize within limits of average managerial perception? while some management knows the domain well enough they can identify issues and talent that add additional levels of productivity... many can't. Or as some say, A players hire A players, B players hire C players, etc.
(2) predictable + control-able levels of productivity may well be more valuable in meeting managerial goals than exceptional levels of productivity. this is a generalization of the fungibility optimization; fungibility is one aspect of predictability and control.
It's exactly as you say. Many aren't capable of identifying opportunities for optimization. Re: the organism metaphor - an organism can be relatively unhealthy and still survive. Some people are ultra-fit. Most are not. Some people really are quite unhealthy. The bar for survival is fairly low. Project that on to companies you can imagine the ultra fit being those companies whose managers can work together to identify opportunities for all the things they are aiming to optimize for and are able to optimize for more than just staying alive.
It's ironic in the adoption of sold-as-Agile processes, since it is precisely the idea that Agile (as in the Manifesto and associated Principles) was a firm reaction against.
- new ideas come to an enterprise
- they decide to apply it and see the "shortcomings"
- apply additional processes to where they feel it doesn't match
You end up with something not so far away from the original process, but slightly changed and more marketable. It can have noticeable good effects, but usually doesn't end up being any radical change or revolution.
It seems that management only responds to top-down change but not bottom-up.
Why don't doctors or lawyers or commercial pilots have people coming along with a methodologies for doing their jobs?
I have a few ideas, but I still feel like there's more to it than I quite understand.
They're usually different across depts and hospitals though.
Similar story with diagnostic medicine. Smaller laboratories shut down because they were redundant, and then lead times for blood work etc go up because of travel time. We used to have a provincial bus/courier system in place, but the govt shut that down because it was losing money, and the health care system suffered for it. Last I heard, rush samples between major centres are currently being shipped by 3-hour taxi cab rides.
This rarely happens in "Lean" implementations in reality. What happens is "change and see" which is not what the process is about.
So basically anything that spectacularly doesn't work is usually not 'actual' [insert method here]. Management failures are rarely obscure even in foresight.
Doctors, based on my limited understanding, are becoming more and more pragmatic about using checklists in surgeries. I am not a doctor, nor am I an expert, I have just read some interesting articles and heard from surgeons that they are trying to improve the process to reduce careless mistakes.
No clue about lawyers.
So processes can actually be incredibly useful, and oftentimes are designed after the fact. Agile has some neat ideas as well, but in practice I'd say 90% of the time it's pretty fucking terrible and the company would be better off without trying it entirely. Not because agile is bad, but because the people in charge of agile are usually bad.
At the end of the day it comes back to something I've learned painfully time and time again throughout my career. No matter how talented you are, you are powerless against a shitty manager.
That's the crux of it. Currently working at an org which is "Agile" and has Scrum masters, but management still insists on measuring work in man-hours and defining requirements in huge chunks then delivering them at arbitrary deadlines. So... it's waterfall with Scrum masters who make it look nice on Jira charts.
Yeah, but it didn't appear out of nowhere. NASA et al did a tonne of research [0, 1] that led up to the idea of using checklists. Likewise Cockpit/Crew Resource Management [2] and the psychology of getting people quickly out of aircraft that are on fire [3], all of which were prompted by one disaster or another.
Software engineering could use some of that rigour, but I'd anticipate a little resistance to publishing research which shows that the process of building some piece of software was a shitshow because the people running said show were bad at it.
0: https://ti.arc.nasa.gov/m/profile/adegani/Flight-Deck_Checkl...
1: https://ti.arc.nasa.gov/m/profile/adegani/Cockpit%20Checklis...
2: https://web.archive.org/web/20131026210937/http://yarchive.n...
3: Lots of which happened after British Airtours 28M, https://en.wikipedia.org/wiki/British_Airtours_Flight_28M and https://assets.publishing.service.gov.uk/media/5422efe840f0b...
There's plenty of software that really matters though, in the same way that using checklists and things makes sense in environments where you won't kill people. We could all learn something if someone researched using, for example, an Agile methodology for building the F-35's flight control software, for example!
Edit: Thinking about this some more, I suppose my point is how to measure this:
> a company that was really rigorous and disciplined and as a consequence a bit slower might well be out-competed by a company that's run in a bit more 'fast and loose' way
while correcting for other factors like marketing etc. You can measure how quickly a group of people will get off an aircraft and vary things like the dimensions of the exit row to test whether your change has improved things but you can't, e.g., re-run a project with TDD and test whether it was "better" (defining "better" in the first place is hard enough!)
Unfortunately, that's not the case. So most business just bulldoze their way to their primary objective.
Also not a doctor but I've built some checklist software for them. At good hospitals there's a nurse counting every screw, piece of gauze and scalpel passed around in surgery, counting them again on the way out, counting them when the go into the dishwasher and counting them again when they come out of the dishwasher and get preped for the next surgery. Apart from emergencies doctors will have a list of instruments needed for any given operation and there will be a special package of them ready.
This was heavily influence by the aviation industry.
Firm-wide common resources such as a docketing departments will have checklists but they are mostly software driven/automated these days. For example, creating a new matter in the docketing system will automatically generate most of the critical dates and checkoffs for that matter.
Checklists used post hoc and imposed on individuals are (IMHO) worse than useless - they give a sheen of process and quality by forcing someone to randomly ink boxes.
"You ticked box AX-two-alpha-seven. Do you know what that means? And you didn't complete the P12a checklist. That's gross misconduct. You're fired."
But I bet there are no methodology merchants who never fly planes or operate aviation businesses selling expensive checklist consulting, wasting the time of busy pilots with actual work to do. Like who is the Thoughtworks of piloting? Or of surgery? They don’t exist. Only software is plagued with them.
That seems to be the problem these management fads are trying, and failing, to solve. People hated waterfall because you have to do so much planning up front, and in software development, that process rarely works. Project requirements change, or planning assumptions are proven wrong. So you need a management framework that works without that level of planning. That’s the core of agile in my opinion, only plan a little bit of work at a time. Really I think it just leaves you with the same problems, only now you have a lot of small failures rather than fewer large failures. If you look at the agile shops that succeed, they tend to have good leadership that simply has the competency to plan ahead well. Those teams would likely succeed under any management framework, only with agile you now also have the benefits of continuous delivery.
Here is one data point for you from personal experience. A few months after got started at a US company operating in about 30 countries (primary business is manufacturing in the Casino and gaming industry) the word came from the CIO that I implement Six Sigma as part of the enterprise architecture activity that I headed. I countered that neither software engineers nor the IT architects and admins worked in the Six Sigma way and wondered at the meeting where this whole idea came from. I was told this was what the Chief Operating Officer and/or VP of Finance wanted to implement "across the company". Such things happen in this world we live in.
A better idea would be to reexamine why we "think" software should be so much easier than it really is. Perhaps it's because when we watch it being done, it looks like typing.
Software development is riddled with unscientific practices that are derived from some person’s blog somewhere.
100 years from now when they’ve found out the optimal way to do development we won’t have all these methodologies.
It's getting better, but it's still present.
For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
Are the tests in ./test/ or ./tests/? how do I run the tests? where do you put the headers? ./inc/? ./include? what are the options available at compile time? how are they exposed? what are the install targets? does it support the usual variables (looking at you DESTDIR)?
Granted more modern languages tend to be more normalized than the previous examples which are valid for C/C++ mainly. But these questions still applies. For example, the Go projects I've seen tend to vary wildly in term of compile tool chain, some have nothing, just run go build, other have a ./build.sh and the better ones have a basic Makefile which is generally not so great (no variable present to override the path to the go binary or set some compilation variables, no proper install target, very weird handling of the ./vendor/ directory in some cases).
It's not that immature. I'd measure the maturity of engineering, or any field really, as it's ability to create reproducible results. That result could be designing bridges that last a predicable amount of time, cars that run for a certain number kilometers before wearing out, or surgery that produces the outcomes predicted. This doesn't mean the surgery always works, or there isn't the odd car that fails in it's first 1000km: just that we know what the rates outcomes are and we can keep below a given level for a known cost. That level is never 0 because the cost for 0 rises to infinity.
For software it's a case of delivering something that does the job asked for and that operates at below a certain failure rate. This doesn't mean a zero failure rate or no bugs: just that we set a upper limit and can reliably meet it. It seems pretty clear there are many large teams of software engineers out there that can do precisely that. Google has even allowed their engineers to write & publish books about how they achieve that: https://landing.google.com/sre/workbook/toc/ It's not rocket science - just a method of doing things team of sufficient size can replicate.
> For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
I've got some news: that happens in every engineering discipline, even the mature ones. There are some things that don't effect the outcome overly. What you name you directories may be one of them. It may be it so unimportant they leave it to an engineers discretion, or it may be an organisation can settle on any name, provided they stick to it.
> But these questions still applies.
No, it doesn't. If you look at that site reliability engineering handbook, it's not about naming directories. It's mostly about measuring failure rates, and then applying well known techniques if it is getting too high. The well known techniques are mostly about change control, which happens to be a very boring subject most young software engineers almost universally hate. Trying to impose change control on a young software engineer is like trying to put a lead on a cat: it will do anything to avoid it including pulling in the reverse direction to where the lead is going, playing dead and refusing to acknowledge it's possible to move at all with the lead on, and attacking the person trying to put it on.
This is both incredibly rude and pretty much irrefutably true.
As a startup cofounder I've had the experience to hire like 3.5 programmers to do front-end work in the past year. I'm not at a specially "hot" location for developers, but my n=3.5 anecdata is that the few intelligent programmers you can find have terrible work ethics and continually try to spend more time with what they find interesting and avoid drudgery work. The hoi polloi (who might have made damn fine designers, copywriters, etc.) rely on stack overflow for whatever they can, and dither on the slightly more challenging tasks they didn't know existed.
Besides filtering on IQ, one thing a nerdy education teaches you is that challenging problems are often solvable if you will just [wo]man up and try.
For pilots, I literally work in a pilots methodology factory :)
But if you want to see crazy methodologies, imposed by the ton, by people that have no idea what they are talking about, go look at teachers.
An unbelievably high level of interdependence between worker teams caused by the complexity of the product.
This produces chaos, the methodologies promise to tame it.
Software development attracts these things because software development isn't as firmly entrenched in tradition as doctors or lawyers are. There are way more people that are open to implementing ways of accomplishing tasks that are more open and efficient than there ever could be in most other realms.
They do. You're not involved in healthcare so you don't read healthcare specific websites.
Here's some stuff aimed at the English NHS, but there is lots more.
https://q.health.org.uk/about/
This tends to be the NHS learning about something and implementing it themselves, rather than hiring a management consultant. That's because they want to protect public money and there has been a lot of bad experience of management consultants for this kind of work in NHS organisations.
Nothing. Management theories exist around managing every specific kind of work, as well as generic theories. Software development, HR, and other kinds of work that are done in (or contracted for by) almost every organization of non-trivial size probably get a bit more because more people are managing them. Many of the theories in software development are direct adaptations of process theories from other disciplines. Waterfall —before it was named—was just application of practices from non-software engineering, Lean is from manufacturing, kanban is a specific practice from the manufacturing organization whose processes inspired Lean, etc.
“Agile” started in software but it's everywhere.
> Why don't doctors or lawyers or commercial pilots have people coming along with a methodologies for doing their jobs?
They do. They are even often the same methodologies seen in software: e.g., in healthcare: https://www.healthcatalyst.com/insights/healthcare-project-m....
Note that the theories around software are often the theories of process improvement rather than the daily work in other domains (interestingly, this is not true or kanban), because software is just automated process so software development is essentially the exact same task as process development/improvement, where some portion of the process is executed on a computer.
Sometimes two or three in one year.
So the same thing applies to software management. And also - in different ways - to management in general.
Law is derived from the competitive sport model. The sport is verbal rather than physical, but it's really a form of mental combat with the physical violence abstracted away.
So in practice it's a combination of business negotiation, corporate politics, national and international diplomacy, book research, a very specific kind of creative writing, acting, public speaking, and psychology of persuasion, with different emphases in different areas.
You could have a process for basic case management, and I would be surprised if practices didn't. But the sharp and pointy end - the part where all the elements are used to create a negotiated contract or a court performance - is very difficult to formalise.
Doctors and lawyers are personally liable for their screw-ups. I don't know how liability works with pilots but they will lose their license if they screw up.
By operating within the medical standard of care, doctors diminish malpractice liability. So having well-defined systems that help prove that you follow the medical standard of care is important.
Lawyers don't have anything like that as a whole. Each lawyer handles their clients their own way. Even in the same firm different partners will have different practices. Some use checklists for their staff to confirm that the certain required forms have been filled out or provided.
In the actual work of lawyering, checklists are not a thing I have seen. Though there are "outline" documents to help with bigger projects.
Lawyers and pilots have a clear cut set of procedures. Pilots aren't inventing new technologies each time they fly, they're just driving the bus from City A to City B and worrying about occasional edge cases. Lawyers follow legal procedures and have their own built-in rules and processes; they're tested on these processes to pass the bar.
You see methodologies for things that vary wildly like sales, military leadership, and of course, engineering tasks. Each project, situation, or client could be extremely different, and it's not possible to build a prescriptive how-to for each scenario. So you create a methodology and framework so that managers and leaders can quantify uncertain situations and build (and alter) their own how-to's.
Got something boring and repeatable? Process / Automate the daylights out of it. You'll be better off.
Anything that requires innovative thinking or customer focus? Requires fitting square pegs in round holes? Probably isn't going to go so well...
Speaking as someone who spends more time trying to manage the former (boring + repeatable), independent humans are generally the key point of failure in any process.
After being a manager, I'd say that is not a wrong goal. If your team's success depends on individual heroes, then it is a fragile setup. People can be absent for a lot of reasons - changing jobs, parental leaves, disabilities, ... A good manager should put in place appropriate processes such that even if someone leaves the team, the show goes on smoothly for everyone else.
Of course a large percentage of the cogs of any machine fly off at the same time, you're in trouble. But I don't think one IC matters much in any large organisation.