The Skill of Org Design
commoncog.com
commoncog.com
As an organizational "scientist" it's amazing to me that organizations are ubiquitous and yet we know so little about how to construct good ones. Software design is in a better state IMO but not by much.
Here's a simple question that should be answerable in any approach to org design. What's the optimal *span of control* for management at each level in the organizational hierarchy? If you can't answer this question, you can't "design" an organization.
I think there can be a useful middle ground, where "fuzzy" descriptions are used with models to explain strategies that are developed organically.
I empathise with the OP on the lack of modelling in this space. I think it shows a lack of maturity of the field since good, simple models are usually used to produce fundamental understanding in a field.
Once you've found the couple with the lowest overall levels, book them on a touring circuit so audiences can ask them how many silly dances they invented. Since the couple doesn't know whether that's a source of their happiness or not, it won't really get us any closer to an answer. But at least the couple's resulting stress from the tour and impending marital problems will teach the audience about the limits of their method of inquiry.
I worked in quant finance for many years so I’m very familiar with low signal to noise in complex systems. You can’t throw your hands up simply because you’ll never capture everything in your models.
This field is so far from my areas of expertise but I imagine there are lots of smart people investigating and putting structure around these questions.
But it is misleading to expect an employee to maintain relationships with 150 people—they also have a family and friends.
> However, enormous 95% confidence intervals (4–520 and 2–336, respectively) implied that specifying any one number is futile.
You can however, decide that the key drivers in your domain are essentially impossible to capture quantitatively and decide not to model the domain scientifically. This applies especially well to cases where 'tacit knowledge' is important. Because that knowledge is hard to formulate, let alone formalize, it is really hard to quantize.
There are people who are comfortable with the zero margin of wearing their last clean clothes while doing laundry. There are people who are unhappy when they get close to that. And there are people who want to be at the top border, where no more than one or two days' wear can be allowed to be unclean at a time.
The important thing for a marriage is that you are both happy with similar rates, or that you are both happy with your partner's rate even though it is not your own.
Of course you can design an organization without knowing the optimal span of control at each level, just as you can design a logo without math and scientific models. The answer anyway will just be 'it depends, and span of control varies not only between different businesses but also different roles and even different individuals'.
Lots of design is done via intuition and experience rather than concrete engineering anyway, and OD is clearly an area where it is more about understanding the goals of an organisation and building a people strategy around it rather than perfect mathematical optimality.
Org design suffers from the same problem as economics and psychology: you're designing based on a fundamental discrete unit (a person) that's incredibly variable.
Except unlike the other two, you're typically not dealing in large enough numbers that you can handwave away differences and substitute averages.
Furthmore, any hierarchical org (which is to say, all, either formally or informally) exacerbates the problem in that you have some (variable!) individuals with even greater ability to influence the sum.
Which isn't to say it's hopeless, but is to say (to your point) that any approach needs flexibility and intuition.
Or as the author puts it: "As a result, you cannot predict how the humans in your organisation will react to your changes — not with perfect accuracy, at least. So the nature of org design demands that you iterate — that you introduce some set of changes, watch how those changes ripple out in organisational behaviour, and then either roll-back the change, or tweak in response to those observations."
This seems so wrong to me. It is the equivalent of stating that unless you can specify the values of all the hyperparameters up front, you can't claim to 'design' a neural network architecture.
All you really need to iterate on (this aspect of) the design of an organization is a way to tell when the span of control is too large and when it is too small.
For instance your span of control question depends on how much individual bandwidth is needed between the levels, which in turn depends on the nature of the work and how it interacts with partner functions and whether it can be routinized or whether there is an aspect of creative problem solving.
It’s a pleasant fantasy to imagine we could get definitive answers using science but it presumes there is a universal maximum when in fact there are many local maxima depending on goals and the individual strengths and weaknesses you’re actually dealing with. And even then org structure is a pretty blunt instrument which is always a huge tradeoff. All orgs rely on extra-organizational effort to address critical problems, whether it be through formal working groups or just individual hustle and resourcefulness.
Note that setting up an organization to be so responsive and adaptive is itself a difficult organizational design problem.
My feeling is that this could be a good way to filter out highly dysfunctional organizations. However I don't think you can find successful ones that easily, let alone come up with an magical organizational structure that automatically leads to success.
That might be because there are a lot of tiny details that might shape a organization much more than the pure structure of departments and roles. Let's say organization A and organization B have the same structure and do the same thing in the same field, but organization A has a good HR department which manages to attract good people and have them work for decades at the company, while organization B has a bad HR department, hires incompetent, fraudulent and downright nasty people, who don't stay on the job for long – wouldn't this make such a huge difference that differences stemming from the pure structure of the organization would be drowned out?
Of course you could now think about how a organizational structure could prevent this from happening, and maybe with the right structure and people checking each others decisions the likelyhood of such a bad outcome could be mitigated – but never fully.
But this author is concerned more with 'productivity' rather than longevity or interpersonal relationships.
Time and motion studies (for example) were part of the scientific management revolution for industrial management [2]. There have been both qualitative and quantitative studies of all kinds of things - organizational forms, people networks (things like, Dunbar's number[3]), power distribution (ex. work of Pfeffer), etc. I could go on.
I do think we are reliving an era of interest in management by data and metrics, much like that of the industrial revolution and scientific management. Nothing wrong with using science and quantitative measures to optimize, but any human who has been subject to purely management by quantitative objective will likely tell you it often becomes.. rather, inhumane. This is often what led to automation, I feel - to remove the human element that was crushed by industrial efficiency.
I suspect this is why the qualitative balance is important (and no less scientific - science can be logic not just metrics right?).
My two cents...
[1] https://en.wikipedia.org/wiki/Organization_studies
I had a similar thought on another commoncog.com post. The author didn’t seem to research deeply before postulating opinions. Looks like this site allows members to post so I don’t know if it’s the same person
I would ask the OP a simple question: what organisations have they built, and where are they now?
Longer, but establishes the epistemology of the blog: https://commoncog.com/blog/practice-as-the-bar-for-truth/ and https://commoncog.com/blog/four-theories-of-truth/
Otherwise, I'm still not sure why you don't include both (well established) theories, in addition to your own practice in your post.
I have not included any such theories because I have not found them useful. The links are to say something simple: my entire epistemology is pragmatic. That is, true knowledge should lead to effective action. If I cannot apply it and get results, then it is useless to me, and irrelevant when writing up notes for other practitioners.
I have noticed that your claim is that 'here is some theory that is well established and rigorous and old'. I have also noticed that your claim is not 'here is some theory that I have found useful, and <insert notes from application>.' I pay attention to arguments of the latter form, because it usually indicates something that I can integrate into my practice. Because you have not included notes from actual application, I am not particularly interested in your argument.
(But if you can provide an applied account, I’m all ears!)
That is not to say that org theorists are useless, or that research is useless. I have found Herbert Simon's work on organisational decision making useful as a lens on practical rationality, for instance. I’ve also spent a lot of time digging into expertise research for applied ideas.
I think the bar I use is simple: when reading a theory, I ask myself if there are actionable handles. If so, I tend to pay attention. If not, or if I’ve applied it and it doesn’t work particularly well, I discount the research.
That said, I have noticed that good organisational builders — with a track record of actually building orgs — say different things from organisational theorists. And I think the reason for why is interesting: I suspect that org builders are interested in actual org outcomes, while org theorists are too far removed from actual application; they are interested in tenure.
(Edited to soften tone.)
One way to pick who to pay attention to is ‘believability’ — meaning that you listen to those with at least 3 successes, and a coherent explanation when probed.
Pointing out survivorship bias is a common rejoinder to this view. But when you’re trying to put things to practice (not get at some universal truth like a researcher would) you often cannot wait for perfect samples. So you pick certain practices from believable people and test them against reality, and then hold the lessons loosely, making sure to update based on further experiments (which is necessary because life is messy and full of confounded variables).
Over time, it becomes clearer what is useful and works for you, and what isn’t and is perhaps a quirk of the other organisation’s context. But I’m not saying anything new; this is how we learn from life.
Related: Brian Lui’s loose feedback loops https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loo...
And the problems of learning from experience: https://commoncog.com/blog/the-hard-thing-about-learning-fro...
That said, I was brought up to find what is useful and good regardless of whether it comes from academia or practice. It's a beautiful thing to be able to find a balance of building an operational world view that borrows from both theory and practice.
It is my personal observation that much theory is based on observation of practice + theoretical projection, followed by more observation - i.e. the scientific method.
I loved Adam Savage's comment that sometimes science boils down to doing stuff and writing it down. I think as long as we find a good balance in learning from the past and building on it, while not letting the past limit our future endeavors for no reason, we can have the best of both worlds.
If it's math you're after, look at Control Theory or Nonlinear Dynamics. They work well for engineering purposes, but good luck modelling individual human behaviour accurately, let alone mathematically.
For example, in modeling an industrial system like a chemical refinery / synthesis unit for optimal throughput, one could also model the human organizational structure needed to safely and efficiently operate that system. Say there were 10 major steps/processes being overseen; failure of any one could be catastrophic. So, perhaps each unit gets its own manager with veto power over the whole process if their unit is down (a flat structure at this level), and each manager oversees a hierarchically-structured team (a tree at this level, perhaps experience-based).
Other organizations would need a completely different structure, but it should be structured around the fundamental goal. Thus, the concept of 'universal organization designer' might be so broad as to be not very useful, i.e. specialization in design domains is probably important.
I recall this coming up in a discussion of why the optimal organizational structure for Tesla is very different from that for SpaceX for example, so just moving 'the best managers' from one to the other wouldn't work out.
Indeed, this is a very good observation from which many ideas occur to me:
Which one do I prefer?
Is one obviously better? (I don't think this is a good question: It's like asking which is better, an API reference (a math textbook with a long list of theorems and definitions) or an API tutorial (a math textbook which holds your hand and explains "intuitively"). It depends on what you need).
Isn't it the case that initial explanations (explorations into a new topic) are like this at first, and over time (usually through work spanning multiple generations) the theories become more mathematical?
All in all I wonder about the difference between these two contrasting approaches towards understanding. And I wonder about it in such abstract (philosophical?) terms that the specific "organizational design" is just an instance of what I'm curious about; which is the different ways to explain the same things and other ways to approach "understanding" in general.
Looking at all the crap software being built today, I am not sure I agree even with the caveat of "not by much".
"Math" in the social sciences is often a smell for "physics envy". Economics if full of mathy looking things to give the appearance of rigor without actually having any.
"When running the Vietnam office, we had many other business-related problems to deal with; building consensus wasn’t something that I always had the time to do. So the way I ran certain org changes was to:
1. Get a sense for team receptivity for that org change, balanced against the necessity of the org change. If I sensed that the team would be resistant to the change, I would:
2. Figure out how much I had left in the ‘credibility/trust’ bank, and if I wanted to burn that capital.
3. If possible, find a smaller, more reversible version of the org change to introduce first.
4. Use disasters to my full advantage (people are usually more receptive to trying new ways of doing things in the wake of something painful).
5. Strategically allow certain things to blow up so that I could exploit the pain to introduce org change, as per 4) above.
6. Or build consensus; consensus was always the best, if most time consuming, option."
4 things worked there 1) it was a group to help people get jobs, which is an ongoing market need 2) it demonstrated success quickly and provided a template for that success, so people were motivated to invest in keeping it going 3) we made early cultural decisions that selected the right kind of people 4) we set out clear 5 year goals, and had every president update the 5 year plan and their own 1 year plan.
https://riverin.substack.com/p/the-canonical-startup-org-str...
There are a couple links to other articles and book on the subject in there too.
I'm trying to build a non profit org in a country with almost no culture of non profits and with zero experience in org design. Most of articles or podcasts on running non profits seems to be all the same – define vision/mission/strategy, plan budget, motivate people, do effective communication etc.
But this article is the first I found that actually provides some framework of thinking about the org design to me. Very refreshing. What should I read/watch/listen next (except links mentioned in the article)? Maybe even something specific to creating/growing non profits?
With Non-profits, your customers are your donors. Who is funding you (sales)? What do they want (features & results)? How do you show that you are spending their money wisely (metrics & governance)? The organization is nothing more than people put in place to solidify and execute those needs.
I argue that this describes a specific type of organization in a specific environment (context). namely a business organization in a capitalist market.
There exist other types of organizations. (However I may be blurring the line between organization and institution)
If possible do not split responsibilty between teams. Give responsibilities (e.g. security) as a hole without splitting.
Think about what discussion you want to have in the leadership meetings, then decide who needs to sit at the table.
>> “This concept for catalysis is as simple as it is ingenious, and the fact is that many people have wondered why we didn’t think of it earlier,” says Johan Åqvist, who is chair of the Nobel Committee for Chemistry.