Bus Number – The GitHub plugin my coworkers asked me not to write
scannedinavian.com
scannedinavian.com
It looks for knowledge islands and relates those to frequently modified code, to identify hotspot, or areas of high risk due to low knowledge distribution in areas of high change.
Another use is if someone hands in their notice you can easily see all the code that only they know, so your handover planning is mapped out easily.
I’ve never thought of it being used maliciously, it’s for visibility. It would be a shitty manager that would use it that way and if they’re already shitty then this tool won’t change that.
You are a member of the intelligence community of a country, let's call it Tussia, which has been locked out of the leading kernel for military hardware in the world. Let's call that Kinux.
You know that the guy down the office has started a project to fork that kernel for your countries own internal usage. You're an over achiever and want a promotion before he gets one. You call acquisitions for 8 female agents with special training for intimacy with nerds, you also make a back up call for 8 doses of polonium in case the agents aren't successful.
In case you think the above is fiction I know a CEO of a unicorn startup who got the first part of the treatment when he was looking for seed funding.
He's built a tool that generates hitlists for any competitor to use.
I've had three jobs where Pluralsight Flow was introduced. At two of them, the managers immediately started using the metrics for feedback, performance reviews, employment decisions. At the third, the developers saw this coming a mile away and refused to engage with or evaluate the tool.
Unfortunately, the absurd pricing of these tools means that people who approve them have to get some sort of ROI. Since they don't have a good way to measure productivity/output/knowledge silos, they instead turn to "Well Jose had less PRs this week..."
But this always lurks in the near shadows:
>>I’ve never thought of it being used maliciously, it’s for visibility. It would be a shitty manager that would use it that way
Therein lies the problem, on both sides. It would just become another arms race, as the developers would use it to identify and move into target project areas/components to get themselves on the list of un-fireable workers. Ideally, the workers would ensure work together to ensure that the truck_factor was zero, i.e., none of them could be fired.
Of course all of this rapidly becomes a (nearly)complete waste of time, proving the blogger's friends original point: >>"My coworkers said it would immediately hit Goodhart’s Law. "
Bus factor is one way to think of it. Another is it lets you spot silos, or engineers who aren't working with others, or places where you can't as easily move engineers around (so you can fix that).
Some developers fear fungability, they think that that one system only they know is job security. I see it the other way, I see that as a technical risk, but also a thing that might be keeping a great engineer from working on more important projects. Or the way to work on something else when you get fed up with that one system you hate.
I don't fear fungibility - if a place doesn't want me there I don't want to be there either.
I dislike idea of fungibility because it creates huge overhead and avoids using talented people to full potential - because they are not fungible.
In some places that's warranted - but in others - the process overhead is far greater risk to your project success than bus factor.
Spoiler he left company within 3 months
"You can't be promoted if you can't be replaced."
I like your framing better!
I hate all sorts of gatekeeping behavior but I especially despise having to wheedle information or effort out of someone like that. They almost invariably have an ego that is out of proportion to their contributions.
Side note: Occasionally I have seen people who take on the thankless task of being a maintainer of something no one else wants- those people tend to welcome others and share as much as others are willing to listen to, so being the single person who knows something doesn’t make you a gatekeeper.
It’s the gatekeepers that I hate.
And this is pragmatical for an employee who's primary goal is not optimizing Amazon's income.
It has never once been an issue and I think fungability is a part of any healthy system. I spent a few years on the other side of the table, and one of the first lessons you learn in management is that everyone is replaceable and that's solely a matter of cost. Because of this, high levels of knowledge may also work against you as management will likely work on reducing the risk you present. Another thing you'll learn is also how random layoffs can be, especially if they happen for economic reasons.
That being said. I avoid working at places with these silly metrics. The more red tape you put in the way of good work the less likely I'll want to work with you. There are a lot of reasons for this but the primary one is that these sort of things tend to create work cultures which just aren't good for productivity and quality as people "game the metrics" instead of doing good work.
Maybe this is born from spending so many years in Amazon (with it's high turn over and near-quarterly re-org shuffling), but what's getting called "replaceable" here I'd call "writing maintainable software."
The goal is to get knowledge out of your head and into the codebase so everyone can reap the benefits. Knowledge hoarding is lame.
Parallel is running 8 git clone jobs at once (as asked for) and each git clone is starting as many index-pack threads as it wants. Temporarily setting pack.threads to 1 (via git config) would help here.
The solution is for only one of the two layers to parallelise.
—⁂—
Given that you talked about pack.threads, here’s its description from `man git-config`:
> Specifies the number of threads to spawn when searching for best delta matches. This requires that git-pack-objects(1) be compiled with pthreads otherwise this option is ignored with a warning. This is meant to reduce packing time on multiprocessor machines. The required amount of memory for the delta search window is however multiplied by the number of threads. Specifying 0 will cause Git to auto-detect the number of CPUs and set the number of threads accordingly.
If you have a common scheduling API, you can manage this much more elegantly. For example make(1) can control the concurrency level across recursive invocations by using a "job server" https://www.gnu.org/software/make/manual/html_node/Job-Slots...
With this, you can have, for example, 3 top level subprocesses, each spawning multiple threads of their own, but never exceeding the CPU count.
Alternatively, parallel could make it's subprocesses think that they're running on a 1-core machine, although this may have some subtle side affects.
Or just use `git -c pack.threads=1 clone`: https://git-scm.com/docs/git#Documentation/git.txt--cltnameg...
The Bus Factor is how hard your team would suffer if you - or anyone else on the team - got hit by a bus.
Ideal Bus Factor for all team members is 0. This might sound counter-intuitive at first, almost like "make everybody expendable", but it's quite the opposite and kind of the point.
Teams should be good enough that they are a) autonomous and b) there are no mysteries. In the ideal state, everyone understands how everything works. New employees should hit the ground running and be able to produce value immediately. Departing employees should feel comfort in knowing that there are no unknowns.
An ideal team with 0 BF across the board is desirable. It means that team members are fungible. It means that every single team member can fill in the gaps if someone is ill, or on vacation, or actually leaves or is removed.
More importantly, a 0 BF is a reflection of simplicity. The software, its build/test/deployment pipelines, documentation, and support, should all be cohesive and coherent. Siloing information in team members is bad, everyone should be able to build and deploy.
0 BF is a healthy metric, but it is absolutely 100% not measured in email rate, commit rate, PR rate, lines of code, timeliness, GitHub heatmaps, etc. Those metrics indicate nothing at all. Quite the oppositve. They are harmful, awful metrics.
Measuring people by these metrics is just monkeys on typewriters. More startups need to hear this.
> Ideal Bus Factor for all team members is 0. This might sound counter-intuitive at first, almost like "make everybody expendable", but it's quite the opposite and kind of the point.
I've always heard bus factor described in the inverse fashion, as in "how many people would need to get hit by a bus for the project not to be able to continue", with the optimal number being the same as the number of people on the team. It sounds like the idea is the same, but I'm surprised to find out that the number people to convey the concept isn't always the same.
I've worked on projects where we had engineers who were one of a countable handful of people in the world with their particular skillset.
The bus factor was most certainly 1 at that point.
> More importantly, a 0 BF is a reflection of simplicity. The software, its build/test/deployment pipelines, documentation, and support, should all be cohesive and coherent.
For projects which push the frontiers of what is possible, simplicity isn't an option. (Granted these are a small % of overall software projects that exist!) When something has never been done before, you aren't worried about keeping the code As Simple As Possible, you are worried about how the hell you can even do this particular thing.
I'm not saying the code should be low quality! However sometimes doing hard stuff involves complex code, and maybe a couple generations later people have figured out design patterns so the hard stuff can have less complex code, but that may be a decade down the line!
> grug understand all programmer platonists at some level wish music of spheres perfection in code. but danger is here, world is ugly and gronky many times and so also must code be
If the most complex thing you can build is a todo-app, then I think you don't produce much value to society.
Also, how many variations of a to-do app does the world need?
The military thinks that way. They expect to lose people and keep going.
I dare say you could likely assassinate half the command chain, and the military will still managed to get where they need to be, when they need to be there. Military command chains have levels of redundancy that civilian organisations wouldn't dream of.
As a concrete example, it's estimated that the British lost ~40% of their officers in the Battle of Albuera, and they still managed to repel Napolean's forces.
> Our estimation relies on a coverage assumption: a system will face serious delays or will be likely discontinued if its current set of authors covers less than 50% of the current set of files in the system
An author of a file is defined as a user who has made significant (based on pre-calculated weights) contributions to a file
I refused to do it because I didn't like where I saw it going, but another coworker did. Unsurprisingly, the person who got the most email and sent the most email was the Sysadmin whose email account was set as sender for all the automated emails from the various servers, as he would literally email himself hundreds of alert emails a day, on top of all the crap newsletters and digests he subscribed to.
On the other hand, all of your coworkers asking you to not do something that could potentially impact their jobs, and you do it anyway as a hobby project? Sounds like kind of a jerk move.
I do want to see if open source software I use has spread the knowledge to increase survival.
I refused the jerk move!
A common misunderstanding in tech companies, I think. You don't want to exchange great developers for mediocre managers.
Everywhere I’ve been part of development, code review is part of ensuring code quality, catching bugs, and avoiding silos.
Unit test aren't a substitute because unit tests check that the success paths are good. That's a good start, but it's not the same as verifying all the possible ways code could go wrong in a complex system, and one of the cheapest ways to spot those problems is with people familiar with complex system looking at new code.
Code review give you the double benefit of building more people who understand the whole system, and having the code looked at by people who understand the whole system.
I'll grant you they can help break down silos, but the question you should be asking, is why your codebase is so convoluted that silos are developing in the first place?
When startups have to do layoffs, the question isn't "who can I fire and keep the existing business going" it is instead "who is the team I want building the next version of the product quickly enough to not go out of business"
But every fork in the road is just exactly that, and not choosing a path quickly enough has been the death of many
Less morbid that way.
People hit by a bus disappear immediately.
It's just not the same thing.
Also, it's not guaranteed that your management will actually tell you if they did - one employer asked me not to tell my team I was leaving until the last day for morale...
It is like when people complain about people using sports metaphors. I say that at least it is not military metaphores.
Also, there is usually no handover anyways. So the suddeness factor is not that important.
Well, see, we can't just straight-up tell employees that we're not going to give them promotions or raises, so they'll have to jump ship in a year... That'd be a disaster for morale!
And still less morbid. :)
To me "lottery factor" seems overly po faced and pious.
I prefer bus factor because... memento mori.
Maybe I missed it, but I didn't see the author mention what came of this. I'm very curious: did someone else take it over, or did the employer go down in flames?
On the other hand, millions of lives have literally been lost on various hills on various battlefields, yet no one seems to mind when someone says, "this is the hill I’m willing to die on." :)
"Lottery winner"
Someone wins the lottery.
(I am old - I start with the poor bus driver scenario - but then try to inject the new phrasing into the conversation - as typically I am the only technical person on my team and they begin to rely on me for EVERYTHING)
Teams are social constructs, and you simply cannot apply an algo to some observable code metric and get any kind of proper result. People leave, others step up, or don’t. End of story.
The problem with these kinds of metrics is that even if people know they are ridiculously off what they mean, some still think that the idea they convey is correct: ie. That if x people were to leave, the project would stall. That premise is simply not true.
Don’t even think about this stuff. It’s stupid. If you want to know more about these risks, talk to the team. People on the team know if anyone is irreplaceable.
How to generate a hit list to hurt a global economy
I think Linux is safe because everyone uses it.
Do not ever accept management directives from someone who couldn't do your job.
Yes, you heard me. If your manager cannot do what they are asking you to do, fire them immediately.
This has the added bonus of ensuring that Peters Principle doesn't become a major mountain instead of a mole-hill.
This does not mean that the manager has to do your job. It does mean that making sure your manager knows how to do your job is your responsibility and something that you, yourself as a worker-cog in a large machine, actually do control.
I have shipped software all over the planet in all kinds of markets for all kinds of users. The most successful project is composed of individuals who have great deals of trust in each others' ability to perform, and who share the load in ways adequate to the task. In the most successfully managed projects, those managers who I knew could do my job, but didn't (because I did it), were absolutely the best to work for .. whereas those who had no idea how to wrangle a single line of code, yet gave themselves the altitude required to be a 'manager' were, across the board, a catastrophe.
BTW, if you feel 'seen' by this comment - i.e you are a manager who feels a bit imposter'ish - don't worry, this is Peters Principle at work, and you can easily fight this by better communication with the folks whose work you cannot do ..
https://codescene.io/docs/guides/social/knowledge-distributi...
I recall seeing the linux kernel repo analysis as a show case, but I can't find it anymore.
Check it out!
It could roughly model the importance per team member or the net impact it would have on the "stargazers" if the maintainers were hit by a truck.
mclare published the other half of the blog post with nice graphs a few hours later: https://mclare.blog/posts/the-bus-factor/
I'd like to build a tree of open source dependencies and find out which parts need the most knowledge spreading.
PS: I suggested to LWN https://lwn.net/ that they might want to contact you about the Linux numbers.
Blindly staffing one's early stage startup with BigTech engineers is a pretty good way to tank a startup.
I want to see for funsies what my projects look like, but I dont want to run some random crusty java.
I've always preferred the term "lottery factor". If this employee won the lottery (almost certainly leaving their day job behind), would you be able to survive their sudden departure?
“Bus factor” is a more realistic reflection of the threat model, and resonates with experiences anyone who has been working more than a short time probably has more than “lottery factor”.
Same discussion but it’s less morbid, and doesn’t end up sounding like your prioritizing the health of your project about the literal lives of your employees.
No one has presented this idea. Your arguing that we should stop paying employees, because they will obviously continue to work unless they are only working because they need the money?
Saying that if someone wins the lottery they’ll likely leave to Perdue the opportunities that bag of money will present them to is not the same as saying the only thing keeping them at the job is money.