Why You Shouldn't Hire More Developers
blog.patchspace.co.uk
blog.patchspace.co.uk
You thought you could handle it, but couldn't, and you should have seen this coming. The team leader failed here and let a severe issue (overwork of staff) cover his ass on costs.
We always have an extra developer on every team, no one does crazy overtime. It's worth the extra $0-$30k for the intern/half-timer. The kid gets experience and we get a backup.
Should we lose a live dev, we prop the intern/half up to full and we're back up to speed. We also track overwork/hours.
Once again, the team leader failed. You SHOULD hire more developers, and you should hire them sooner, specifically because a training period is involved.
This is something anyone who's spent time in a growing organization could advise you on, and a good reason to have mentors if you're inexperienced in business.
This is what I was coming to say. Of course Alice is going to take some handholding because she's new. She could be an A+ developer and will still require team time to help her get up to speed. The problem is that the team in the article waited too long to bring Alice on. They were already far too behind the 8 ball to have any time to get her up to speed. If they had brought her on sooner then they might not have ever been in the situation to start.
That is: they should have hired her yesterday and hiring her today won't help the current cycle, but you have to hire her today to have any hope for the future. [1]
[1] Unless you have incredibly isolated projects, in which case you can arguably put off hiring Alice until after the release, but you'll still need to bring her on well before the next one gets under way.
If you aren't making enough money to cover the extra 100 hours of development, perhaps you need to change your pricing strategy or have lower expectations for margins.
we are located near SF, so far we had a hard time finding interns - cause the young kids are busy building start-ups themselves...
Everyone has to weigh the benefits of starting a startup over having a regular job. If all the kids are starting startups, that indicates that the benefits of having a job are so low that it is always worth the risk of startupdom.
Money may not be the only consideration, but I'm sure it is a big one. Would these kids be all starting startups if the programming jobs paid, for instance, $500K per year?
It applies not just to developers, but office furniture, conference rooms, processes, and most other employee types too.
In it, he postulates Brook's Law: Adding manpower to a late software project makes it later. Surprisingly how long we've known this (1975), yet we constantly need to be reminded of it.
And the person in charge's response is predictable: "We have to hire more people!". If the business is lucky enough to be successful, you end up with a hundred developers doing the exact same thing since hiring began: working very hard to get very little done.
TRUE.
Hiring more devs won't solve the complexity problem unless you happen to lasso a few certain devs with the right experience, perspective and willingness that can identify the true core of a problem and solve it fully and with expedience. Sadly many ("smart", "rock star") devs I've seen are content with shortcut, local maxima solutions that only buy a little bit of time before the team starts hurting again. I don't know why this is. Maybe b/c it's more fun (or feels safer) to code and produce behavior than to do the vigilant things required to stave off complexity. But this is a suicide run -- you feel the wind in your hair b/c you're hacking, coding, etc. but you're actually just running back and forth exhausting yourself and not pushing the line forward as fast as you hope. Don't let this evoke The Tortoise and the Hare fable. The team that does development right will be less like a tortoise and more like an elephant. Ground speed somewhere in between the tortoise and the hare, wise, and pushing the line at the greatest speed.
General intelligence is a necessary condition, but far (very, very far) from a sufficient one for being a good programmer. Most of the inept developers I've met have been smart. Intelligence wasn't the problem.
This is why I hate Java and C++. Because it's impossible to evaluate code for quality in these languages at 200-500 LoC (while you can easily do this in Python or Scala) you can't hire based on code samples. So you have to hire based on general intelligence + interviewing skill and you end up with about 15% of your team being outright duds (worse yet, intellectually brilliant and persuasive duds because they passed your hiring process, but still bad programmers and dismal architects) even if you're extremely selective.
http://tele-task.de/archive/video/flash/14029/
"Knowledge is silver. Outlook is gold. IQ is a lead weight." (~00:30)
My experience is not nearly so dire as this article, but aspects of it do come up at times. I'll have to check out the suggested reading.
I know there is not the perfect way, but can anybody recommend me a book with a way you can confidently execute? Maybe a guide how Google does it? It may be not perfect but they produce relatively high quality code.
Has everybody/company figure these things out for themself?
Oh, the things I've seen.
Perhaps because we have a framework and documentation and a guide to get new hires up to speed. Also we have documented our practices, the tools we use, etc. And the new hires are instructed to do a project before joining the team. Once they have done the project they are familiar with a large part of the framework.
And as for those "business edge cases", new hires and all developers are personally given tasks by me, and we make sure to QA their work and code-review work by new hires.
So these are the main factors for why hiring new people has so far been relatively smooth. Still, let's not maintain any illusions -- new hires will need months to really get to the same place as existing developers.
But you don't have to have your own documented framework to do it. If you are using an open source framework and sticking closely to its principles, if you have documented your company's systems and made it clear what is expected of developers, hiring can be a great thing!
They both delivered valuable new features in the next release a few months later, fully pulling their weight. Now, they're bringing new hires up to speed.
We spent a long time (several months) looking for very good people. We imported one of them from Germany. Perhaps Brooke's law only applies because the sort of developers you can hire in a hurry aren't the sort of developers who are going to hit the ground running.
TL;DR In my experience you really should hire more developers and you should do it before anybody starts working a 60 hour week.
Interesting read, thanks Ash.
I've found myself in this position a few times, and in general, it was more likely to happen in Sales-focused organizations where non-technical sales people were all too eager to make unsustainable promises to land big customers for lots of cash.
Often the customers' real business problems could have been handled more elegantly with minimal code churn and a more sustainable pace for the dev team. But without a solid Product Manager to guide them to the right solution, an incoming request from sales was more likely to just be reformatted and then turned into a user story and/or bug.
I always felt the only sensible solution was to re-align the business expectations and help the product find it's focus, but frankly I was never successful at pulling this off.
"adding manpower to a late software project makes it later."
http://en.wikipedia.org/wiki/The_Mythical_Man-Month
These debates are "evergreen" in the sense they will happen forever on tech sites. That is because they address an issue that is fundamental to the creation of software. There are 2 sides:
1.) most people can not work 60 hour weeks forever without eventually burning out
2.) "From a business point of view, long hours by programmers are a key to profitability"
This #2 is a quote from Phillip Greenspun, and I'll quote the whole paragraph because it is good:
"From a business point of view, long hours by programmers are a key to profitability. Suppose that a programmer needs to spend 25 hours per week keeping current with new technology, getting coordinated with other programmers, contributing to documentation and thought leadership pieces, and comprehending the structures of the systems being extended. Under this assumption, a programmer who works 55 hours per week will produce twice as much code as one who works 40 hours per week. In The Mythical Man-Month, the only great book ever written on software engineering, Fred Brooks concludes that no software product should be designed by more than two people. He argues that a program designed by more than two people might be more complete but it will never be easy to understand because it will not be as consistent as something designed by fewer people. This means that if you want to follow the best practices of the industry in terms of design and architecture, the only way to improve speed to market is to have the same people working longer hours. Finally there is the common sense notion that the smaller the team the less management overhead. A product is going to get out the door much faster if it is built by 4 people working 70-hour weeks (180 productive programmer-hours per week, after subtracting for 25 hours of coordination and structure comprehension time) than if by 12 people working 40-hour weeks (the same net of 180 hours per week). The 12-person team will inevitably require additional managers and all-day meetings to stay coordinated."
http://philip.greenspun.com/ancient-history/managing-softwar...
This issue won't go away and there is no easy way to resolve it. You have a human need (avoid burnout) and a business need (move quickly) and in the world of software they tend to be opposites. The upshot of the real life stresses of businesses tends to amount to: have as few programmers as possible without inducing burnout. But management can not know how many programmers that is since it is difficult to know how much work a given developer can handle without burnout. (On a personal note, I have had good years and bad years, years when the work fascinated me and years when it did not, years when working 40 hours a week was a struggle and years when 60 hours a week was fun -- and never have I had a manager who understood my motivations so deeply that they could figure out which kind of year I was headed for.)
More so, we can not expect any general improvement in the quality of management. Phillip Greenspun makes the argument here, that there has been little to no improvement in the management of software projects over the last several decades (as he says, what improvements there have been have come from new tools, such as SQL (or in modern times, git)):
http://blogs.law.harvard.edu/philg/2005/11/10/can-we-fix-the...
After a quite short span of overtime (a week or two), where development speed is increased by a relatively short amount, productivity drops to around 10 or 20% as far as I recall.
The conclusion is that even if the developers worked 24 hours a day, they would still not be as productive as a well-rested and motivated team running standard 8 hours a day.
If you realized you were over extended you should have hired earlier. If you didn't that's another failure point.
Most work cycle books Agile or not talk about how increasing team size does not necessarily increase throughput especially initially. If you don't have a team strategy then yes you are throwing money at a problem without understanding what the problem is.
Assuming the anecdote, I thought it was a useful example that I have seen occur too many times. I am also sorry to inform you that the scenario presented was actually quite ideal. In all likelihood, the new developer would be incompetent. At the very least, he/she would have a personality conflict with the team and/or argue with them about why everything they are doing is wrong (which should seem obvious anytime you are working 40+ hours).
[1]www.lifewithalacrity.com/2004/03/the_dunbar_numb.html
Let's assume this company has nobody with a true technology background at its leadership level, because if it did it you wouldn't have this mess to begin with. That means it's basically never an option to say, "can we stop working on business projects for two weeks so we can fix problems with our infrastructure that will make us more efficient later on."
Suggesting something like that never works because:
-- The idea of halting "forward progress" on "revenue-driving initiatives" is anathema to any executive. I've once worked on projects where the conversation with my manager has gone like this:
"[Executive] wants us to make progress on the SuperProfitGenerator Widget this week."
"On top of everything else? That's a HUGE project! How could we finish it this week?"
"She doesn't want it to b finished, she just wants forward progress."
"What's the definition of forward progress?"
"Well she just wants to know we're working on it and we're closer to finish than we were at the beginning."
"But I'm not supposed to have anything functional or anything, just 'forward progress.'"
"Yes."
"By what metrics? Lines of code? If I tell her I wrote 10,000 lines of code for the SuperProfitGenerator Widget, that will make her happy?"
"Yes."
"Wow. Okay. Um, yeah, we can squeeze that in."
[proceeds to create SuperProfitGenerator.pm class and copy-paste 10000 lines of Wikipedia into it wrapped in a "print" statement]
-- It's incredibly hard to quantify the actual efficiency gains and business people basically just assume engineers are lying anyway. Unfortunately I've seen this sort of Catch-22 play out -- it seems like the business will relent, but they want to try and quantify the efficient gains. The engineers give incredibly optimistic answers because a) we're typically terrible at estimating to begin with b) the rosier we can paint the picture, the more likely they'll finally buy-in. Unfortunately then it's very hard to deliver on those efficiency gains, and even if you do deliver, it's hard to make non-engineers realize the impact. Everyone realizes when an extra $50,000 is in the bank account. Not everyone realizes that 5 engineers have each saved 10 hours, even though an extra 50 hours of productivity over a year is easily worth more than $50,000.
-- Even if they do agree to let engineers start working on systemic/infrastructure problems, it's typically the first thing that gets prioritized. Some client will come in talking about some deal that could be worth some non-zero amount of dollars but requires additonal engineering work. The executives sit around looking at the current priorities and someone says, "It looks like the top priority right now is 'Data Model Refactor,' whatever the fuck that is. What the fuck is that? Oh it's something about database the engineers wanted to do? Fuck that, make them work on this instead."
So what can you do? If you have the option of fighting to get technical debt and infrastructure issues addressed, or just hiring more, than in an envirnoment like this it's almost always the better move to hire more. Hiring someone for a later project does make it later, but the OP is talking about a systemic overall inefficiency. It's very possible the new hire will come in and gum up the works for awhile during his ramping up phase, but on a long enough timeline they will eventually out-produce the overhead. It's politically easier to bring on someone new, deal with the productivity hit, and then have them start addressing the systemic technical issues.
Although the real answer to "what can you do?" is to not work at these kinds of companies to begin with. Companies with leadership that has true technology backgrounds don't allow these kinds of situations to arise to begin with, and if they do it's very easy for them to understand the cost-benefit analysis of dealing with them. This isn't the case at your typical organization led by a bunch of MBA idiots, unfortunately.
this attitude is contributing to the problem; it takes continuous and disciplined investment in quality. the idea is that slowing everything down (forever) by a linear factor of two or three will cause exponential returns in productivity later.
I can see the reasoning behind that and it would be wonderful if that tactic actually worked. In my experience however, it does not.
I work at a company that is heavily driven by a non-technical management. They don't understand the concept of "technical debt" and your characterization of how it is to work at such a company is _exactly_ like I have experienced it. But: the managers/MBAs/what-have-you at my company are not idiots. The problem is: if they bring in a new hire into a team (which happens fairly often, we went from ~100 engineers to more than 250 in 2011 alone), they also raise their expectations of said team's output.
The team-leads (which is the highest technical position in my company) do, of course, try to fight that back and try to use the plus in productivity for fixing underlying technical problems (of which there are plenty). Sadly, they have a hard time explaining even the productivity drop that stems from bringing the new team member up to steam. After that, there is just not enough argumentative leverage (I hope that's a term that exists) left to explain why we could not move even faster, now that we are one engineer more.
So, in the end we are exactly where we were - and find ourselves longing for another new hire to work exclusively on the technical debts.
Yes and no. Politics and systemic issues are hard to face up to, but sometimes they make the problem worse, so they really do need to be fixed to make the problem any better.
Say you have everyone working 70 hours a week, and someone thinks to throw more people at the problem. You start interviewing.
2 weeks later, Bob quits in a fit of rage. (Or maybe hospitalized for exhaustion and quits. Whatever.). Now you're down an engineer, and you have to find two more engineers, making the HR department work even harder.
3 weeks later you find _one_ developer. You can't seem to find another - between good talent being hard to find, and people suddenly getting cold feet after doing face to face interviews with your blurry-eyed engineers... well there's a talent crunch going on after all, don't ya know? ;-)
And you're now 6 man weeks behind (and no closer to your goal of another engineer).
Boy, this new guy better hit the ground running, because you really need the help...
From immediate experience, this is false.
Here's what I see needs to be done:
* Stop using agile if you don't have buy in from the owners. Agile is much more dangerous than traditional methods if you do not have the right buy-in from the owners--and that buy-in is not easy to get. Without buy-in, you've just given them the ability to minutely control the expected output without the expense of researching and designing a base set of functionality up-front. It's "free" for them to change their mind--so they will, at your expense.
* Tell the lead to stop being a robot. Alice is given a task that is blocked by an old bug. Instead of just assigning a more suitable task to Alice, she is told to bring in another developer to help her work through it. Why stick with this task? The world isn't binary, so break up your task and assignments into something that fits the talent you have.
Yes, developers are not fungible completely, but there should be tasks that anyone should be able to slip in and do. It's the lead's role to be able to decide which tasks are assignable to someone new with reduced risk. Maybe start them out on just identifying the cause for Bug XXX--not actually fixing the code. Or have them confirm and document repro steps for the bugs that are coming in every day. You don't have to always give developers development tasks, especially their first week.
* Tell the lead to start taking care of the team. The story mentions developers who got pulled into meetings, preventing them from doing their job. As a lead, you should be providing some interference and getting in the way of these type of requests.
* Tell the lead to stop over complicating things. Why are there so many bugs? Part of this is that the employees are overworked. I suspect that part of it is also that the framework and processes are over-complicated. A custom build of a tool that completely breaks everything? That's a process smell. The lead needs to figure out why its so easy to introduce bugs in their system. "The requirements are all necessary" phrase also fall into the implementation as well. Think YAGNI and KISS.
As for the loss of productivity by "Alice": every new developer adds growing pains to a team, but that definitely should not persist. It's a short-sighted view that one week of lost productivity will erase any long-term benefits of an additional employee. But, you need to plan training time for a new employee into your schedule. Also, since it doesn't sound like you have it already, have the new employee document the steps for the next new employee. That will help reduce the growing pain costs over time.
It is observed that the time fixing bugs can be used in a more productive way. But those bugs still needs fixing.
Quote: "If your team is spending time any significant amount of time fixing bugs, it has much more capacity than you realise. That’s not to say it’s an easy reserve to tap into, but it is there."
Ok, I agree. So how exactly do you tap into that resource. I am currently facing exactly the scenario described in the article - a successful sales team shoving new stuff into the pipeline, an evergrowing backlog of things to fix and do (bugs, architecture, clean-up).
Our solution so far consists of:
1., We serve multiple platforms. As Blackberry, iPhone, Windows Tablet are pretty much dead against iPad, we're shifting those developers over to iPad. New technology, but at least same business processes.
2., Hiring a few more devs.
If I understand the article correctly, the investment should go into more Q&A, finding bugs asap to get out of the long tail of production fixes.
What's the HN community' take on this?
One thought is to look at what can be reasonably offloaded to a new person. For example I recently had to step in and sort out the HTML for a project for which we did design and where the developers were struggling and overwhelmed. Not having to worry about how things looked anymore was a big burden off their shoulders, and it's a well enough contained area that I could be productive pretty much instantly.
This the culture we are trying to change at work. Shift from a drop-everything-and-fix-the-bug-now culture to a fix only true emergency bugs now and mainly focus on developing new, high impact features. In other words, expanding the business capabilities of the software. Note: This will bring in more profit and grow the business which eventually will lead to hiring of more developers to handle new projects.
If a significant amount of time is spend on fixing bugs, the problem is elsewhere in the process. Solving that problem should be the highest priority, because that's were things will continue to escalate.
All of this is why you need to make a decision really fast with a new hire if you're going to cut them loose. Imagine the productivity you loose (it's happend to me) when you have to let someone go after a year because they can't get to where you need them! You have to start from scratch.