Guide to scaling engineering organizations
stripe.com
stripe.com
Find good managers! Or groom them. Whichever. Just make sure you have good managers.
I see endless posts about recruiting, and I see tons of posts about engineering culture, but I almost never see posts from these same sources about the nitty gritty of how to evaluate and grow leadership and people management skills. Ironically, I see a lot of that from MBA programs.
Actually getting good management in place is extremely hard to do and does more for talent retention than almost everything else. For every company blog post I've read that talks about scaling the organization, including Stripe, I also hear corresponding horror stories from friends and acquaintances who are leaving because they hate their manager and because their company isn't doing anything about it.
Sure, it's relative, on the grand scale of managerial quality I'd wager that Stripe is on the right side of average, but I'd really like to see more focus on scaling leadership.
You can see a few people with little technical acumen, shallow knowledge of the latest trends, very bad on processes and no work ethic that still rise in some organizations.
These people spend a lot of time "managing up" on a side and on the other side they are also nice with their employees. They help them move up the ladder and pay higher bonuses demanding less work than other bosses. It is hard to find even good employees give a bad review to somebody that give them extra money and make them work "comfortably".
1/3rd of my job is "managing up," that is, getting my boss what they need to get their job done. This can be defining a hiring process, or writing part of a powerpoint deck or getting them the information they need to write it up for their presentation.
1/3rd of my job is "managing across," or working with other managers - the "shit shield" is often from other managers who have their own pain points and are trying to work through issues like whose team has to deal with this ill-defined goal that doesn't really belong in any of the teams but is important. (Or saying that it isn't important after delving into the issue!) Or it's identifying a department-wide problem and working with a small group to have a proposal to solve it.
That leaves 1/3rd of the work on actually "managing down." That is mentoring, performance evaluation, conflict management, 1:1s.
And if your team is hiring, you spend another block on that. Most of us can't work 60 hour weeks consistently.
Are there bad managers? Sure! But some of what makes a good/bad manager is completely hidden. (I'd argue that it's a flaw in our organizational system in general - I'm starting to think we need more people whose job it is to organize the department as a system, and then have the engineering manager as a tech lead++.)
Back to the "managing up" I was talking about in my previous comment. I have seen some people spending more than 1/2 of their time on it. The reason I put it in quotes is because sometimes is time spent on getting visibility, becoming "good friends", and even brown-nosing. For any employee it is vital to have a good relationship with their boss in order to maximize her impact and develop influence. But when building camaraderie with her boss becomes the primary activity, it tells you something about the culture that that person is reinforcing and people that is attracting.
There is a difference between aiming to "create an alliance to get things done" and "develop a relationship to get recognized and promoted" Even if the two do not have to be mutual exclusive, the motivation for bad managers is predominantly in latter more than former.
And what is an order of magnitude better for a company? Someone that looks more for having an impact or a person that looks more to their advancement on the corporate ladder?
A lot of poor managers I've worked with have a knack for taking work that is actually quite easy and communicating it as very challenging such that they (or their team) receive a lot of credit for it. As you stated in the other post, they may also be good at making their employees feel that there is great value in work that is also fairly trivial or non important. In a micro scale, this looks like a good thing (people are happy), but in a macro scale, it may not actually move the business forward very much.
At the end of the day, it often sadly boils down to "it's not what you did but what people think you did" that matters.
This seems like the answer to the first line question of "how do I scale an engineering organization?" but you are not at an addressable solution yet. It's valuable but not sufficient.
I think the full answer to your question would deserve at least blog post ;-)
Someone who brings their team achievable goals, and the resources they need, and then gets the hell out of their way.
Someone who encourages leadership (in all its many forms) to come from within the team, and doesn't see it as a challenge to their authority.
Someone who understands that respect is not magically granted by position.
Someone willing to make decisions - sometimes tough ones - on behalf of the team, to communicate both with and on behalf of the team, and defend the team's integrity and autonomy.
Someone who understands that communication starts with listening.
1. Organization skills/project management skills: ability to pull together internal/external resources, ability to create clear plans, ability to build relationships with other teams to make cross-team collaboration easier
2. Communication skills: ability to deescalate and mediate conflicts fairly, ability to give both positive and negative feedback regularly, ability to create psychological safety, ability to deliver bad news while maintaining psychological safety, ability to run efficient meetings, ability to make unpopular decisions while still commanding the team's respect, ability to engage with different personality types and communication styles
3. Delegation skills: ability to translate ambiguous directives from above into actionable tasks for the team, ability to arrange tasks so that individuals have the creative freedom to figure out how to do it, predictability in decision-making, ability to shield team from external pressure, ability to give negative feedback immediately, ability to fire low performers before they fester into team morale and culture problems
4. Coaching skills: ability to give targeted advice, ability to match employee's own aspirations and growth objectives to projects, enough technical expertise to point people in the right direction, ability to evaluate skills and hire for them
They also often take theory. If you’re reading this and trying to learn more about management, don’t shy away from taking some time to read good books which teach about the theory of, say, how to be good at feedback.
Get those right and everything else will follow. So simple but only a fraction of managers get it.
1. Create a continuous feedback loop with every single person on your team. This is why 1-1 meetings are extremely helpful.
2. Understand where each person on your team wants to go (career goals). Do they want to become X in 2 years? Do they even want to work in tech at all? Find ways to help them get there.
3. Document all the important conversations you have. Share it with the person. Most miscommunication happens when you talk about something and it's interpreted in various ways. Documenting these conversations and sharing them helps cut down on this.
4. Act like you care about them as a person.
Some companies completely screw up themselves because of this. Startups usually do not think about this early enough and when the founders realize what is going on is too late and the company culture is screwed.
My favorite Steve Job's video on the subject: https://www.youtube.com/watch?v=rQKis2Cfpeo
Having had good and bad in my career has saved me from realizing that this was happening and that I shouldn't sit idle but if I'm not idle and do something it becomes extremely disheartening to watch this happen (and then I leave when it becomes apparent that nothing will be done and I'm miserable under my manager)
I have improved a lot technically because there are a lot of programs that allowed me to learn.
But not once have I been taught to manage people.
After realizing that I am getting nowhere career wise as an individual contributor, I am currently seeking out opportunities to learn management.
Stripe uses the same process for scaling our management corps as we do our ICs. All the steps are tailored to the role the person would be filling at the company.
That being said, I think this is a great point. All the considerations others have brought up in this thread can be multiplied for manager roles. It's very important to get this right when you're scaling up a team!
- hiring by committee as opposed to whims of a given manager
- introducing bar raisers in hiring loop
- promotions requests reviewed by independent committee
- demand for metrics driven automated dashboards
- enforcing skip level 1:1s
- anonymous surveys and complaint box
- enforcing culture of proper credit attributions instead of "we did X"
Many assume the people that they hire are flawless. That's why you see all the posts on hiring. It's easy for people to think they know everything at this point. It's like celebrating after you acquire a customer...what really matters is if they stay (and how long).
Currently, there's no feedback loop in place to surface this information. Some companies will do a culture or 360 survey that highlights a few issues here and there, but it's a poor process for determining competence and it's administered by HR (or some function of HR). It needs to be owned by the team leader (and hopefully their boss).
I've been working on an app (https://www.fridayfeedback.com) for the last three years to try to help this. Eng teams love it because it's asynchronous and surfaces new insights to help teams improve.
What isn't obvious is how to find the right people to help us do that, or how to avoid the wrong people. Because the wrong people on the recruiting side of things can easily drag down the entire organization very quickly. I've only worked with a handful of recruiters that were actually worth anything - I'd like to see more about how to find them, what qualities to look for, and how to vet them.
Building a process that doesn't suck is like bread and butter for day-to-day for a lot of engineers. A lot of us have been part of working recruiting pipelines and we've seen how it's supposed to work. It's usually pretty straightforward - what wee don't know is how to kickstart it with the right people. I'd love more information on the softer part of it, and the part that is often invisible to people iterating on the process - the specific contributions of individuals to get it up and running.
I learned how to hire & train recruiters to work for me pretty effectively for certain pieces of recruiting. To you, what makes a recruiter good and/or not good? Maybe I can share some thoughts on how to improve the not good parts!
Once you develop that working relationship with a recruiter, never let go! I have a handful of recruiters that I work with because they quickly absorb the feedback I give and I never see another candidate with that missing requirement again. If a recruiter shows they can't adjust/learn from your feedback, find one that does.
I was talking more about the building a recruiting pipeline to help us scale product/engineering. Generally, a good recruiter is one who saves the org time, and a bad one costs us time. The costs could come from having to iterate our process and get aligned on candidates, or it (usually) could come from having to reject a lot of candidates that might seem good but have some kind of systemic issue). Sometimes a recruiter can only seem to pull in C+ or B talent, and after a while you start to get convinced that's your bar for greatness because you're not even seeing solid candidates anymore.
How can I tell if a recruiter is any good when I'm interviewing / vetting them? It's kind of hard to tell without them getting some candidates in front of us. I can look at their past, but how much of that might be attributed to the company they were working for? When you recruit for a sexy company, it's probably really easy to build teams cause good talent seeks you out. Our company isn't sexy and well-known yet, so how can I tell if a recruiter will be able to get the talent we're looking for?
Bingo. If you have to go out and find people, you're gonna have a hard time.
Care to share more about this process? Or link out to a blog post or something?
But also, you gotta people what you really want, starting with the job description. In my personal experience, 75% of unqualified applicants apply because they thought they were a fit when they never had a chance. Saying "seeking high energy team player who likes coding" is gonna attract everyone you didn't ever want to apply to do just that.
They can be told to look for certain keywords, sure, but that doesn't, IME, seem to be sufficient to screen; obviously bad résumé use the right words (but in the wrong ways / in ways that to an expert clearly indicate a lack of understanding) so they pass the filter.
Brief technical screens can be done, but again, without technical knowledge, the recruiter can't know if the answer given by the candidate matches the answer they have in an answer key, if they are provided with one. Even relatively simple technical questions might have more than one right answer, or the answer might just be phrased in a way that a recruiter doing a human version of lexical edit distance isn't going to think passes, but any engineer would say would.
The problem in all of these is the lack of technical knowledge. Tech recruiters, on the whole IME, are trying to recruiter for a role that might as well be "town wizard". Any theoretical recruiter with technical knowledge would never work in recruiting — they'd fetch more in pretty much any real technical role, like an engineering role.¹
The other problem, again IME, is that there is a wide pool of candidates with very little actual experience, though they may have worked any number of years. Finding someone with actual knowledge and understanding of software engineering requires sifting through a lot of chaff.
¹One might see this as people aren't willing to sufficiently pay enough for recruiters / if you want a recruiter with technical knowledge, you need to compensate them adequately enough that other opportunities are not worth their time. I would agree here.
Here's basically how I did it:
- pick one job to focus on, where it's near impossible for a non-technical person to tell if someone is likely to be a fit, such as a full stack web developer with experience in a JavaScript and a backend language
- get 5 resumes, 1 obvious fit, 3 maybes, and 1 obvious no, all of which look great based on keyword soup, so an automated QA person and webmaster might look like a full stack developer, and the waiter who took a programming course 10 years ago has no shot
- provide any additional filtering criteria
- in a verbal conversation, ask the recruiters to rate, and explain, each rating
Try that. It's not perfect, and it's better for positions you can describe discrete hiring criteria. The bar for the recruiter is not filtering out the no and maybes, it's screening in the maybes and yesses into a guesstimate sorted list. You can focus in 3 & 4 list. Then you can start with obvious matches, keep going until the list starts feeling pretty thin, and draw a line based on your judgement call & supply of applicants.
How to hire your first engineer - https://blog.ycombinator.com/how-to-hire-your-first-engineer...
Convincing Engineer to join your team and a basic structure for the process - https://blog.ycombinator.com/convincing-engineers-to-join-yo...
General thoughts about hiring - http://blog.samaltman.com/how-to-hire
The book "Who" gives a good idea about a good process. Hard to apply when you are not looking for a "generalist" - https://www.amazon.com/Who-Geoff-Smart/dp/0345504194/
Some of the manager tools podcasts episodes - https://www.manager-tools.com/map-universe/hiring-0
Edit: formatting
I wonder if they miss out on whole groups of job candidates because those candidates have good job prospects that don't involve this gruelling slog.
Of course, Stripe does seem to have an excellent team.
It's an expensive (painful) signal that you actually do know what you need to do and are capable of it.
FAANGS may have the funds together to hire out probably the rest of the entire software developer market but it doesn’t do them as much good to have laggards that can’t pass a FizzBuzz test. Sure works for a lot of defense contractors’ business models though, especially if the candidates are cleared.
If you just need someone for narrowly-scoped tasks like fixing specific bugs, it might be better to hire a contractor, so the role expectations can be well-defined and agreed to up front.
It’s pretty evil IMO to hire people as FTEs into such organizations when you are probably better off with consultants that are at least used to short term gigs.
Then there's the "unintentional" body farms that organize due to structural reasons of massive funding combined with labor structure dictated to support a massive enterprise. Within defense, almost all the major defense contractors like Northrop, Lockheed, General Dynamics, and maybe Palantir these days (the bar for hiring is higher overall there, but not quite the full story) fight for huge, bloated Pentagon budgets that resembles VC funding except with military officers as partners and minimum headcounts are usually specified in contract vehicles. You'll oftentimes see at least n PhDs on staff, m veterans, p program managers, etc. as competitive factors for getting a contract ahead of other companies because with too small of a company it's deemed a risk to the government (people do take vacation, get into accidents, etc.). While there's been some efforts to make the RFP process better aligned towards outcomes, everything I've heard since I left the community has not been encouraging.
For most companies missing out on an excellent hire is worse than hiring someone subpar.
I'd actually say the opposite: standout engineers can be very valuable, I'd rather work on a team with a couple lemons and one really good engineer than 3 middle of the road engineers. So I think that false negatives can be damaging, especially because it's probably the best people with the most options who won't bother with an excessive interview gauntlet.
Will your standout engineers enjoy working with the lemons?
I had one hire who was brought in as a junior dev, thought he was better than he was and that he didn't deserve to do the grunt work he'd been assigned, and then proceeded to constantly bitch about anything he could. He was so negative that when he finally moved on I was super relieved.
That psychology of "why even bother" can just sink into people's subconcious and become problematic. It's hard to spot who's going to be like that at an interview though.
Generally a hiring process is a good hint to the dysfunctions of a team before you join. Panic about bad hires and a quintuple-filtered pipeline is a good sign of weak management and a structural inability to fire.
But what about a very good hire that you end up missing that has been a very expensive mistake?
I think we have a pretty high bias here. Organizations can see the effect of a bad hire and have more data on what went wrong during the hiring process, retrospect why that person passed and put more "tests" to prevent to repeat the mistake. But organizations can never measure the differential between the impact on the company of a candidate that was not hired and the present condition. There is simply no data about the impact that that person could have had.
I heard a few detailed stories on "the guy that should not have been hired" and I can add my own stories. But the stories on the cool people that should have been hired are brief and rarely go beyond "I thought that guy was very smart."
So I think we should make sure we check our hiring processes against negative positives. How? To be continued...
A false positive often has negative contribution to the overall training (at the very least taking time from current members without giving back, but often also creating new problems).
A false negative has zero cost (and zero benefit).
I think it's a no-brainer that false negatives are much preferred to false positives in the overall tradeoff (and ... there is always a tradeoff between false positive and false negative)
> A false negative has zero cost (and zero benefit).
False negatives means that your hiring process is longer, more difficult, more costly, and you therefore find it harder to grow, or to replace people who leave.
It may well be that avoiding false positives in general is better for the business, but it's not as clear cut as you make out I believe, it's a balance.
Exactly! If a couple of people on my team of six leave and they can’t be replaced for 6-12 months then I’m going to quit as well.
There will be more pressure on the remaining team to produce and most of the interesting long term work will be put on hold.
That's the case with Amazon and Google also.
The top tech companies can reject 10 good candidates for every 1 that they hire because there is a massive surplus of engineers who want to work for these companies and so time to hire for them is actually reasonable.
99% of tech companies don’t have the overwhelming surplus of good applicants to make this work. So when they try they see time to hire jump from 4-8 weeks all the way up to 6-12 months without any real decrease in false positives.
2 phone calls and an onsite doesn't seem like an excessive amount of interviews.
We don’t do whiteboard interviews for coding. If we have a coding interview, candidates can do it on their choice of their machine or a loaner machine from us with one of a few environments installed.
Most interview loops have approximately two 45 minute coding interviews. The other interviews might focus on (depending on job/seniority/etc) design, architecture, product sense, industry knowledge, communication style, etc. (Each interview tests one cluster of skills against a rubric, as explained in the linked article.)
I’ve mostly seen algorithm / data structure questions at megacorps, which are probably too big with too much inertia to quickly reform their processes.
In all other companies i have to hit team specific bars.
So if you work at Stripe and are reading this, good work improving interviewing process
To be honest, I’m not even sure how they’d interview me again. It seems like it’d be a lot of work to have multiple versions of their interviews and at this point I know the answers to everything they showed me
Would you mind saying what you mean by 'lurking'? In all my life on the Internet, it's a positive thing when joining a community. You're invisible to everyone and just read. Why should people not do this? It seems harmless to be in read-only mode.
I noticed that in both places they use the term, they say "lurk in", almost as if it's a term with an accepted meaning within Stripe.
The fact that they say "read to your heart's content" also suggests that they aren't using "lurk" in it's usual meaning (which to me is reading but not commenting).
Stripe has a very specific culture of email transparency that they've posted about before, and they've likely developed company-specific jargon around it: https://stripe.com/blog/scaling-email-transparency
In the sense used here, "Lurking into someone's thread" means sending an unsolicited comment to a thread you were not an expected participant on.
This is something we're broadly supportive of but which, if done poorly, can be socially contentious. You could imagine pathologically bad examples like "Hey thanks for working on this project for the last 8 months, but before you ship it, just wondered if we had considered adding a teal bikeshed? Taking the liberty of CCing seven people for their thoughts on the bikeshed you'll be building."
I've seen the opposite by senior management called the "swoop and poop". They haven't followed the thread at all, but they will swoop into a meeting without any context and poop all over the ideas because they are missing the context around the complexity of the problems.
I'd sooner have a lurker that read all of the conversation speak up than the opposite.
Some time ago I went to the Dublin open house because they were opening news teams here and one of the positions was backed api engineer. Considering that I've been doing the same for almost 10 years and that I got personally invited to the event I decided to apply to the role.
They rejected me even before having a personal interview. I wonder what is happening behind the scenes but it was very demoralizing. Was anything wrong about my CV? About me? Am I too old? Right now, I prefer to stick with smaller companies, they have clearer intentions when contacting you and they are much more easier to approach.
</rant>
Honestly, it might have just been a sourcer/recruiter fuck up. If you can at all, ask a friend (or even a friend of a friend) if you can get an internal referal and bypass the broken resume filtering step.
Good recruiters accept internal referrals directly.
I went through a job hunt a couple of months ago. CV screening was by far the most stringent part of the process for me. I passed all of my phone screens and got offers from most of my on-sites, but getting an interview from an application was a crapshoot. Thankfully most rejections at that stage were prompt and unambiguous (though as you say, not helpful/informative.)
Lyft was an exception, though. Their application page said something along the lines of "We respect your time, so we won't respond back if you don't make the cut." Pretty ironic, not responding back is about the most disrespectful thing they can do at that point in the process -- certainly worse than a curt "fuck off".
(By contrast I loved Stripe's interview process.)
However, I think we need to flip it on its head. From my point of view as an engineer, your pipeline doesn't really matter, no matter how smart and well thought out it is.
The weakest link in the pipeline is what isn't mentioned here, how do you get candidates into the pipeline.
I get 5-10 recruiting emails a week and about the same number of linked-in messages, none of them ever even come to a phone call. (and yes, I have gotten a few from Stripe as well).
How do you email/call/text candidates? How do you define them? Do you have the same email for all or you distinguish between levels?
A lof of companies have a good pipeline and they say "we don't do whiteboards" but their recruiters are doing a disgusting job at finding people, just absolutely shameful.
Is that still valid? I thought the tests to support this concept had failed to be replicated?
http://nymag.com/intelligencer/2017/12/iat-behavior-problem....
https://www.thecut.com/2017/01/psychologys-racism-measuring-...
Most training to overcome unconscious bias is about defining criteria before you see a candidate. If, before you view resumes, you've written down "We want someone with a master's degree and 4 years of experience in field X," you're much less likely to pass over a qualified resume, regardless of the candidate's personal characteristics.
A critical component of unconscious bias training is creating awareness for implicit bias. Since 1998, the online Implicit-Association Test (IAT) has provided a platform for the general public to assess their unconscious biases. Although the IAT measure has come under scrutiny, it has sparked conversation about unconscious bias in both popular media and the scientific community. In addition to the public’s increased awareness of the influence of implicit biases, the reality of racial and gender inequalities in our society has led to the creation of many unconscious bias training programs.
From the abstract: Commitment to hiring criteria prior to disclosure of the applicant’s gender eliminated discrimination
I'm sure there are many sitautions where training is less effective, but the hiring example is well supported.
1. It convinces people to use objective, methodical criteria to evaluate candidates. (Whether or not the "unconscious bias" rationale is valid, the actual remedies boil down to, "have a process instead of winging it", and in today's political climate, "unconscious bias training" is an easier sell than "not winging it training".)
2. It makes it harder to sue the company for bias, because they can turn around and say, hey, look, we made everyone do unconscious bias training.
Similarly, anti-harassment training mostly boils down to, "these are the legal categories of harassment that you and the company may be held liable for; consider yourself forewarned".
I think now as well as "Security Theatre" it looks like we have "Diversity Theatre" as well.
- Stripe is a large and resourceful company with a relatively large engineering team. They can afford to have people focused only on things like this.
- The demand to work at Stripe is high. Therefore they can afford to have a long, grueling interview process. Most engineers will not want to go through a 7 step hiring process at a low prestige company.
You can see this happen in places that have layoffs every now and then, or that hires a dozen contractors for each fulltime employee. When that happens, you have people doing automation job while hoarding knowledge of which button to press, then you get to the point you describe.
For instance, the idea of “rotations” doesn’t make as much sense when there are 5 engineers versus 100. There were several instances where I would’ve liked to have known team size as a benchmark.
Yet everyone here will only see the bad points of the article, which is pretty much a recipe for bro-culture and hiring fluent english speakers exclusively.
As you iterate, your win percentage (AKA how many people you bring in and actually hire) goes up. We got to a point where we hired 75% of the people we brought in to the office.
Do you have numbers to back that up?
Even worse, you have to go through the hiring process again! So we can at least say it costs twice as much.
As mentioned, this kind of investment in the hiring process requires you to actually iterate it and continuously improve your win percentage. If not, you might be throwing money away.
Why is hiring fluent-speakers of the language in which the company conducts operations a bad thing?