Ask HN: How do evaluate a Senior engineer?

I interviewed A LOT over the past year. Most of the time senior meant some kind of framework tool monkey. When the interviewer started asking senior level questions it was clear my definition and their definition of senior were very different. Most of time is was clear they were looking for anybody would could put text on the screen with confidence, which feels extremely junior to me.

The interesting moments were when somebody with less than 3 years professional experience were elevated to management because it’s a startup. They were so lost trying to interview somebody with 20 years experience and prior experience as a corporate director level. I completely stopped talking to start ups.

Eventually I got very lucky and found a company that is awesome with incredible leadership. They were actually looking for a senior to come in and fix some architectural things and in return they would teach me areas of programming I had never done before.

Here are the common danger patterns I observed over and over:

* Most importantly look for strong leadership in the employer first. If it just isn’t there stop talking to them. They are more than happy to waste your time in repeated interviews.

* Determine the level of honesty present in the work culture. Look for brutal honesty up front and don’t be shy about it. If they are timid about honesty in an interview you will spend your employment walking on egg shells and playing politics, the kind of place where shitty brown-nosing people get promoted over you.

* Next evaluate for the quality of work. Do this by asking about writing original code. If they are timid about original code or are horrified at the thought of stepping outside the yellow brick road to produce a superior product I don’t want to work there. There are many people thrilled to work at someplace average building a shitty product so they should hire one of those other people.

* Finally evaluate for retention. Does that employer work extra hard to retain people? If not you need to drill into this immediately.

Don’t be shy about asking tough questions. You will be devoting a block of your life to these people.

A senior engineer has sufficient knowledge to successfully avoid companies that do not know how to evaluate senior engineers.

Personally, it took me 20 years to reach this understanding. I wouldn't be surprised if other senior engineers, smarter than I, reached it in much less time.

Indeed, part of becoming a senior-level-anything is recognizing what problems exist and having the skills (experience, knowledge, etc.) to work on solving them.

I can't put my hands on the source at the moment, but there's a cliché that I've read somewhere that goes like this:

"Juniors focus on the how. Seniors focus on the what. Principals focus on the why."

If you're a junior or senior who asks why, you're screwed. That's my experience.

To me, becoming better at my career should mean that my job options have expanded greatly. Being shut out of some opportunities because they have different metrics does not sound like a win.

Get objective not subjective. Use CircleCI's competency matrix.

Concept: https://circleci.com/blog/why-we-re-designed-our-engineering...

Matrix: https://docs.google.com/spreadsheets/d/131XZCEb8LoXqy79WWrhC...

Evaluate yourself critically and honestly within every single row. Learn to convey (market/sell) yourself to an interviewer on those dimensions. Be truthful, people can tell.

In the event these interviewers are actually picking up on something, it may be that the "senior" for them is along a dimension they consider more important than a dimension you think matters.

For good measure, begin to work deliberately on those rows where you aren't as far along as you'd like.

I suppose that's semi-objective. Although I doubt most people fit into this. I certainly see many seniors who don't fit, and many mids that do.

The real problem is that even if you meet the criteria listed here, the criteria or examples at other companies will be wildly different.

Are we talking about the same matrix?

Six levels from E1 - E6, and objective level details for each of 27 competencies?

What do you mean that "seniors who don't fit"? What do you mean "mids that do"?

What criteria are "wildly different"? Granted, a hiring firm is unlikely to have rationalized the evaluation this well. But it seems unlikely they're looking for something that isn't at least represented here.

"objective level details"

Are they? They seem like a manager can justify someone being at that level or not. There's certainly nothing in there about how consistently one must meet each of these. I guess seniors never need any guidance or help from a teammate...

Most rankings / evaluations will be subjective with or without a rubric.

I think that matrix is a good tool. It is objective, and it does seem "the least you can do" - so in other words if you tick all the boxes you are doing OK, and if there are some missing that is definitely stuff to work on. Excelling would be to go beyond the matrix. And I think this is how the matrix should be.

I sort of hate that we have to define it so formally, and it would be nice to think "she's a good dev" and that instinct is it. But in reality a benchmark is needed to be fair and prevent bias.

In addition to be a good engineer you need to code, and coding tests are a way to find out. Ideal ones would be open ended, plenty of time, and with the interviewer "searching for success" to try to counter interview nerves. "Oh yeah I made that mistake I should have done XYZ" is as good as getting it right first time.

In an ideal world there would be some standard basic coding tests to avoid fizzbuzzing everyone. Sort of like AWS or Azure certifications, but for actual coding, that are respected, as a base level. Do them once every 2-3 years say and that is your passport past some useless interview stages. But I digress!

It's not you it's them.

They are just leveling you internally against some internal metric. They might have a lot of well qualified non-senior in title folks. Bringing you into the club may ruffle some feathers. It will depend on how isolated the teams are and the cross communication requirements.

If they say you are not senior ask where on the scale you fall as a non-senior. Then ask for their rubric that describes the levels and the promotion requirements. All of that should be standard and is fair to ask. Say you need it to determine next steps.

That'll give you the information you need to make a decision.

If OP is interviewing externally then I wouldn't bother asking for this. They probably don't even have a formal scale or rubric, or if they did they wouldn't share them with an outside job candidate.

Eh maybe. It’s just a game to entertain the workers. Solve business problems and create value that’s how you move up.

Really, the first question you should be asking the people who interviewed you is "Given your evaluation, what areas do you guys think I need to work on?" If you're in a big company you can then go off and get training etc.

The biggest difference in a senior role is that you need push back if required, and hunt stuff more information if required. Asking the above sort of question would be the first step to getting there.

Senior developer is complete BS. There's no objective definition. Even if such a definition exists, it's not applied evenly.

Edit: why disagree? This topic constantly comes up on here, with no consensus.

I sadly agree, I have seen standards of senior developer which make sense to me but even the orgs whom wrote them do not apply them evenly.

In the past year I saw way too many intermediate/senior engineers almost hired as staff.

But you have to be pragmatic, man. I don’t give a damn about titles… unless companies have salary ranges according to them. If company X pays Y for the Senior Engineer I role, and pays Y+Z for the Senior Engineer II role, you bet I’ll sell myself as the latter even if there’s little difference between the two roles from my perspective.

I don’t tell my colleagues/friends /family that I am a Senior Engineer II, I tell them that I’m a programmer. Now, on my CV and to recruiters I say: Super Senior Software Astronaut III, so pay me more. Obviously in the interviews you have to fake it until you make it, but when you have been in the industry for more than 10 years like me, faking interviews to fit whatever expectations companies have it’s not that hard.

Very true, comp is likely the most level-set comparison (at least within the same geographic area and industry).

Ability to take on responsibility for a broad and ambiguous scope that involves technical complexity. Communication skills to distill those challenges into resource asks for the business side. Ability to develop and share vision, plus measure and communicate progress against milestones to implementing vision. The ability to suggest at least a few key metrics to measure success.

Senior (one step below staff in my company's hierarchy) engineer who does interviewing here. I don't know you, as you say, but I can tell you what I understand to typically separate junior, mid-level, and senior engineers (without making you wade through charts), and what I look for:

Senior is the "career level" for a software engineer - that is, it's the level you would expect most basically competent people to be at after 5-7 years of experience. The key difference between senior engineers and the lower levels is that seniors guide the planning process and technical architecture of the software they work on. They don't just or mainly do individual ticket work. As a senior engineer, I have multiple projects within my team for which I am the lead engineer or "feature owner". That is, I conceive the technical approach, I work with product managers and stakeholders to gather requirements and create and estimate the work. I do investigative engineering work to demonstrate the value and feasibility of the work being requested. I understand our systems to the point where I am comfortable devising high-level, large-scale modifications to their inner workings. I am also a mentor and guide for the other engineers on my team and in my organization.

Essentially, I am a mini-manager (of technology and planning, not people). The difference between me and a staff engineer is that I do this at an intra-team level; I don't do it for a whole business line or working group of multiple teams. I also probably spend a lot more time working on implementation of individual tickets than a staff engineer typically would, and less time monitoring or planning. The skillset is broadly similar, though.

If I am evaluating someone for a senior role, therefore, I am asking:

* Do they have a depth of knowledge which would make them effective doing that kind of technical leadership work?

* Do they have experience leading projects and/or architecting software, not just implementing tickets?

* Could I trust this person - after they've ramped up into our company - to independently do the above?

* Will this person be an effective mentor for other developers who are more junior than them?

* Will this person be able to communicate effectively with management and other stakeholders?

And I want to see examples of all of these things. I think that it's likely that, if you are being turned away for being "not senior enough," it's because you don't have or aren't making visible your experience in these areas.

Also, for what it's worth, when you mention:

> As one example, walking through my take home project one engineer asked me about a kind of React gotcha. I said I wasn't quite sure, and then he admitted that he only just come across that in their own code recently. But I felt like it had already counted against me, even though he himself hadn't known about it. I guess it felt like I should have been the perfect version of who they aren't yet.

If it actually counted against you, it is a yellow to red flag - but I ask these kinds of questions to assess how a candidate thinks about issues. I love it when a candidate says "I haven't experienced that, but the way you describe it suggests X Y so I'd look up Z and do A B C", or otherwise has a conversation about it with me.

Hope this is helpful!