Hacker News Reader

Comment by Waterluvian | Hacker News Reader

I remember an experience many moons ago where there was some "emergency" situation and our manager was just insufferable. Every 40-60 mins wanting to check with the ~3 engineers working on the issue for an update or to offer futile advice. I could palpably feel their anxiety and helplessness. I remember it reminding me of the trope of a parent in an ER on a hospital TV show. Obviously there was a lot of pressure put on them from above. I am sympathetic.

Years later, a very similar experience happened with a different manager. But they handled it very differently. They called a meeting, led us through understanding the situation and us working out what the solution should be. They got our feedback on how long we thought it would take and then said they'd come by for an update in half that time. And that was it. I was so impressed.

View on HN

Something to note is that there are two important variables in the second story. The first, as you say, is that you had a different manager. The second is that you had more experience. In every situation a manager is only ever as effective as the information they have, and that comes from how good their team is at communicating with them. I'd hazard a guess that you were better at telling your manager what they needed by the time the second event happened.

If the team has good comms, and experience of both working and communicating updates to their manager regularly, and the skills to explain things in ways people understand, generally speaking you find the manager needs to check in less often. They can trust you to raise things promptly, to ask for them to do things for you, and that when you're quiet it's because you're busy rather than stuck. I find that teams who complain about their manager very often don't trust their manager and vice versa. Building up that trust by getting both sides to work on improving comms fixes a ton of problems.

> In every situation a manager is only ever as effective as the information they have, and that comes from how good their team is at communicating with them.

So much true in these words

> > Every 40-60 mins wanting to check with the ~3 engineers working on the issue for an update

> I find that teams who complain about their manager very often don't trust their manager and vice versa.

I get that sometimes it's not (only) the manager's fault if there is no trust, but if as a manager you need a status update every 40 minutes while the people are fire fighting and working on an incident, I feel like you failed as a manager.

I feel like there's a good and bad way to do this at the same interval. Of course it depends on the situation as well and what you say, but when I'm dealing with an actual emergency, I can't imagine not posting an update every hour or so. Usually much more as a way of documenting what's happening for a later PIR.

If a manager doesn't get the required details to know what's happening (I'm situations where they do and still ignore it), then the whole process could use an improvement.

From my SRE perspective, if an incident takes longer than 30min, I want my manager/coordinator to be constantly updated and handle communication with the rest of the company/customers.

I’ve been that SRE senior manager in some longer-than-ideal outages.

One of my biggest roles was chasing off the lookie-loos and fending off the VPs and CIO (usually by telling them to GTFO of the war room but then going to give them an update away from the team with what we knew and what I thought).

Usually there’s a primary and a secondary. One person handles the actual debugging and the other handles the communication and coordinating. While I agree it’s poor management to ask for an update every few minutes, it’s not because asking for an update is bad, rather because they themselves failed to communicate and train their employees to properly handle communication. Looking back, this was something we did in our team when I was just <2 years out of college. I was the secondary and posted updated to the tickets and emailed the team every 30 minutes. I was also the one calling in people to help when primary needed help. My manager didn’t bother us because he had already clearly laid out how we handle issues of each severity and who does what. It was also clear what the chain of escalation was and who and how to reach out. Incidentally, it was also my job to close the issue by identifying the cause and fixing the issue from recurring - because the secondary was only observing and communicating they get to see the bigger picture and also a welcome break for the person working hard to fix the issue.

Learnt a lot by working with a team that knew how to handle themselves. And we weren’t SRE - just regular devs. This manager - my very first one - has also been the best one I’ve ever had. I wasn’t surprised at all when he quickly scaled the ladder and is doing really well for himself now.

While I do think the manager bears the vast majority of the responsibility in situations like these, the reality is that you're right, and oftentimes managers do a bad job because they're getting pressured from above to communicate and don't know what to do.

You can often get them on your side by giving them a plan that makes them feel like they're in control (again, totally not your responsibility! But better than dealing with them being obnoxious constantly). If they ask for an update and you say, "I don't know what's wrong, but we're looking into it" then they don't know what to do. If you tell them "I don't know what's wrong, but right now we're investigating x because we suspect that's the source of the issue. I'm checking the logs and other engineer is trying to recreate the issue in a clean environment. Unfortunately I don't have a good answer as to how long it'll take, but we'll update you either when we've learned something useful or in two hours, whichever is sooner," then they have something to say to the people above them and they understand when they'll get an update, so they don't need to bother you.

A helpless manager is like a child - if left alone, they'll just shout and cause problems. What they really need is direction and structure.

> A helpless manager is like a child - if left alone, they'll just shout and cause problems. What they really need is direction and structure.

What they really need is to find a different job. Needing to "manage up" is the ultimate waste of resources IMO and puts a lot of stress on the employees, why even have a manager at that point.

While I tend to agree with you, this doesn't look like anything actionable.

I've even tried it by saying something like: "I get the impression you don't really enjoy your job" and getting a response something like "no, I hate it, I hate trying to manage people", and then suggesting the look at job ads.

Nothing has changed since.

I agree, and to make matters worse, IME managers of managers tend to be very apathetic of their reports and don't have the same kind of scrutiny of the folks under them that they expect those same people to have of their reports. So the people that shouldn't be managing others don't really have a "check" above them to confirm it, especially if they're good at deflecting blame/sacrificing their reports.

I'm a big advocate for no confidence votes from reports to signal to someone that this person needs to go.

Right, circular logic. If I have to manage my manager, can I just manage myself?

I think more often than not what they need is training - I tend to see this kind of thing in people who were promoted from IC to manager but never actually trained for the job.

A manager can definitely be helpful in this situation - a good one will be a barrier between you and whoever is above them, credibly telling those people that the best thing to do is leave you alone.

And what's their managers' excuse, then?

Strong disagree. Managing up is an increasingly necessary and valuable skill, the further in your career you progress.

I never said it's an unnecessary skill. My point is that having to do it should be a red flag for dysfunctional leadership.

That doesn't seem fair to me. An example from my job—we had a SEV 0 incident, and an _extremely_ important partners was calling our CTO directly to find out details, estimated time to resolution, etc. Being able to clearly articulate the problem at the CTO → 3rd party level in a way is a pretty important example of managing up in my opinion.

Re-read the post you have been commenting on and point out where it says that managment isn't important.

What that post said is in fact that managment can be so important that it can't be left to people who are bad at it.

Huh? Maybe your comment was aimed elsewhere in the thread; I never said management isn't important. I responded simply and directly to this comment:

> Needing to "manage up" is the ultimate waste of resources... why even have a manager at that point.

This kind of blanket refusal to engage meaningfully with one's manager and provide upward feedback creates a vicious cycle that compounds the problem.

(PoV based on 25y in the industry, spanning startups and enterprises, in IC and managerial roles incl VP.)

> I don't know what's wrong, but right now we're investigating x because we suspect that's the source of the issue. I'm checking the logs and other engineer is trying to recreate the issue in a clean environment. Unfortunately I don't have a good answer as to how long it'll take, but we'll update you either when we've learned something useful or in two hours, whichever is sooner,

Just don't get too specific, especially with even a bit technically inclined ones, because they will have ideas to help and bother tech people to check their random guesses when trying to be helpful

People in management have a wide spectrum of technical proficiency and to imply that someone in a management position could not legitimately help reeks of inexperience.

Experience varies. No doubt some can help and there are better companies but there are often those that can't as well. There's a side where all the managers / decision makers are so far removed they don't even know what the tech we use on a day to day basis is.

We must have different experiences. I’d expect someone with more experience to lean towards management not being able to legitimately help (with a technical issue).

It doesn't matter, it's not their job. They weren't hired to perform that job, they don't perform that job daily, and so they can and should leave the people who's job it is too do it.

Part of the job is to unblock their teams. Sometimes that's achieved by their own technical knowledge, sometimes by coaching, sometimes by coordination with other teams/resources. It's very situational.

I obviously did not written that to encompass every single manager in the world

> and to imply that someone in a management position could not legitimately help reeks of inexperience.

You reek illiteracy at best, purposeful misunderstanding to have a fight at worst

30+ years of experience here. 8+ years of successful management experience:

9 out of 10 managers I have ever worked with were incompetent.

are you saying the employee should propose a plan of action to control their boss?

I'm reading your post and it sounds like you're trying to heard cattle

Obviously yes? You know what needs to be done to fix the issue, you simply need to communicate upwards that it’s in check. A certain amount of herding is to be expected.

> You can often get them on your side by giving them a plan that makes them feel like they're in control

Only if they hear you. Had one that's too focused on the pressure from above that my advice is being totally ignored.

that comes from how good their team is at communicating with them

Communication is a two-way street. Proactively calling a meeting with all those involved to understand the situation and then letting the team get on with their work is 100% a managerial skill, not a function of the seniority of the engineers.

My knee-jerk reaction is to offer a riposte of additional data demonstrating it really was about the managers. But I think you've got a valuable and generalizable point here. I'm going to reflect a bit on that.

You know what's difficult? Looking back at a younger self and being truly critical. Gosh, it's so easy to just say, "yeah well his employment future at that company proves my point." But then I learn nothing else that I can grow from.

Having that knee-jerk reaction in the first place is something you want to lose ASAP.

> I find that teams who complain about their manager very often don't trust their manager and vice versa. Building up that trust by getting both sides to work on improving comms fixes a ton of problems.

Sometimes it's beyond that. The problem can be layers up where neither the manager nor the rest of the team can solve. The team wants changes / improvements but the manager is powerless or forced to act in a certain way.

When I interview for a job I always ask my prospective manager about the is. “What’s your process for getting status updates during a crisis”. This question is graded pass/fail.

Examples of passing and failing responses?

Can you share some of the good/bad responses and was your inference correct?

Not having a process = fail.

I am sure you know this, but be careful with this strategy. The worst managers of all will tend to give you the answers you most want to hear in these situations.

The best will tell you the truth. This might be the same in the best case but is often less than perfect in reality (though, per being a good manager, can at least improve).

> The worst managers of all will tend to give you the answers you most want to hear

i don't think the worst managers even know what the right answer is.

They might repeat something that they heard.

>The worst managers of all will tend to give you the answers you most want to hear in these situations.

How is that different than employees giving the answer employers want to hear in the interview just to get the job? Do you think everyone is 100% honest? Everyone puts on a face.

I've worked in >1 companies where the SLA requires hourly updates for incidents, and they have to have substance beyond "still fixing."

This is in large scale finance and/or investment sectors where it is immeasurably important to keep clients (both "those paying us" but also the "those integrating with us" types) well informed of progress so as not to cause market panic.

Sometimes communicating is worth it for interrupting the team.

There's communication by interruption and there's observation. If you have a situation where there need to be hourly updates, there should be a person explicitly tasked with observing and noting what's going on to be able to provide those updates with minimal impact to the group that's actually working the problem.

Unfortunately, that means the observer has to be able to understand and follow what the engineers are doing while staying out of the way as much as possible. Tricky, but something you really need if for no other reason than to communicate and look for any holes that the team/person working the problem may not have thought of in the interim.

Yes, this role is defined as the "journalist" at my company and is an engineer tasked with writing status updates and the post-mortem report. It is important to have this role assigned from the start so that expectations are clear!

If that is the case you should pair program, where one of the pair's only job is to sit behind the engineer doing the job and listen in, then take the meetings. This needs to be a full engineer who is fully capable of doing the work, but instead is just understanding the work, but not doing any.

If the main engineer has a 'family emergency' your communication engineer has enough context to take over, while another engineer is assigned to communication.

Agreed. That was how I've seen it run well. eg we'd create 2 channels for the incident - the first was for the technical team trying to fix it and would have quite a bit of noise as debugging went on. The second was for more senior managers and the client/account people to coordinate comms with customers. The only manager in the first group was the teams direct technical manager or their eg Tech Lead type of role. Their job was to not work on the fix but to communicate what they were observing to the 2nd group who would deal with the customers.

Smaller companies though are a bit less structured, and have to wing it a bit more depending on circumstances.

Assigning a "communications officer" at the start of an incident seems to work well. They are a person with the same technical skills as the people actively working on the problem, but their job is primarily to observe and filter/transform communications to and from the team. A nice side benefit is that having someone who isn't actually responsible for fixing things seems to free up their mind to notice things that their hands-on-keyboards colleagues missed.

Clearly you need non-technical manager to communicate status to the customers. You know, because they have people skills

I knew this conversation was going to eventually end up in an Office Space reference...

Sometimes communicating is worth it for interrupting the team

Perhaps, but the first thing any emergency responder will tell you is that there should be one incident commander, and that all communication and allocation of resources should go through that one person. And that the person with the incident commander role should not carry out any operational duties besides coordinate and control, because those other tasks require the same mental faculties, but use them in a different way, making them less effective in either.

If there is a requirement to update customers affected by an incident, the company should have processes in place to streamline the flow of information without interrupting the team. And the team responsible for updating customers should only communicate with the incident commander, and absolutely steer clear of the operational responders. If specific details are required, it falls to the incident commander to gather those details from the team, not to outside people running interference.

The key component in this system that understands human psychology is to set reasonable expectations. Most places don't have that kind of SLA, but when an event occurs you can set an expectation of when the next communication will be and as long as it's not crazy, people will go along with it. But if you just leave people hanging, not knowing when/if an update is coming, they have nothing to do but panic about the lack of new information.

> Sometimes communicating is worth it for interrupting the team.

If you need constant updates from someone who knows the subject material, you need to nominate a single person who knows the material to take point on communication. They can help in between communication updates, but they can’t be tasked with leading or owning the work.

The worst way to do it is to have a manager who can’t understand the subject material themselves, then let them interrupt the entire team over and over again. Then the team has to stop and educate the manager about what needs to be communicated so they can relay it to someone else.

Having non-technical managers relaying information back and forth from the entire team is always a mistake.

Scheduled communication is far easier to handle than random interruptions.

If your target is to get update every hour you just dedicate a person during incident to communicate with the "outside" and make rest of the team post the updates on shared chat, which you want anyway if you don't want people to get on eachother's toes

I've seen this distinction and it always boils down to two factors: is the manager technical and what's the caliber of the engineers.

Technical managers typically have a pretty good idea of what the rough requirements for a given situation will be. They might not be familiar with the exact details but should have some clue, so they just want to make sure everyone has a plan and it's sound. Non-technical managers don't understand what's going on so need to micro-manage because they perceive everything as high risk and complicated.

Caliber of engineers is pretty self-explanatory, but maybe that's selection bias; a 10x engineer won't stay long where he's micro-managed.

definitely this ....

non-technical managers have literally no tools at their disposal other than to bug other people to do things. in a crisis, therefore, all they can do is dial up how often and how much they bug everyone else to do the actual work. At best they act as communication hub, gathering crucial information and coordinating to ensure it gets to the right people as fast as possible and they act on it in the right way. But even in that role they are limited because they don't have enough insight to really know who needs to know what and what they can do with the information.

I had a manager at my last job who tried to do the "what's the update" during a crisis and I just decided to make fun of her saying "I solved it but decided it would be funnier if didn't tell anyone" and she stopped

Crisis communication is an specific sub problem of team communication. Good takes on this often come from experienced disaster responders, like a fire chief or FEMA.

Their version of this is very real: 1) the annoying manager bugging for updates is the mayor's office or other powerful executive branch, 2) the emergency from which people are being distracted is life or death, 3) a jumpy executive can give wrong messages to the public which increase the workload or danger level.

Saw a lecture about this once, takeaways were 'clear lines of communication' and 'clear chain of command'. Wherever possible, put a person or a process between non-operational managers and front-line responders.

All the good managers I've ever encountered have the same skill:

They ask smart questions.

====

At the beginning, they try to understand the situation by asking the very dumb questions - but they're not shy about it.

They'll say, "Let me pick your brain", "Sorry I'm an idiot, so let me ask this dumb question" - and so on.

No ego.

But in no time, their questions will become smarter.

Coupled with their big picture view / perspective, their understanding of the system and the business - they quickly became a very helpful assistant for us to understand the problem, and in many cases, even able to suggest the, best, solution.

=====

Bad managers gives no added value, and instead distract you from your job.

Any senior management should understand these, and should be able to pick the good ones.

Otherwise the company/institution will be mired with persistent problems in no time.

God I've been feeling this lately. Manager turned into full micromanage mode. I started regretting waking up and checking slack, because there'd be an "Any updates on X?" message sitting there, often coupled with some passive aggressive remark. When I feel like I have more autonomy, I intuitively feel the urgency and act appropriately, updating people as updates happen.

Had something similar. Deadline to go live was Monday at noon. We didn’t even have a working proof of concept on Friday.

Has multiple managers at my desk every 30 minutes on Monday.

At 11:58 they announced they were calling client to go live with them. Product was not working.

Manager. Hey they say it’s not working.

Me a few final key stokes and a save. (On prod server) Try now!

Client says it’s working now!

Went to the ER. Thought I was having a heart attack.

Never again.

I had some emergency a while ago where they insisted on being on a call with 10+ people while it was resolved. After an hour of badgering and applying fixes that all those helpful people thought up I told them I couldn’t focus, went off the call, and fixed the damn thing in 15 minutes.

Being on a call with people trying to be helpful is extremely aggravating.

You gained seniority. I have employees who don’t take a support case with a customer waiting with the straightest path, instead they take the opportunity to refactor code, so I check on them all the time; Others who refactor the code after delivering the bugfix, and those I leave them alone.

This is a result of always having the refactoring put on hold for feature development. Eventually engineers realize the only way to address tech debt is to refactor as part of a feature or bug fix.

This doesn’t happen when developers have trust they’ll be allowed to refactor although sometimes it’s a case of learned behavior from previous employment.

> This doesn’t happen when developers have trust they’ll be allowed to refactor although sometimes it’s a case of learned behavior from previous employment.

To my surprise, this is not always true. Some engineers do things because they have fun, even senior ones, and they will keep on refactoring things until there is a hard stop.

Best is to never generalize. It may depend from team to team, etc.

Yeah, everything is situational. Someone can be a very competent developer, even an congenial coworker, and still be a terrible employee.

If you don't let them do this from time to time they will quit eventually, IME as a manager. Whether that's beneficial to the org or not is debatable. You can kind of "release the pressure" in a planned manner via hackathons, but those almost always never have the appropriate frequency.

The prioritization of refactoring is strongly context dependent. A small startup with a fast moving product trying to acquire and retain customers cares much more about serving said customer right now than an established business who has more freedom and engineers available to dedicate time to forward thinking projects like refactoring.

I've worked at both large companies and small startups, and I've never seen taking on tech debt pay off. Features never get delivered as quickly as anyone wants, but sometimes tech debt completely prevents even thinking about features, and not being able to even imagine implementing something is a much worse problem than being 2 extra weeks late to the newest pivot. If I were starting a software company today, I'd want to keep my POC code about as clean as I'd keep production code at a large company. Well, cleaner. It's going to be alive as long as you are, and there will never ever be a good time to rewrite it.

Having said that, I am very skeptical about what some people identify as tech debt. Things like "we're using X logger instead of Y logger, this codebase is unmaintable" I tend to not prioritize, but things like "this system looks weird and we don't understand if the code is wrong or the documentation is wrong" is something that should be addressed with high urgency. It's very easy to spend a lot of time maintaining misbehavior that should just be fixed, and the first step is understanding whether or not something is wrong.

When my coworker did it, it was the result of zero self discipline or focus, so he straight up got distracted by thinking of a way he could rewrite the whole app instead of making the two line *already understood* fix. He was a very good programmer and a terrible engineer and terrible coworker.

I do that on non-time-critical cases, if it doesn't matter whether I deliver it within an hour or day better to just do it now when I have it fresh in my mind.

But anything longer gets thrown to backlog

I think this is an interesting comment because it seems to try to re-locate agency back onto the manager. Ie. It’s not that the other manager was better, it’s that I somehow earned the better management. Correct me if I’m misinterpreting you.

It seems like a pretty standard (and simplified for HN) classification of employees who are and aren’t able to prioritize well.

Employees who can prioritize well will naturally be given more independence to make choices than those who don’t.

This of course assumes the management is able to make good decisions about prioritizing but I think the commenter should be given benefit of the doubt here.

This is an interesting point and may be part of the factor. I.e. There are two variables, and chances are that playing with either affects the outcome - experience and self sufficiency of the hands on person, and experience and self sufficiency of the manager (plus, I'll argue, other variables like maturity of processes, and client relationship / management state, etc). There is an inclination to focus on one variable only.

Fwiw. I've been a hands on techie for 20 years, then ops manager for 5. I've certainly been an Inexperienced/been a bad manager at times (especially as for a while, highly functional specialized vols reported to me as well), and I'd like think I'm becoming a decent or even good manager. But even still, there are definitely teams and individuals that I will feel the need to manage and focus ; vs others that I feel I can let prioritize and self manage.

(In the ops, of course, establishing formal urgency levels and resulting comms expectations and paths, can reduce stress on EVERYbody)

How do you coach the former to behave more like the latter?

I like to practice incident response on a regular basis, so people, including managers, know what to expect out of the process. If you do a practice incident every week, a real emergency will feel routine and people will be less anxious. If people are really distracting on the incident response call, you can always kick them off and elect someone to keep them updated, of course. I've never had to kick anyone off a call, though.

I was introduced to Pagerduty's incident response guide and like it a lot: https://response.pagerduty.com/

As a VP I had some conflicts with subordinate managers who were super anxious and transferred their anxiety on their own teams. Basically they had no clue how to solve problems due to not being technical and as a consequence made their whole teams neurotic like they were. I typically had to take over for a while to cool down the team and shield them from their managers whereas managers were supposed to shield the team from me.

In that scenario, it's often because the manager has someone breathing down his neck too, so you have bad communication and high stress all up and down the org chart. But sometimes the manager is the source of his own stress and all you can do is manage that.

Hah, when an emergency is happening only one person asking for hourly status updates would be a goddamned reprieve.

When stuff hits the fan I expect to be getting constant questions from the business analyst, the helpdesk, and manager, as well as having to explain to every other person who hasn't heard that I'm busy that no I can't help them with that today.

Yep, a manager makes or breaks role. Good manager manages up and acts as a bulkhead against pressure coming from above.

Reply on news.ycombinator.com