Hacker News Reader

Disclaimer: I have been employed by Google and my views are my own.

I can also see the value for a large organization in culturally enforcing interchangeability and replaceability of workers.

I always considered this bit of culture (and felt the message this way from coworkers) to be even more focused on on well-being of the would-be hero. It always comes alongside messages about how to deal with burnout or achieve a work-life balance that works for you. That's always been my preferred perspective on it, anyway.

Still, I'm with you in pushing back against the interpretation you quoted, which is at best a weak (and at worst toxically macho) understanding of real motivation behind this anti-"hero" sentiment. It's mildly absurd to say there's a culture of holding back hard workers in order to lighten the expectations for everyone else. Anyone who works "twice as hard as expected of them", broadly speaking, just gets recognized/rewarded/promoted until expectations and output match.

Edit: reorganized order of thoughts

The reason I'm skeptical of heroism is that it hides inefficiencies from view (which often exacerbates them), by papering over them with hard work. I have written about it in the past: https://two-wrongs.com/hidden-cost-of-heroics.html

Amen to this. Years ago I did a contract for a company that was very pro-heroism. Every single promotion email I saw had some anecdote about how the person had, say, stayed at the office multiple nights in a row to save an in-trouble project.

You know what? Every project needed heroics just to get done on time because the place was profoundly fucked up. And each round of heroics left the code in poorer and poorer shape, demanding more heroics next time. Instead of doing a truly brave thing, like telling executives that a schedule was absurd and the team needed more time to do it right, people did the cowardly thing and said, "three bags full sir" to every absurd request.

And that was the right choice for promotion, because everybody higher up had succeeded by heroics.

Yea but then you also disincentivize the one guy who could potentially make an outsized contribution to the innovation of the company.

What you’re saying is a virtue in peacetime and a sin in wartime. Precisely the argument the author is trying to make

Google doesn't disincentivize any of that, they just don't encourage it. Google has mechanisms for rewarding people for doing extraordinary work: peer bonuses, spot bonuses. People who make outsized contributions are compensated, and people who consistently contribute more than others are promoted. But systems that require heroic efforts to continue functioning are fundamentally broken and unsustainable, and need to be made easier to maintain.

I left Google recently, partly over the fact that they strongly disincentivize taking technical risks.

Let's just level set for a second:

* Peer bonuses are a fixed $175 and spot bonuses are usually under $1000, and pretty much always under $5000.

* The coveted "feats of engineering award" (or whatever they changed the name to this time), which has the highest prestige factor of any of the internal awards, is $2000 cash or a trip somewhere.

* A senior SWE at Google makes about $300k. A Staff SWE makes $400-500k. An entry-level SWE makes $150-200k.

What this means is that the only REAL reward on offer (other than a token amount of cash and a serotonin-inducing certificate) is a promotion. Promotions almost always come when you launch products (actually, they changed the benchmark to "landing" products, which seems to always happen about 6-12 months after launch after the bugs are worked out).

Launches at Google are almost always significant efforts - either in terms of engineering around the company's arcane systems that are changing underneath you or in terms of getting through all of the process checks that ensure your service is "googly." Further, to go from a "launch" to a "landing" is mostly a political process: you have to convince enough of your management team that you have met some arbitrary milestone that indicates a stable product. As such, people are not too happy to take significant technical risks if they involve spending more than a few months on a project - if you want to get promoted, wasting time on something that might fail is bad. Why would you want to "land" a risky product when you can "land" something that is already on the roadmap?

Additionally, Google offers pretty much no promotion credit to engineers for "prospecting" - what I will call the process of identifying products that should go on the roadmap. That means that if you go out and do this sort of work, you are actively hindering your career: if you go prospecting, you are going to spend months not "landing" things. Conversely, prospecting seems to be a very good way to get peer and spot bonuses.

This is why Google provides no real benefits for taking technical risks. The only real reward is promotion, and promotions come mostly from following the roadmap, not from rocking the boat. The other awards and bonuses are basically toys by comparison.

Maybe there could be a Prospecting career ladder?

And maybe well paid titles, but where one couldn't hire more people, one just got paid better? To mitigate over hiring?

**

Or what are your other thoughts, about how to solve the problems you noticed? ... In addition to:

> promotion credit to engineers for "prospecting"

I don't disagree with your broader point, but I'll make a small nit that I don't think peer/spot bonuses are a meaningful-enough portion of TC that they matter from a financial perspective (unless the numbers have changed significantly over the years). The 'kudos' part of it was more meaningful to me.

People just want their work to mean something, to be contributing to success. Heroism is the full commitment to that. Of course it’s riskier, and based on people over processes, but again and again it’s the inspired that build the future. Work-life balance needn’t suffer, work just needs to be more than a job.

> Heroism is the full commitment to that.

Definitely not. Heroism is nearly the opposite, in that it wants heroes to be significant, not the work.

Heroism at a corporate workplace is, first and foremost, self serving and egocentric.

I don’t think Big Tech needs to overtax its engineers to deliver innovation. They can afford to hire enough folks to keep the innovation engine humming without requiring heroics from ICs.

I understand there are some exceptions: sometimes during outages/emergencies, you do want folks to fix issues ASAP regardless of how long they are up.

You can just barely story point, Gantt chart, and JIRA ticket your way through routine CRUD development. I don’t think too much innovation is happening that way. Innovation in my experience comes from “off the books” deep dives by individuals, pairs, and occasionally groups as big as 3-5. It comes from going into territory that is not well defined, doesn’t have predictable timelines, and is otherwise anathema to formal BigCo project management and communication structures.

Absolutely, but you can do all that without heroics.

Work done outside the formal assignment and tracking processes is normally what I would consider “heroics.”

I don’t think innovation hums, it bursts.

Mine itches.

The opera singer shatters the wine glass with a humming

Being a genius isn't being a hero. We need geniuses to create, not heroes to paper over our flaws. If you are busy being a hero, you don't have time to be a genius. And what happens when Superman takes a sick day?

> What you’re saying is a virtue in peacetime and a sin in wartime.

interesting! it's precisely the point made at the end of The Caine Mutiny.

https://www.youtube.com/watch?v=Jw6gwGawbXA

The problem is that Heroism is subjective.

as the old saying goes: "fix a production stopping bug after 30 seconds, you're an arsehole, fix it after 10 hours, a hero"

The problem with heroes is that they tend to be single people with a specialised and bottle-neck like ability. They step up because of a systematic failure.

If those heroe-likes are needed more often than in the odd Sev 1-2 bug fixing, I always consider it as a red flag. As you said, they are needed for a reason, and that reason, if not some odd special cause, is more often than not systematic.

This.

Problems that actually require such efforts should be exceedingly rare. If they're not, then there's a deeper issue in the production process that desperately needs fixing.

You don't want developers (or anybody) to be working that hard on a regular basis. Burnout is real, and in the long run the practice will cost the developers and the company dearly.

>If they're not, then there's a deeper issue in the production process that desperately needs fixing.

Five Whys.

https://en.m.wikipedia.org/wiki/Five_whys#:~:text=Five%20why....

I broke something the other day. Before I could even catch up on the messages they found my culprit commit. Oops. But also...yay. Easy rollback. Did not need me at all to figure out what buggery I had done.

>systematic

systemic?

who says that old saying?

The saying I know is "fix an outage => hero; prevent an outage => what are we even paying you for?

That's the DBA and sys admin's lament. If I do my job properly, no one knows I exist. If I let Rome burn and then later save the day, I am a hero.

Keeping the lights on and innovating are two very distinct workflows.

Heroes are a bottleneck in KTLO, but they are a necessity for innovation.

It's not a coincidence Google no longer innovates. They threw the baby out with the bathwater.

This got a lot more comments than I expected. I highly recommend reading https://sre.google/sre-book/eliminating-toil/ (also referenced in a nested reply below) to understand a bit more about the toil-specific aspect of this I was describing.

I think your clarification is important, but also within the context that he painted, it doesn't seem like this is the type of "heroism" that is "staying up all night fixing the bugs", it's heroism to take risk with your product:

> There are documents that explicitly and proudly deride “heroism” and assert that not only should product teams not encourage “heroes”, they should actively dissuade them. If someone chooses to work twice as hard as is expected of them, they usually will be prevented from doing so because they have to work with others and doing so would force the others to work harder too. If someone says they can finish a project in a month, their manager will tell them to be realistic, pad it to four months, and tell the VP it is six months.

To me, that doesn't sound like avoiding all-nighters by lone wolfs so that everyone can support the system when it fails, it seems to me like it's preventing people from giving their all to create valuable products.

^this. I think it belies some of the things I don't agree with in this take. Overall I thought it was good, but I think his is a very old sentiment that leads to more worker burnout which is bad.

I read hero as someone who works hard(er), not the one person keeping things up. I mean, the one person is a hero, but heroes can just be the one doing most of the work. I have a couple of heroes at my current job and they're the ones working 12+ hours a day sometimes weekend to deliver on a NEW project. So it isn't much about bus factor, but trying to deliver even if it means sacrificing personal life.

The "heroism bad" sentiment OP's article is describing within Google's culture is specifically the "one person keeping things up" variety, and the characterization in the article is inaccurate.

https://sre.google/sre-book/eliminating-toil/ is a good public example of this sentiment.

I don't think a hero is someone who works harder. A hero is someone who resolves a critical issue that nobody else could resolve. How many hours a day they work doesn't enter into it.