Engineering Principles and Values
engineering.zenpayroll.com
engineering.zenpayroll.com
I'd caution against the term "over-engineering." It is a straw man[1] — no one is pro-over-engineering — so it is only ever used to discourage long-term thought. It's a good thing to try and avoid, but maybe pick something neutral like "predicting the future."
Hacking, on the other hand, is sometimes advocated for (e.g. Facebook) and can be good or bad depending on circumstances.
I've had to argue against over-engineering many times in my career. I wasn't about discouraging thinking. I was fighting against actually implementing specific features that attempted to solve poorly-defined problems whose details had not yet come into focus. Developing those features would significantly impair other parts of the project.
For example, the problem might be "What if our database becomes overloaded one day?" Choosing to implement, say, sharding, could lead to an over-engineered system which gets in the way of solving the many different future performance problems that could crop up.
I think it's quite reasonable to use the term "over-engineering" in this sort of situation.
In your example, if you tell the other person that their database sharding is over-engineering, they can say one of two things:
a) No it isn't. b) Yes it is and I think over-engineering is a good thing.
I have never heard someone say (b); if they thought it was over-engineering then they wouldn't have advocated for it. The negative quality is part of the definition.
What you're likely disagreeing on is how far in the future you're team should be thinking or how much data is necessary to make a confident decision.
I don't think it was your intent to stop long-term thinking, but if someone says "we shouldn't over-engineering things", I don't think that solves any ambiguity because no one was intentionally over-engineering before.
It's akin to saying "don't do bad things." People never do bad things, only things that are bad under a different set of principles or based on other values.
My feedback is that this document should define what over-engineering means in their context (since it, like what is "bad", is context-specific) and lay out what engineers should look out for.
Absolutely, and that's why I agree that the term Overengineering is an unhelpful shorthand that implies criticism to an unnecessary extent.
Stating that you are against over-engineering makes it explicit and as a value it is something you expect your engineers to think about. "No over-engineering" should sit at the heart of any good set of coding practices and it should be there because it means engineers have to think about and understand what it means.
"there are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies; the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." - Tony Hoare
I completely agree that over engineering that takes the form of "this feature will be cool, lets build it" without validation or demonstrated need from the market is generally a bad idea.
However taking time to understand the fundamentals of a system and build a robust core on strong principles is almost always a good idea.
Not exactly.
The term is used to discourage long-term thought that almost certainly won't pay off. As such, it usually will cause more trouble in the long term. So it's not just a failed investment with a missing long-term win, but an "investment" that will cause a long-term loss.
Over-engineering usually happens when people either over-estimate their ability to predict the future. Or, when they just learned a bunch of exciting technologies or patterns, and drift into some kind of hammer/nail thinking.
It is when you hand them a blank check and allow more engineering ot justify writing down a bigger number.
Especially in engineering organizations, a consistent problem is moving people into management positions based on their skill as an individual contributor or tenure. Both are orthogonal to people-management skills and, though good to have from a team respect standpoint, far less useful.
Is it a priority of yours to avoid those problems? If so, it would be great to see it listed out. How do you organize teams to be optimized for Employees?
Especially as you grow, I would imagine managing people towards their goals and those of the company is going to be a larger and larger part of that employee happiness.
edit: There is also nothing on employee growth. If that is a something you value, it would be cool to see it in there.
I've heard some larger companies "solve" this by providing some title advancement past "3rd Senior Engineer from the left". But is it satisfying to you folks on these tracks? When you go from Senior Software Engineer to Staff Software Engineer to Super Staff Software Engineer to Super Duper Staff Software Engineer, do you consider it to be real career growth? Do you find that this is truly a _parallel_ path with management, or does it eventually plateau (i.e. the management path ultimately ends up reaching farther)?
Each of those roles also came with leadership expectations: Architects were responsible for the overall technical health of the systems they were responsible for, with Sr. Architects being responsible for multiple. They were empowered to reject system changes and were expected to review code and designs on a regular basis. Really good engineers that weren't system thinkers or didn't want responsibility for other people's code, could increase in compensation but weren't considered part of the leadership team.
If you're a dev you're more likely to have to bow to pressure from marketing or other departments and won't have any real impact.
What I would like to see is a working scheme for more fluid resource allocation scheme than fixed tree-like structure. This holacracy thing promises exactly that but I am not sure if it is really viable.
That said, it makes sense for a company in which individual contributions can revolutionize the business.
There are cases where being an individual contributor is only so valuable (though that "so valuable" number may be very high.) There comes a point where there is only so much impact you can have on your own, which is why _great_ managers absolutely deserve higher compensation.
It's hard to individually make a larger contribution than a manager making every one of their 5 teammates 10% better.
So to answer your question, I believe it plateaus. Where it plateaus is relative to how much individuals can impact the course of the business.
The Tech Fellow track was created to grant executive level compensation to retain subject matter experts. Beyond a graduate degree, becoming a Tech Fellow requires demonstrating your subject matter expertise within the company, with a key measure being that other groups seek out your help, so it does require some politics and networking. At the highest level, you pretty much have to be a national authority. A lot of work, but executive compensation at a megacorp is significant.
That said, unless your specialty is avionics or CFD, I don't recommend aerospace companies for software engineers.
Above senior you are allowed and expected to go "heads down" (by yourself or a small group) and knock out solutions to technical problems. However you are also expected to identify cross-cutting problems, rally the right people, coherently frame your arguments, etc to move the "implementing solutions to technical problems" needle (even if you don't directly write a lot of that code).
Management also get's the "fun" things like "let's talk salary!" or "you need to improve along X, Y, Z axis!" :P
Just my two cents :)
What you said about those cross-cutting problems and all that is spot on and that is basically what I see sr. engineers and architects. However, how do we or should we even further delineate career progression and responsibilities? And what does that look like? What is beyond those things?
But I've never been in the position to either be promoted or do the promoting to that level :)
Senior S/W Engineer -> Tech Lead/Specialist -> Architect -> Senior Architect -> CTO/VP-Technology
> It’s a framework that we can turn to whenever we’re faced with a difficult decision, whether it be about how to best architect something, how to prioritize our time, who to hire, or how we interact with each other.
Why would you then leave a growing priority vague and up to interpretation? Isn't that exactly the type of miscommunication and lack of guidance you're trying to avoid?
P.S. I do think the exercise is great and don't want to sound unnecessarily critical. Especially for such an important part of companies (people) and common issue in engineering organizations (management), though, it seems like one of the first places you should put a stake in the ground in terms of action, guidance, and specific commitments.
I stopped asking because the answers were always some variant on "that encourages programmers to be lazy, because they know QA will be there to catch it" and because it seemed like lot of companies treated that question as a culture fit red flag. The responses were pretty hostile, like I'd asked why none of the engineers were wearing suits.
I have no idea whether it's a good idea, but the current tech culture seems pretty opposed to it.
As a new manager, he was who I leaned on the most to find out things, and who saved my ass on a number of occasions.
He deserved more respect than he got.
If the QA person has skills along the lines of a software test engineer (like at Google and other places) and can actually create tools and automate things and write some unit tests or acceptance tests, then they're half-a-programmer and I'll be happy to rely on them and share the work. Otherwise I assume the QA is there just to bow to management/marketing and give the OK for deploys.
It's also worth mentioning that when coming up with our values/principles, we started with the understanding they can and probably will change over time, especially as we grow. So while we don't have dedicated QA team today, we'll keep an open mind do whatever works best for us in the future.
That phrasing is a bit different than what I think they meant. Programmers familiar with the problem domain should QA the code of other programmers is how I took it.
I've never seen the value in "pure" QA people largely because you need to be a developer to be an effective QA person [writing automated, repeatable tests] at which point you are just looking to use the developers that are more meticulous than the others to write those tests.
They are still ultimately software developers / programmers / whatever. Its just the problem domain they spend half their time on is QA instead of Ops or whatever else.
Maybe I'm just used to being part of an IT department embedded in a larger company where we tend to mix a couple things in a single position. :P
But these are engineering principals, not company principals. And "employees" is last, not first. Nor is it particularly clear in what sense employees matter. Saying they matter does not describe in what way, it's a blanket statement. Do they matter that you will pay them at or above market rate? that you will help them with training and certifications? that you'll take care of them when times get tough? You have to spell it out. Really spell it out. Otherwise, it's too ambiguous and it'll get swept under the rug.
each page is 75% art, 25% text. I think this ratio needs to be flipped.
Still, this is something. I don't think it's "done" but it's more than most have.
It's easy to ramble on for 100 pages, but capturing the essence of your values in a few slides with a couple of tidy examples each is really hard.
Maybe this is good for a TL;DR, but if you don't spell out the details you leave room for ambiguity. Details help everyone think the same and gives them all the same reference points. They all have examples they can pull from already built in, so they are more likely to make the preferred decision.
It's easy to hide behind "We", and easy to say "we isn't me". I might suggest something like "I own everything I build and everything my teammates build". That eliminates a whole class of avoidance and excuses for bad code.
Overall it's a nice distillation of general corporate engineering teamspeak but I didn't grasp anything new or enlightening. This document would be pretty easy to ignore if I were in that group.
I guess "we" really means "you"
[1]:https://en.wikipedia.org/wiki/Marx's_theory_of_alienation
From Founders At Work:
"I'm organizing this company like McDonald's. Each restaurant is going to be managed by a few people, and they're going to have profit-and-loss responsibility. If they make a profit, they get to pocket half of it. If they make a loss, we're going to know who's responsible, and we're going to go there and fix it, and there are going to be consequences for those people." People have all the right incentives to make their customer happy, to do the thing on time, to take the customer's money, deposit in the bank, and then move on to the next one and get their bonus at the end of the year.
And apparently at one time Accenture had this philosophy too:
At the time Anderson Consulting (now Accenture) didn't have any salespeople. They always had the people who were executing the project sell it. "You eat what you kill" was the phrase at Accenture. You don't have a salesperson go out and tell the customer, "We can do this", making promises and then handing it off to a programmer"
Can you imagine if a Sales team came up with a set of values and part of that document specified how engineers should do their jobs? There would be revolt!
I think what they meant was that when you code something, you have to treat your work like you completely own it so you do a good job but to me that's self explanatory and is only repeated to enforce rules that are obvious.
Repeating the "obvious" is very important. There is a constant fight against entropy, miscommunication and attrition of experience in any team, and things like these add up in a huge way. The more years I spend in the field, the more I realize the need to "over-compensate" affirming and syncing up on stuff like this.
And aggregating stuff like this into a list is tougher than it seems. This is one of the most balanced values summaries I've seen (for my subjective taste!) - not being too vague, but also not "hard-coding" specific methodologies or buzzwords.
How do you handle situations where there's a lack or breach of trust? If an engineer comes to you saying "After $INCIDENT, I find it hard to trust $COWORKER", what do you do?
Trust is very important & I'm glad to see you've given it an entire slide. I particularly like "There’s little benefit to setting hard deadlines if you know everyone is doing their best".
There's no magical solution to fix it. I think the best thing to do is acknowledge it to the person you've violated and tell them you're going to do your best to earn their trust back over time (and actually do it).
its an essay in slide deck form. why make it a deck? just write it as an essay.
I know that's too high level of a description. but what I'm saying is... that you always pay the price for poor quality, rushed work, whether you pay that price in an hour, tomorrow, next week, you always do, somebody does. so better to get quality right.
however, I see lots of astronaut architecture and too-much-abstraction -- especially in all-Java/enterprise shops -- where, I see folks spending lots of time on things which objectively don't have any business value and don't objectively increase quality (reduced failure rate, reduced bug rate, etc.) instead arguably seem to increase it, or increase the complexity and quantity of moving-parts which can fail or otherwise bite people later. Quality good, quantity bad.
Simple-but-perfect is better than bigger-but-shoddy.
Good engineers are always focused on delivering value to the customer, and good engineers will always select stable systems rather than be swayed in some kind of hormonal feature porn frenzy.
On top of that, in my experience, humility is not something you have to ask for in an engineer. He or she will be painfully aware of every mistake they make, it's the only thing driving their development.
tl;dr if the head of accounting thought you were a rowdy bunch of code monkeys, then read the agile principles you'd end up with this