10x or not: You’ve got to do things right
devup.co
devup.co
Perhaps this is a true statement; however, I have no doubt that 10x programmers do exist who do not build technical debt into the application. Some people really are that good. I don't think the average person realizes exactly how good the best in the world are at what they do. For instance, I ran track for 8 years and competed in college. You don't realize exactly how insane a 12:37 5K is until you've trained as hard as you can for years and you're not even close to that time. Nor do you realize how difficult a perfect rendition of Liszt's transcendental etudes are until you've spent years trying to learn them on your own. I imagine the best programmers in the world are similar, but it would take the average programmer decades of programming before the magnitude of the skill gap becomes evident.
If a 5K of 12:57 was really "10X", an average runner would take just over two hours. That performance is nothing like 10X.
How much faster would John Carmack produce a new graphics engine than a recent bootcamp grad? How much better would Carmack's be?
I know I've gone to unfair extremes with the question, but it still illustrates the point. Really excellent developers are waaay more productive. The upside is huge. All the time saved avoiding blind alleys based on experience, all the cases where experienced intuition identifies the cause of a bug that would have taken much longer, etc.
To carry the analogy forward, a 10x runner might take a shortcut, use a better implement to complete the course, etc.
Physical capabilities of organisms (e.g., strength, speed, jumping height, etc.) tend to fall into normal distributions.
Knowledge work where you can take advantage of powerful (mental | software) abstractions don't tend to fall into normal distributions, as previous work can make a bottleneck disappear, leaving you with more mental bandwidth for other concerns, which quickly expands your capabilities.
I'm talking about productivity in knowledge work which is different.
Now in chess, as in programming, there are clearly some individuals who excel by orders of magnitude over beginners and even average competent players. But because of the vast scope of possibility, programming "grandmasters" will do things that no one else could have even thought of in one domain, while in unfamiliar domains they might require a good deal of study before they surpass the average journeyman practitioner.
Meh, analogies suck.
At Microsoft during its heyday in the 90s, it was common for people to say things like, "That's the dumbest fucking idea I've ever heard" and the behavior came straight from the top. Similarly, Steve Jobs and Jeff Bezos are well known for humiliating engineers in front of an entire room after a presentation.
http://www.businessinsider.com/things-amazons-jeff-bezos-tel...
There's no way to prove the issue definitively either way, but I suspect this blog post suffers from the cognitive bias of confusing how the world should be with how it is. At the very least, a "no shaming" rule isn't a requirement for building a world-class engineering organization and attracting top talent.
If an engineer feels they can admit a mistake early then the rest of the team can help to fix the issue before it escalates to the point where you lose a client. If an engineer hides their mistakes because they're afraid of how their boss might react, things are left lingering, and they don't get fixed. That's how really big problems happen.
Another data point is the vastly greater acceptance of giving workers a tongue-lashing in China vs the US. In that case as well, the harsher culture has certainly been growing faster in terms of engineering-driven companies. In fact, I've never seen a US company of comparable size that can move as fast as a Chinese tech company. Part of this is due to differences in IP and compliance laws but I believe culture also plays a role.
To be clear, I don't like that employers treat their staff harshly. That doesn't necessarily mean the practice is less effective, though.
Don't get me wrong, I've raised my voice at people but only after they've persisted in a particular error for months, after receiving many many external reference to why it's an error.
I've no interest in working for a Bezos-type because I see that as a "cult of personality". It's something people have to work around.
If people are risk averse to sharing ideas, then you'll get fewer of them. Perhaps that's a good thing. But it costs very little to learn to provide feedback in a non-confrontational manner.
In my experience at Amazon, there's a lot of free discussion and invitations for criticism of management ideas, but once a decision has been made, everyone is expected to get behind it for at least the next 6-12 months.
I don't think most teams at Amazon engage in "shaming" behavior, as much as they set a standard. Ie, if we agreed that all new modules are going to have 100% or near 100% test coverage and someone turns in a commit that doesn't have any tests at all, they're going to get some shit from the rest of the team.
There's a difference between an implementation that doesn't work as well as hoped that needs to be refactored, and intentionally adding technical debt and going against best practices to save yourself time. Quirky implementations are treated pretty indulgently at Amazon as long as you fix them once it's clear they're a problem, while laziness and causing harm to other developers who have to maintain the codebase is clamped down on pretty hard.
Most people would be in favor of "shaming" an airplane mechanic who doesn't follow the safety checklist after they've been taught how to, but not so if it was their first day.
If I worked for someone who treated me like that and then, decades later, I heard that they'd finally died, I'd say IT'S ABOUT FUCKING TIME! and have a little party.
The worse part is they want to have "followers" themselves as well, subconsciously, repeating the cycle.
I still have many properties of Authoritarian Followerism in myself, but I'm making progress.
One of the difficult parts is many people around me show blatant signs of that followerism and it's easy to get sucked back down with them.
E.g., a coworker protested my protestations against our company stack ranking employees based on "quantitative" metrics, and then many months later backpedaled on his words when someone who had a more important title said the same concerns.
They take the success of some area of the world so personally, and hold themselves to such a high standard, that everything they themselves do is insufficient, everything anybody else does is woefully insufficient, and everything the rest of the world does is evil. When the closest anyone, including the person assigning grades, can get to "good" is merely "doesn't obviously suck right now" it makes sense there would be a lot of shouting.
Michael Jordan had a reputation for being a jerk too, but it's cargo culting to think that's how you get world class performance.
Jordan could do it and be idolized for it because he was the best on the court usually. Microsoft in the 90s was the powerhouse. Working for Apple has (had?) carried a certain panache, and Amazon is a similar powerhouse.
Most people, when they buy a house, go into debt (e.g.: a mortgage) for the initial purchase - without that debt, they would never be able to save up the money needed to buy that house.
A startup seems similar - the first goal of a startup should be to show traction and market fit before running out of runway - and if building up technical debt lets them achieve that more quickly, then that is the right thing to do. This first phase seems to be where the 10x programming would shine.
A company only needs to build something that works. If it works in spite of violating elementary design precepts, shoddy components, bugs, and necessary evil, then it still works. The only thing that matters is the result. Investing resources into optimizing it so that it works better than necessary is merely an option, whereas getting it to work wasn't.
The iron triangle includes cost and schedule in addition to performance. If it would take more money or time than you have to make it perform better, then it just won't, because you don't have that much money or time. A product that frightens the maintainers, but keeps the lights on, is a good result.
Of course, you might optimize yourself into a local maximum if you're weak on strategy. That's why nearly every organization in the world has some kind of separation between "officer/executive" and "enlisted/staff" in the hierarchy. Someone needs to balance the triangle and someone needs to write code and there isn't enough time for the same person to do both.
Technical debt to me is uncessary dependencies, anti patterns etc. The only way to prevent this is a good code review were people pay attention to such things. Ideally the code author and the reviewers have the right thinking: after checking for correctness, they ask themselves "is this code modular/reusable/easily testable/easy to debug/..."
Hopefully the testing is already here, but that's in a perfect situation
My solution is radical, but necessary. 99.999% of code should never be written. Let's start saying no to businesses with arbitrary demands, and let's all agree on a set of standards. Only then will we have affordable and quality software.
Part of what gives software it's unique advantage in today's world is it's ability to change to meet evolving demands quicker than any other way of formulating solutions. As such unclear specifications are somewhat of a pillar of software engineering.
However with unclear and perhaps rapidly changing specifications comes a lot of judgement calls on what to build, how to build and what are reasonable shortcuts to take. Software engineers are almost alone in this, most other professions have the "correct" way and all the ways you can do it wrong. In software there is no "correct" way and just many different ways of kinda achieving something "right".
Interesting. So if you are a freelancer, you will reject 99.999% of would-be customers. Not exactly a winning formula.
Or, if you work for some kind of organization, you tell them, 99.999% of the time, that the business demands are "unreasonable" and that code should not be written. That will get you fired.
Either way, I am not sure where you are going with that narrative.
Problem is identifying those 0.001%
In the real where requirements change constantly, libraries that you rely on rely on left-pad, budgets are short - or hell people just learn to write better code - so the old need to be refactored - the debt accumulates.
Code debt is like entropy - always goes up. The normal state of a software project is under staffer, under budgeted, in constant flux requirement wise and for yesterday.
Doing it right with reviews, refactoring, consistently is a luxury. You can try to have a process that minimizes chaos to minimum, but every project will devolve into unmaintainable mess. You cannot afford to be pure while you have the huge state monad called rest of the world that gives its best to crush you.
Given that and trying to use it as starting point for something more helpful: Sofware is nothing special here. Your things will - depending on their quality - break down with time, especially when you tinker with them and/or use them. You can push this further into the future using maintenance, repairing damaged parts and so on, but at some point you will have repaired so much that you will in essence have a new thing, not the old thing anymore.
The task is to balance your efforts with your use case. An engine which has to work for one formula one race is constructed very different from one which has to work 200 or 300k kilometers. Again, software is nothing special here. If your software has to work for one run "doing it right" may be a luxury. If you need your software for years to come your luxury becomes necessity.
There are places currently where you'd expect an extremely high degree of rigor to be applied to a variety of engineering and technology problems because the potential financial impact of a cluster of interrelated faults would be irreparably financially and reputationally disastrous. But instead you will often find the overwhelming culture is to instead hire legions of employees/contractors ranging from demonstrably incompetent to tacitly complicit in knowingly producing garbage just so long as that garbage can be produced on reliably unrealistic (were any amount of rigor to be applied) timelines, and where the optics of "progress" vastly outweigh the quality of releases. And while none of this would be a spoken or agreed upon culture, the set of incentives in place guarantee that results do not align with the institutional rhetoric.
And as much as it pains me personally to see cultures like that (as a 1x engineer who labors naively over correctness in design and implementation)... for the most part I think the marketplace got this right. If you consider realistically that the percentage of software programmers/developers/engineers/architects who work on something truly "mission critical"/"carrier grade"/"matter of life and death" is a rounding error compared to the number which work on the next Dashboards as a Service, or Fitbits for Dogs, or next-gen tween sexting mobile platform, or just toiling away fruitlessly on shelf-ware in large directionless companies surviving on the carcasses of well-heeled dinosaurs in their "enterprise sales channels", then quite obviously doing things "right" comes along with almost no actual value other than just the ability to claim you did it that way.
If building most software came with anything remotely resembling the risk and liability of building a high-rise apartment building, or a suspension bridge, or the hydraulic controls of a jetliner, then one would likely see much more common practice, conventions, and thorough auditing of software that gets created. Rather than "best practices" being outliers, they'd be the norm, and they'd be much more codified and immutable than simply the most recent blog post about scar tissue earned from a metaphorical software skyscraper collapsing into The Bay one sunny Tuesday afternoon.
As an informal support of the above claim. How many among us have worked on projects which included, as a central pillar of the engineering process and culture, all the following forms of testing... unit, integration, property-based, concolic, end-to-end, chaos, regression, soak, and mutation, and have indefinitely pushed back releases until all those forms of testing could be exercised and validated? Whatever the number I'd wager a year's salary that there are more people who don't even know what most of those things are than the number which have been required to diligently do all of them. Probably by an order of magnitude. If we're talking about mythological 10x unicorns, perhaps we can start with that one?
Fwiw, I work for trading firm, which trades 10+ billion dollars daily and mistakes can be incredibly costly. I would argue the lack of shame associated with admitting mistakes is vital to our level of code quality.
I have yet to meet the developer who does not make mistakes. I have met a few arrogant enough that they'd never admit to them.
[1] http://www.amazon.com/Failure-Not-Option-Mission-Control/dp/...
There's an old improv game where someone suddenly points to you and you have to continue their story without missing a beat and without doing a bunch of other arbitrary things. It's designed to be impossible. But if—or rather when—you fuck up, the rule is that everybody applauds, praises you, and gives you hugs. It's disorienting at first, but then you can feel yourself acquiring entirely new patterns in response to mistakes.
If you want them to get fixed, there can't be any incentive to hide them. Which is where "no shaming" rules come from.
It's critical for people to be able to be open and transparent about their mistakes and the process leading to them without fear of being put through the ringer for it, but that's not mutually exclusive from an environment where the type of appropriate embarrassment/remorse necessary (usually in the form of a thorough postmortem) to truly learn from one's errors is expected.
You can see it when you read NTSB AARs, TSBs, even the regs (FARs) themselves. Even the language is chosen so as to avoid shame.
When blame is ascribed it's explicit and for the purpose of explication of root causes, but, more generally, the tone of the narrative is always, "we've critiqued _the process_ and here's how we see to improve it."
People are only seen as participants in the process and aren't targeted for blame or shame.
The amazing safety statistics of commercial flight speak to the efficacy of this philosophy; software project management would do well to adopt much of these strategies, I'd argue.
Shaming is a poisonous behaviour but so is the belief we're capable of perfection. So also is the idea that we should be tolerant of those who constantly fuck up. Fuck those who can't/won't improve.
If we put the infrastructure and attitudes in place that assumes that we will definitely fuck up, that sub-optimal solutions and people will be introduced, then fucking up becomes less common, communication isn't inhibited by shaming/bulling attitudes and, god forbid, we can raise our individual and collective standards and become more attuned to stink and how best to amortize it.
Everything can be automated, even quality checks.